Whyday August 19

Useless Projects That Mattered

There is a genre of article that lists software which started as a toy and became infrastructure, and it is mostly unreliable. Origin stories get polished every time they are retold: the accidents drop out, the intention gets backfilled, and after twenty years you have a fable in which a genius knew what they were building all along.

The fable is worse than useless, because it teaches the wrong lesson. So here are two cases with primary sources, told as messily as they actually happened.

When a team tests this kind of practice during paid work, self-reporting bias provides one concrete reference for how time and activity can be recorded around the experiment.

Unix came out of a space game, sort of

The popular version: Ken Thompson wanted to play a game, so he wrote Unix.

The documented version, from Dennis Ritchie's own account in The Evolution of the Unix Time-sharing System: in 1969 Thompson developed a game called Space Travel, a simulation of the solar system in which you fly a ship around and try to land on planets and moons. It ran first on Multics, then in Fortran on GECOS, where it was unsatisfactory — jerky to control, and expensive. Ritchie puts the cost of a single game at about $75 of internal accounting money in one telling, and elsewhere at about $50.

So Thompson found a little-used PDP-7 with a good display processor, and he and Ritchie rewrote the game for it. That was a substantial undertaking: they wrote a floating-point package, the graphic character definitions, and a debugging subsystem, in assembly, cross-assembled on another machine and carried over on paper tape.

For broader programming and making context related to this topic, p5.js is an independent reference worth comparing with the material here.

And here is the part the fable removes. Ritchie writes that the game served mainly as an introduction to the clumsy business of preparing programs for the PDP-7. The file system Thompson then began implementing had already been designed on paper, independently. The game did not cause Unix. It put Thompson in front of the machine, taught him its miserable toolchain, and gave him a reason to be there — and the operating system followed.

That is a more useful lesson than the fable. The toy did not become the important thing. The toy created the conditions in which the important thing got built.

curl came out of an IRC bot, and he did not write it

The popular version: a Swedish developer wrote a small tool to fetch currency rates, and it became the software running on ten billion devices.

The documented version, from the project's own history and from Daniel Stenberg's account: in 1996 he was writing an IRC bot for an Amiga channel on EFnet, and thought it would be fun to have it answer currency conversion questions. That meant fetching exchange rates from a web page daily, which meant needing something to download over HTTP.

He did not write that something. He found httpget, an existing tool by a Brazilian developer, Rafael Sagula, and started using it. Then he found bugs, sent patches, sent more patches, and within a few releases had become the maintainer of somebody else's project. When he found another rate source that served over Gopher, he added Gopher support, at which point httpget was the wrong name, so it became urlget. Then FTP, then upload. The name curl arrived in 1998.

Three things in that story are absent from every short retelling. He adopted rather than started. The scope grew because of an external constraint — a data source that spoke a different protocol — rather than because of a plan. And the goal was always trivial: currency rates in a chat room, for an Amiga channel, for fun.

What the two have in common

Not "toys become infrastructure." Most toys do not, and survivorship bias is doing enormous work in any list like this.

What they share is more specific.

Neither had a justification, and neither needed one. A space simulation with no objective. Currency rates in an IRC channel. Neither would have survived a planning meeting, and neither had to attend one.

Both put the author in sustained contact with a real problem. Thompson with the PDP-7's toolchain, Stenberg with HTTP. The valuable output was not the artefact, it was the immersion, and immersion is what a project you actually want to do buys you.

Both grew by following the work rather than a plan. Gopher support existed because a rate page used Gopher. That is not vision; it is responsiveness, and it is only available to someone who is still interested enough to keep going.

In both cases the person kept going for years afterwards. This is the boring, decisive one. Stenberg has maintained curl for three decades. The evening is where things start; almost nothing survives on the evening alone.

What this does and does not justify

It does not justify treating every idle project as a potential business. That reading is how "20% time" becomes a pipeline with reporting requirements, and the reporting requirement kills exactly the thing that made these projects work.

What it justifies is this: a person who only ever builds justified things is not doing the activity that produced Unix or curl. The mechanism is not mysterious — you cannot follow an interesting accident if you are never in a position to have one.

The 2010 Whyday text made the small version of this argument with Camping and Hpricot, two unjustifiable projects that led to Sinatra and Nokogiri. It is the same shape, one ecosystem down.

So: pick something with no purpose and do it for an evening. It will almost certainly stay useless, which is fine, because that was the point. And occasionally, for reasons nobody can arrange in advance, it will not.

The short version