Category: 386BSD & Unix

The successful low-earth orbit of a Dragon capsule mock-up by the Falcon 9 rocket was a great achievement by SpaceX last week (June 4, 2010) and a harbinger of the new age of private space transport. As I watched their success, the excitement from the press and space enthusiasts, and the unexpectedly vindictive response from many inside NASA, I was reminded of the launch of 386BSD – and why those most able to understand your achievement often are the most parochial.

Space exploration is a family tradition for the Jolitz clan, starting with William L. Jolitz developing transponders and thin and thick films for many spacecraft at Ford Aerospace (some still transmitting telemetry long after his passing) to his son William’s work at NASA-Ames on an oscillating secondary mirror for the Kuiper Airborne Observatory as a high school intern, to his three grandchildren working at NASA in astrobiology (Rebecca Jolitz), orbital dynamics and space fatigue simulations (Ben Jolitz) and spacecraft logistics for science projects (Sarah Jolitz).

Ben and Rebecca Jolitz also had the opportunity to meet Elon Musk, founder of SpaceX, at the 2007 Mars Society Conference held at UCLA and were inspired by his vision and determination (it was at that conference that Ben decided he wanted to major in physics at UCLA). Though still in high school, they had recieved honorable mention in the Toshiba ExploraVision competition for their highly creative concept of a Mars Colonization Vehicle based on using an asteroid in a controlled bielliptic orbit as a transport vehicle to provide the “heavy lifting” of supplies and personnel between Earth and Mars. They were invited to present a more detailed talk on their project for the Independent Study track. The speakers at this conference were a great spur to their scientific enthusiasm.

So it was with great satisfaction that I watched SpaceX demonstrate that “rocket science” is no longer the province of great nations but instead will bring about a democratization of space – cargo and transport, experimentation and eventually mining and exploration.

This is no quick path, however. The struggle for open source software, beginning with Richard Stallman and his remarkable GCC compiler, Andy Tanenbaum’s Minix system, Lyon’s careful documentation of version 6 Unix, our Dr. Dobbs Journal’s article series on 386BSD Berkeley Unix and subsequent releases and Linus Torvald’s amazing synthesis of the prior Unix, 386BSD and Minix works to achieve Linux occurred during a enormous burst of creativity that actually totaled about five years (1989-1994). After this came the long process of usability design – driver support, GUI support, applications support, new scripting languages – which is still a work in progress after another decade. Big vision projects take a lot of time and are not for the timid.

It is no secret that NASA has been struggling for many years with a lack of purpose. Just like Unix in the mid-1980’s and Windows in the mid-1990’s, technology which is held too tightly to a single group or company or national agency tends to calcify. Innovation becomes too risky. Agendas and interest groups override design decisions which may theoretically impact their funding. It becomes easier to add to a design than subtract from it, resulting in an unwieldy project which never converges in form or function.

Eventually, more effort in put into maintaining the flaws than eliminating them. Bugs and unexpected interactions begin to dominate, resulting in more meetings, workshops and conferences. Tools to manage the side-effects and flaws of the project become the object of research, while the actual project suffocates as it becomes more and more obese.

The life cycle of an operating system, like the life cycle of a space exploration vehicle, encompasses a brief burst of risk-taking and innovation followed by a long series of “rational” decisions which add heft and gravitas, followed by bloat, loss of purpose and final collapse. But during the long period of bloat and dementia, the lack of satisfactory execution provides an opportunity for newer faster designs leveraging new technologies in other fields to pry into previously unobtainable market niches and slowly eat out the old markets. This happened with open source, and it is happening with space exploration.

The shuttle itself is over 35 years old and encompasses aging technology which can no longer be retrofit – and has been long scheduled for decommission. This schedule has been put off again and again for two reasons: 1) the US has refused to properly fund and schedule a replacement because the costs and commitment are very great and 2) the maintenance and rocket groups are based in key states dependent on continual funding. Politics as usual has been to fund existing projects when we are long overdue to redefine NASA’s mission and goals. And, as is often the case, in refusing to examine other options, we have been left with only one option – end the shuttle program and depend on other nations and consortiums for transport – Russia and the European Space Agency primarily.

The Bush-era Constellation program was in theory supposed to provide an alternative, but the results of this program were laughable – it became a symbol of a bloated self-referential insatiable rocket bureaucracy that couldn’t build a real rocket to get pizza, much less get to the moon. And there was the tragic side to this – so many Americans love to complain about their government by saying “If we could get a man to the moon, why can’t we do” whatever, when in reality America lost that ability over 25 years ago with the decommission of the Saturn 5 rockets. Since then, like many other government makework projects, “rocket science” has devolved to fantasy powerpoint presentations and one-off prototypes that might have been flown, except for the risk of failure.

So while the science side of NASA, with their unmanned probes and experiments and space telescopes, has continually advanced despite the occasional loss, the rocket side has cowered, fearful of failure yet addicted to the status quo of “no risk = no failures”. And this stance, while appearing to play it safe, has created more opportunities for the SpaceX’s of the world as space transport, satellite maintenance and other niche markets look for more effective and less expensive approaches.

Competition, we are always told, is good for America. After all, it was competition with the old Soviet Union that launched the space program – and the need to hire rocket scientists to get us up there. So in principle NASA’s rocket guys should be pleased with SpaceX – they can leverage SpaceX’s experience while encouraging their own demoralized workforce to become more innovative. Like open source, the knowledge that “it can be done” should provide both a relief to fear and a spur to greatness. It’s a win-win, right?

So why the malice and anger? Why did so many within the agency that could most benefit from this knowledge wish SpaceX ill? Why are they running down their achievement? Why aren’t they rising to the challenge? Aren’t they eager to break out of their repressive paradigms?

While envy and fear of change play a great role here, the loss of status is most pernicious. During the rise of open source, a new set of designers and developers began to set the pace for innovation. Many programmers frustrated in their work in industry found an outlet in open source. An avalanche of ideas – good, bad and indifferent – could no longer be repressed by groups controlling proprietary operating systems source. These groups – corporations, standards committees, technology “gurus” – derived much benefit from the old system. They were the leaders at conferences, the movers and shakers of agendas. More than even money, they had the power to elevate or destroy ideas and people on a whim. And believe me, I saw what happened when people didn’t “get with the program”. It wasn’t pretty.

When 386BSD was born, I was told by many in the hard-core Unix side that it would be “strangled in the cradle” – either by lawsuits (which of course, never happened) or by ridicule (which did occur, constantly). I didn’t believe it. I just couldn’t believe that the experts I knew in the biz would wish it ill when they had an opportunity to finally work with BSD without all the proprietary license rigamarole. For years I had heard people complain about all the agreements and licenses and restrictions and “If only it were unencumbered”. Now that they had their wish, wasn’t it great?

Boy, was I misled. What I saw as an opportunity, many other good talented people saw as a threat to their comfortable professional existence. I understand comfort, and I never wanted to make anyone unhappy. But in giving them what they had wished for, I did make them unhappy, because I also gave it to everybody else – and that was inconvenient. Well, I plead youthful enthusiasm here to misunderstanding their desires. But if given the chance, I’d do it again, because it was the right thing to do – even if I did it the “inconvenient way”.

So what were the claims? I was told nobody would use open source because it didn’t have a big company behind it – and we see today that was wrong. I was told that nobody would make money off of open source – and today we see many companies developing profitable businesses off of support and new design. I was told that nobody would use open source to innovate, and yet I use entirely new applications and languages that were not even thought of at the time Dr. Dobbs Journal launched the “Porting Unix to the 386” series in January of 1991. I was told that the only way to distribute software was by selling it on a disk, and that we were crazy to put it out on the Internet, and yet now this is the way even proprietary software is distributed. When I talked about Internet-based OS’s, I was literally laughed at by experts I respected – and it hurt – but now we see the beginnings of the “webOS”.

The ridicule did have real and lasting effects. The constant intimations by Unix groups of pending lawsuits that never arrived but always “loomed”, the personal strain caused by creating entire OS releases on a shoe-string budget funded mostly by writing articles and refinancing our house while raising three young children, the ever-escalating expectations of a consumer audience demanding a commercial OS with all the bells and whistles dissatisfied by traditional Berkeley Unix research releases (with the traditional demands of self-administration – in a “damned if you do, damned if you don’t” moment I actually insisted the Dr. Dobbs OS release installation and administration be automated by default, with the traditional installation process selectable if desired, and was then ridiculed for not making them do it the hard way – sigh) and finally, the relentless badmouthing of any new approaches in the kernel – the raison d’etre of Berkeley Unix but not, admittedly, of a commercial corporate proprietary system. The last of these was the hardest to bear, frankly – and I understood why many other designers, seeing this, fled to Linux. After all, the ridicule, badmouthing and blacklisting was a piece of what they had experienced in their companies, so why endure it in a supposedly “open source” project?

So like 386BSD, the NASA badmouths and their corporate masters could potentially destroy SpaceX. Yes, SpaceX is better funded than a two-person project like 386BSD – our original “Falcon 9” rocket was a 300-400 kbyte kernel plus some apps (386BSD Release 0.0) and 17 5,000-10,000 word articles plus code on how to do it yourself – but getting out of a gravity well of Earth, not to mention the psychological gravity well of believing you can do it (which seems to be more like Jupiter in terms of magnitude) is a heck of a lot harder. Ridicule, the inevitable technical setbacks SpaceX potentially faces, liability laments (ah, there’s that “lawsuits pending” stuff again), a steep learning curve, American impatience (doing new releases with some new innovative work in the kernel took us about 8-12 months – doing the next stage in rocket / capsule design will take longer) and media disillusion when the audience fades (no audience = no money) add to the burden.

But even if, somehow, SpaceX is marginalized, their accomplishment are *real* and will spur others to try. Linux was able to grow and thrive during this time precisely because it was *not* an American project – based in Finland, the canards thrown at 386BSD were deemed irrelevant to Linux. Linux was a safe haven to many serious programmers disillusioned with the threats, lies and distortions promolgated around Berkeley Unix precisely because it was an outsider, uninfluenced by other interests.

People of ill will can kill an innovative project for a while. But they can’t kill the idea on which that project is based. It may be delayed for a while. But somewhere, somehow, it will spur others on to try. SpaceX, like 386BSD, is only the beginning.