Fire in the Hole

Well, it seems that Ken Brown's latest paper is causing a bit of a brohaha. What I find most annoying is that people want to censor outright a sponsored (yes, by Microsoft and others) paper instead of allowing sensible people of good judgement to read it and form their own opinions. The Internet is supposed to allow a variety of opinions - not just a mob suppressing opinions they find inexpedient (and that means corporate-sponsored mobs too).

But since I'm not interviewed and I have nothing to do with Linux, I think I'll let Andy T and others wrangle that part of it out. Ken Brown's a DC guy, and clearly knows his politics. And since this is a tech blog, I think I'll stay in technical considerations.

Actually, what distresses me is the vitriol level all round. It's just a paper, for goodness sakes - not a subpoena. One doesn't have to agree with everything in a paper, or even really anything in it, but one can act professionally about it - even "let's invent a new protocol" papers (or, then again, maybe not on that one).

First person accounts should matter most though - preferably at the time the work was done. Andy T wrote a great deal at the time, as did we, so published works from recognized sources of the time like major magazines like Dr. Dobbs Journal, book publishers, and journals and conference proceedings are a pretty good guide to motives and goals, just as Dennis Ritchie and Ken Thompson did earlier (yes, I know this isn't the case for Torvalds). Given all the controversy, especially if there is a "gap" in the records, there is bound to be speculation and supposition, just as I've seen a great deal of nonsense written about 386BSD over the years by complete idiots who didn't even bother (or wish) to ask or read the source material.

So, what did I think of this paper. Well, I decided to completely ignore the political treatise, because frankly, I don't know anything about this subject. If it had something to do with physics or the history of science or anthropology, I'd feel on firm footing. But it doesn't - it's polisci. So I'm going to leave this one to the polsci majors because I'm not a subject matter expert here.

But I am a subject matter expert on one particular subject - 386BSD. So I decided to look at it only with an eye towards my direct experience between the global network storage workshop plus volunteering for the big Silicon Valley Forum for Women Entrepreneurs auction fundraiser.

Here are a few personal observations...

1. Mr. Brown mentions the the Lions book in his paper. It was officially published with the cooperation of Michael Tilson, CIO of SCO in 1995. I have an original review copy, plus the publishers letter in my bookcase which I am referencing. Dan Doernberg, who established Peer-to-Peer Publishers, and publish both the Lions book and the Basic Kernel Source Code Secrets book series, founded with his wife Rachel Computer Literacy Bookstore chain that was later sold to FatBrain (in turn acquired by Barnes and Noble). Dennis (Ritchie) wrote the forward. It was a great accomplishment to finally get this book out, and Dan deserves much of the credit for making this so.

2. According to Mr. Brown's timeline, Linus acquired a 386 PC in January of 1991. Coincidentally, that was the launch issue date of the 17-part feature series Porting Unix to the 386 for Dr. Dobbs Journal entitled "Designing the Software Specification" which contained all the pertinent information on the practice of a port of Berkeley Unix to the 386. By April, we were discussing language tools cross support for Unix ports, when his timeline states Linus started his Linux project. By September, we had already discussed the creation of a Unix kernel in detail and were moving on to multiprogramming and multitasking, when the timeline states Linux v0.01 was released.

All this was due to the support and encouragement of Jon Erickson, Editor-in-Chief of Dr. Dobbs Journal. I doubt without his support very few people would have heard of our work, nor been able to freely obtain the source code for these articles through the magazine.

3. I am pleased that Andy's Minix work is once again front and center, if only to get people to read his books again. I have several of his books in my bookcase. He is a fine writer and teacher. Perhaps this controversy will lead more people to buy his books and study the subject with the seriousness it deserves.

4. Fred Brooks book is quite valuable - even after all these years. I remember his lecture at Berkeley close to 25 years ago on the Mythical Man Month for the CS department. You can see his influence on our design in the Dr. Dobbs Journal essay Research and the Commercial Sector: Where Does BSD Fit In? (June 1991), where he is quoted.

5. According to Mr. Brown, Andy took 3 years to develop the Minix kernel and minimal device support. This seems about right. It took roughly 2 years to create the basic 386BSD kernel - the 386 is such an arcane processor, but William had done ports of Berkeley Unix to the 32000 microprocessor as head of National Semiconductor's Project Mesa team, as well as worked extensively on Berkeley Unix prior, so perhaps that accounts for the year improvement.

Much of the 386 support had to be interpolated from Crawford and Gelsinger's book. That process (1989-1991) was documented in the Porting Unix to the 386 series in Dr. Dobbs Journal, along with source code on the 386 specifics discovered in the course of development of 386BSD. The practice of a port to a new processor architecture uncovers many dilemmas and trade-offs in design and architecture of the OS as well as utilities, compilers, and so forth.

We have gratefully acknowledged that Richard Stallman's GCC C compiler was a valuable tool in the process of development, for example, along with the hard work and effort of many thousands who have worked on Berkeley Unix applications, tools and utilities. Compiler work, for example, like kernel architecture, is a life-long effort and requires dedication. As such, it was not "bug-free", and we were often sending in fixes to RMS during the practice of the port.

6. I wish people would distinguish more carefully between the development of a kernel from that of a full release like 386BSD. Release engineering is a very complex practice, and kernel development in tandem with release enginering (tools, utilities, applications, languages, games - all arranged) is an art as well. It's a lot more than six files - of course, six is about the highest a pigeon can count, so maybe that explains this odd number.

William handled the 2.8BSD and 2.9BSD architectural issues while at Berkeley, as well as Unix issues for USGS and other government agencies, prior to National, handled a software release as well as port for National Semiconductor as part of the porting process to their new processor, and then founded Symmetric Computer Systems, a 32000-based custom-built computer system (with people from NASA, USGS, Berkeley, ...) and Berkeley Unix, so releases were commonplace there.

We recently gave a talk on Symmetric for the Vintage Computer Fair - it is a legendary system, and interest has only increased over the years. Many innovations in 386BSD were based on experiences in Berkeley Unix work with customers learned at Symmetric. That's why the talk was Before 386BSD: The Symmetric 375 and Berkeley Unixand discusses such November of 2003 - with a nice mention in Network World Magazine, Michael Cooney (January 5, 2004).

So there were many years of experience built-in to this process on our end. And it still took easily a year to two years between quality engineered releases that contained novel work and also worked well.

7. Mr. Brown make a number of claims for damages in a lawsuit. I'm not a lawyer, and I wouldn't know anything about such a lawsuit.

But I do understand the author's dilemma well. The costs for plagerism lawsuits generally disincents authors from pursuing claims. Several years ago I was shown a book on Linux kernel design which had entire sections of writing taken whole cloth out of my earlier book on kernel design, discussing work that has never been incorporated into Linux at all, but representing it as part of the Linux kernel in their book.

The person who showed me such was a Chief Architect at Oracle at the time, working to support OPS 8 on Linux, who couldn't understand how this part of the Linux kernel worked, and asked me about it (we worked together on a semiconductor project).

I discussed this with my publisher, but they decided the costs of pursuing a claim was greater than the actual damages. They suggested we send a letter requesting an acknowledgement for quoting our work. This seems the best approach when handling such issues.

8. The "free, non-Unix operating system" has its roots in the earlier Berkeley Unix "egoless" movement (late 1970's to mid-1980's), whereby all attributions were stripped from modules in order to focus the work entirely on research aims - 2.8BSD suffered this fate, with much of William's work ultimately unattributed even though the PDP-11 overlay work was part of his senior project at Berkeley. Attribution was, it was claimed, contrary to the communal work of Berkeley Unix. A practical aspect was that as names were added as works evolved and branched from other code, "attribution-overload" (too many names) developed.

Attribution was added back in as protests developed, but many people were never re-engaged with their work. This practice continued in later Berkeley Unix releases as selective attribution, whereby only the primary developer was considered relevent. But a true window on those who contributed to the works was lost in the process.

Thus, speaking only of history and not politics, Tuomi's "reinvention of history" is very apt and very human.

9. I find that everyone seems to forget history, especially when making a political point which doesn't always support the fashion of the moment. Some of the historical reasons that resulted in Richard Stallman devising the copyleft appears in Copyrights, Copyleft, and Competitive Advantage in "Porting Unix to the 386: Language Tools Cross Support", Dr. Dobbs Journal, January 1991 long before this debate got started.

10. I remember Andrew Binstock - he was editor of Unix Review Magazine when I was working with Dr. Dobbs. Before he left he published a very nice review of my book, naming it as one of Top 10 Books of 1996, which was most kind because no other Unix publication bothered to review it until Linux Journal many years later - which was also a nice review, though quite long after-the-fact. Better late then never.

Dr. Dobbs Journal, by the way, was never considered a Unix journal. It specialized in CP/M and DOS at the time, later Windows, and we wrote specifically for a non-Unix (much larger) audience.

11. Very elaborate "comparison programs" were used in order to separate Berkeley work from AT&T licensed works for the BSD Networking releases. It examined comments as well as structural portions of code. Keith Bostic did much of the work. It did reduce the portions of questionable code significantly. However, code still needed to be hand-examined, and contributors interviewed. Sadly, even UC Berkeley did not have sufficient resources to complete the job in a thorough manner, as the USL/UC lawsuit later underscored.

12. The sources for open source are often company programmers with access to licensed source code - we can't simply blame students. SGI programmers, for example, were reported to have "tossed code over the fence" (according to one programmer there) to Linux in the mid-1990's, in retaliation for the UC Berkeley lawsuit (actions not endorsed nor condoned by UC Berkeley or 386BSD).

Unfortunately, SGI ended up undermining their own Unix business very rapidly, and are now only a marginal player on the high-end in graphics. Why bother purchasing an Irix when you could obtain Unix just like an Irix (sans graphics) on a 386? The business consequences were not well considered, probably simply because the management of SGI was unaware of the activity - it fell "under the radar screen".

13. While the Oracle database example is interesting, it seems to divert the reader from Mr. Brown's central argument. Using an open source operating system to develop a database is not ipso facto an improper act. Using Linux for development of an application like a database does not mean the database is somehow tainted. The mp4 franhofer case is also more complex than first appears. In sum, use of open source doesn't mean the result is tainted anymore than an open source word processor means my words are tainted. It was a bad argument in 1992 to claim 386BSD was tainted because of another's legal battle, and it is a bad argument now.

14. The scenarios section is reminiscent of the paper we presented at the Software Developer's Conference in 1996 entitled Why NT Will Bury Unix - a provocative title that resulted in an overflow crowd and resulted in a follow-on talk the following year ...And Why the Internet Will Bury NT. We presented "Sliders" scenarios (if you recall that science fiction series where a scientist was lost and traveled all sorts of different timelines) about open source. However, we focussed entirely on architectural issues as the primary agents for whether one or another scenario would succeed.

15. Mr. Brown's observations about research and code are important, and work such as this in academia needs to be available to all. However, given the complete lack of interest in research in OS and networking design in academia, it seems unlikely that such a scenario is possible.

I do not work for a university, yet I have still written many articles and papers on operating systems and networking, such as my October 2002 Grace Hopper Conference paper From 386bsd to OSPREY: The Evolution of an Operating System. Does this mean that I could not participate in the continuing research of my own work?

Dr. Dobbs Journal is not an academic publication, yet it has published many key new innovations. Is it possible that they would be precluded from publishing new works?

These and other details can tie up in logical knots the supposition that academic progress is simply one of keeping the works cloistered, or simply using a Berkeley license. And commercial interests themselves cannot be relied upon to fund such work, because it may enable competition and depricate expensive investments in older solutions more rapidly.

The conflicts between research and commercial interests and directions are with us, and will never go away. It seems we need a new approach that is pragmatic and not ideological.

16. Then there is the human factor. When 386BSD was released, we expected some 600 downloads. In one week, there were a quarter of a million, and mirror sites had to be set up to handle the load. Bear in mind, this was before the modern graphical Internet began. So it was a tremendous response. I don't think anyone was prepare for this.

Don Clark of the Wall Street Journal was chatting with William at a talk last month, and reflecting back on those times (for he knew him then and has quoted him in the WSJ) and asked (I'm paraphrasing) "No one took seriously what you were doing in creating an open source operating system - everything done before was a toy operating system like Minix and Xinu, done to teach students. Why didn't anyone take you seriously then?"

William thought about it and basically said "When we crossed over the line, and it became something real on their computer, then there was this incredible push to make it look like something they had before. They didn't want to do it the way they should do it - they wanted it done the way it had been done before."

I saw it a bit differently. I noticed we were loved, and a lot of our admirers wanted to inundate us with more wonderful gifts. Unfortunately, many of those gifts were code that we couldn't use precisely because it belonged to others. And since we spent a great deal of time wading through code we found we couldn't use, it slowed us down, and our suiters became annoyed. And when they demanded to know where their gifts were displayed, and they found out we didn't use it, they became angry. After all, it was a gift, and we should use it, right?

Of course, 386BSD was never involved in any lawsuit, and Dr. Dobbs continued to publish works about 386BSD and Unix, and were never involved in any lawsuits, so the process worked. But the open source community never did see the need for process until, of course, the lawyers get involved. But then gain, they may not, and that is the gamble some are willing to take. I'll do startups, and new technologies, and keep an open mind on new ideas. But I'm not a gambler. :-)

For those who have the foresight, the undiscovered country is truly the future. But that is a rare gift, indeed.