In the DataCenter: A Tale of Two Opinions

Santa Cruz Operation, the company that purchased the rights to Unix from Novell and then launched a series of lawsuits against IBM and high profile users of Linux, has had a somewhat difficult time of it enforcing what they claim is their “rights”, enduring reactions ranging from denial of service attacks from hackers to legal wrangling over just what rights they bought from Novell in the first place.

So it’s no surprise that once again, they are tacking into the wind. But is SCO sailing into calmer legal waters, or is it simply a lull before the storm? Did the “Eldred” case championed by Dr. Lessig of Stanford Law School provide the key to a new approach? Please join me In the DataCenter as I examine SCO’s new direction in A Tale of Two Opinions. [Format: mp4/Unix or QT6+/Mac or Windows].

California, Missions, and Astronomy…

One of the nice things about Silicon Valley is the plethera of colleges and universities who offer all kinds of unusual lectures. Where else but here would we get to hear a talk combining, for example, astronomy, ancient cultures, and the California Missions?

My 4th grade daughter, an amateur astronomer, also did a California missions project this year as mandated for all California elementary students. She did a movie on Mission San Jose, a walking tour through the recently renovated mission describing all of it’s interesting history and features. One viewer said she was the “next Sister Wendy”.

The Dying American Dream and Irrational Joylessness

Mike Cassidy of the Merc wrote a nice essay on the casualties of the dot-com bubble selling out and leaving Silicon Valley. Not all of the people who worked hard here cashed out or got rich — actually, only a few did really well, although most everyone here likes to pretend they did better than everyone else. It’s a peculiar SV conceit.

I’m fourth generation Californian, born in Fremont and went to Berkeley. I’ve always lived in the Bay Area. I remember the orchards, now long gone, and how I used to ride my bike through them coming home from Parkmont Elementary school.

But I don’t resent other folks who came here trying for a bit of the gold. After all, that’s part of the American Dream. Does anyone remember the American Dream anymore?

So it makes me sad that young people have to sell everything and leave, just because so many businesses have gone on a bender about outsourcing. It is “irrational joylessness”, an almost armageddon wish-fulfillment. It is a maxim that a man who thinks he will die tomorrow will somehow make it so.

And all Craig Barrett can say is “life is tough”, as John Paczkowski noted a few days back in his column. What a wonderful guy.

Mike also spoke of experiencing a lack of enthusiasm about google, as John’s column quoted. Sounds like a few people will make out like bandits and it will assuredly be successful given it’s backers, but it won’t save that young couple Mike wrote about yesterday, nor a lot of others who have contributed to the success of the Valley.

What, me worry?

This little article just in from a dedicated Cisco engineer. Looks like Cisco is taking a “broadside” from Broadcom in the “TCP offload” universe.

Of course, notice the weasel words of “selected network streams”. In contrast, at InterProphet we showed a 10x advantage of all network streams on NT at Microsoft in their offices in Redmond in 1998 with a patented design.

So it’s taken Broadcom and Microsoft working together about 6 years to kind of make something work but not really. Not very impressive.

And SiliconTCP works for Unix and other OS’s as well. You don’t have to use Windows to get the benefit. In a world of “choice”, shouldn’t a chip be OS-agnostic? Or do they think choice is a bad idea?

Don’t know how often I read about a TCP offload mechanism which doesn’t really do the job. If a network stack worked like these chips, the Internet would be a lot more frustrating a place.

But I doubt Cisco cares, even though their engineers do. With China’s massive Huawei on the bottom end and ruthless Broadcom moving in, don’t bet on Cisco to defend their turf. After all, they’re too big to beat, right? And if a company lets things go long enough, figuring that “someone else I like better will invent it”, they take the risk someone they don’t like will do it.

Reliable Wireless and Link Layers

It is often the case that a “different” puzzle presents an opportunity for a young scientist to say “Oh, I can solve this problem because it’s different”. Well, sure… but is it simpler, or simply different?

The question begs in a discussion of using reliable link layers in wireless to solve the problem of retransmissions and poor QOS. The problem is that artifact of retransmission distorts the use of the medium, because too many retransmits / congestion events occur, biasing the statistics and becoming unfair. The solution for the wireless approach is to use the same thing that causes artifacts of noise as an architectural solution.

In the wired case, the solution is smaller packet sizes, so that when congestion occurs, congestion recovery impacts fewer events. But the increase in the number of packets than distorts the results of reliable link layer, so you get the same problem, only it’s less obvious, which is why I mention the wireless case first.

In the first case, it’s a first order effect. In the second case, it’s a second order effect dominating the first. But in neither case is it a “different” effect that can be solved independent of the other. And there’s where the delusion lies.

An old physics trick — look at the really lossy case first, and then once you’ve figured out what’s the problem, ask if a similar problem hides in the less lossy case. Given the scope of the Internet, little problems become big fast.

Of course, I got told last year that this wasn’t a problem in wireless. Oh, and we’ll eventually find weapons of mass destruction, too. Are you holding your breath?

Flamethrowers and Memories

Alex Cannera dropped an interesting paper on my desktop discussing congestion control in grid networks. And it’s results confirm what I and others have seen over the years; Vint Cerf seriously saw in 1998 that hop-by-hop reliability preserving end-to-end semantics in the routers was the real key to handling this issue. Vint also is a renowned wine expert, and treated me and William to a wonderful tour of fine wines at the Rubicon Restaurant in San Francisco where we had a memorable discussion on exactly this issue.

Of course, their terms-of-art are different from mine, since we all seem to invent new terms. So their “network of queues” is my bucket brigade mechanism. And their test demo is similar to one at InterProphet we called FlameThrower devised by Senior Hardware Engineer Todd Lawson and Software Engineer Madan Musuvathi to literally flood the other side with packets and see if it falls over.

Todd and Madan built a wirewrap version of SiliconTCP on a DEC PAM card with a NIC wired on (and that’s exciting with 100MHz logic). We demo’d this to Microsoft, venture, and lots of other companies back in Summer 1998. I have the wirewrap on my wall alongside a production board.

But the solution presented in this paper to “back pressure” the plug by disabling TCP congestion control selectively is where we part company. Herbert and Blanc/Primet quite correctly point out some of the barriers to FAST, Highspeed TCP / Scalable TCP, and XCP, but then fall back on the old link layer solution approach (which we diddle in the stack software). If only it were that simple.

Reliable link layer isn’t enough, and Vint (in looking back) clearly knew this. That’s why he saw SiliconTCP as fitting best here. This was the key reason he joined the board of InterProphet so many years ago.

Many people have made reliable link layers. We’ve done it with the boards we have here right now. But no one else made a reliable network and transport layer that spans many hops, maximizing the capacity of the aggregate network. Our boards also do this. So we did demo Vint’s vision in practice.

It’s only now that people are starting to thrash the problem that Vint saw many years ago. But they lack his insight as to the real nature of the problem. It isn’t turning off congestion control — it’s using it effectively.

So long as engineers think the answer is a simple “stack hack” instead of rethinking how to more effectively meet the protocol demands — not new protocols, not turning off the congestion, not cheating by biasing fairness — but really simply doing our job better, we’ll continue to run into this problem.

That’s a lot of video and stills

From an article entitled Dotcoms a hot domain in the SF Chronicle today:
“One of the standouts was Sunnyvale’s Sandisk, the world’s largest maker of flash-memory cards, which are used in digital cameras. During the past year, its revenue doubled to nearly $1.1 billion, and its market value nearly quadrupled.”

That’s a lot of video clips and stills. This is the revenue driver for new PCs and disk drives that’s been picking up in the consumer sector. Gotta put those pics and vids somewhere, right? Or has anyone thought about this yet?

Oh where Oh where did my protocol engine go?

In a walk down memory lane, Craig Partridge and Alex Cannara discussed Craig’s mention of an XCP meeting and Greg Chesson, Alex saying “But, we still have suboptimal network design, insofar as we depend on TCP from the ’80s and a glacial IETF process — all this while now having complete web servers on a chip inside an RJ45 jack! So maybe his ideas for SiProts were something to consider, even if they weren’t right on target?”

For those not in the know, Greg Chesson stepped on a lot of “TOEs” (hee hee) first in the early 1990’s with filing a lot of patents with protocol engines (PEI – backed by HP at the time).

I have a slide from a presentation that I did for Intel back in 1997 explaining why he failed — simply put, preprocessing likely conditions based on heuristics always failed in the general case, with the preprocessor commonly falling behind the processor even though it was put there to speed up the processing — so the software stack on average was usually faster.

This same process in FEPs has been repeatedly repatented in network processors — I reviewed several — but they never got the methods that allow for completion of the processing without falling behind (esp. on checksum, but there are also other conditions). I always thought Greg could sue a number of network processor companies for infringement, but since they all fail in the same way, who the hell cares.

Greg made his money in SGI, by the way, and look how that company eventually turned out — lots of “throw code over the fence” to linux, which undermined their own sales of systems. Very self-destructive company.

Your paper is silly, smells, and is ugly, ugly…

Jonathan M. Smith has an interesting idea on how to avoid blackballing in tech paper reviews.

For those not clued in (or fortunate enough to have avoided academic paper submission follies), in order to have an academic paper accepted, one must submit to double-blind review by anonymous experts in the field to evaluate whether a paper is interesting and appropriate to the conference venue without being dazzled (or tainted) by knowledge of who actually wrote it.

While in theory this approach seems quite reasonable, in practice one tends to find that papers which push the envelope, contain ideas not within the accepted compact, or even radically new treatise often meet with less-than, shall we say, open-minded and even-handed analysis?

And since it’s pretty easy to guess who’s paper it is anyways, or even find out using a google search on the keywords, which everyone does anyway to figure out if “someone else wrote something like this before, so I can use their results in my analysis”, the “double” in double-blind doesn’t really work.

So Mr. Miller has proposed (at SIGCOMM in the OO session) a simple process: 1) That all reviews be public, and 2) signed by the reviewer. According to Mr. Miller, “That way history gets to see who was right – and who was wrong.”

Sounds good to me – I’m willing to take on the judgement of history in my work, since that’s only rational. Any other takers?

Excuse my Bios, but…

Check out the Phoenix Technologies announcement on their BIOS hardware authentication scheme.

People like Bruce Schneier lecture us all on how hard it is to create a trusted network verification model that holds up under a variety of conditions and needs. And who needs this (very few)?

Some of you may recall how Microsoft had to pull back on their extra evil hardware authentication when it turned out XP didn’t work when you added a device like a DVD. Everybody complained, vendors got really annoyed with customer support, and Microsoft got blamed (well, it was their fault). They’re trying to moderate this in practice, but it’s not a deterministic strategy for a host of technical and practical reasons.

But Microsoft has an OS monopoly and can get away with failure. Does Phoenix have a monopoly on the BIOS? Don’t think so. The customer hates this process of verification – it makes him feel like a criminal. So the vendor notices the customer hates the product, buys another BIOS from AMI or any of the hundreds of others, gets rid of this nonsense for the customer, and the problem is self-correcting.

And now Phoenix is going to disallow network access from people who don’t match this same faulty hardware profile? Great – it didn’t work before, so let’s make it even bigger this time. Sounds like lunacy? It is. Phoenix has been trying this trick for years and years – and it won’t work unless you keep people from upgrading / fixing their PCs.

Can you imagine how people would react if you told them they couldn’t change the oil filter on their cars, or take it to the local oil changers and buy a standard filter? You’d have to throw the car away – sorry Charlie.

As the real security guys will tell you, verification of trusted networked sources is actually a very difficult game, even using hardware and secure links. You fool yourself into believing that you can live in a black and white universe. In reality, with security this brings more problems, because too narrow a window results, so exceptions are made to the rules and soon you’re back to the same corner case issues as before. The real success comes down to knowing the character of the individuals and the use / practice of measures, such as the friend/foe rules of engagement process used in the military.

The last thing everyone needs is an Internet where people are refused access for arbitrary reasons, subverting the entire point of TCP/IP and networked communications – to exchange information – by preying on people’s fears of the “bad guys” gaining access. Maybe if Microsoft would just secure their OS better, we wouldn’t have this fear in the first place.

Now that I’ve pointed out the problem from the perspective of an Internet expert, we could all use a crafty rebuttal from a security expert as to why this is just another brand of snake oil. Hey Bruce, where are you?