What, me worry?

This little article just in from a dedicated Cisco engineer. Looks like Cisco is taking a “broadside” from Broadcom in the “TCP offload” universe.

Of course, notice the weasel words of “selected network streams”. In contrast, at InterProphet we showed a 10x advantage of all network streams on NT at Microsoft in their offices in Redmond in 1998 with a patented design.

So it’s taken Broadcom and Microsoft working together about 6 years to kind of make something work but not really. Not very impressive.

And SiliconTCP works for Unix and other OS’s as well. You don’t have to use Windows to get the benefit. In a world of “choice”, shouldn’t a chip be OS-agnostic? Or do they think choice is a bad idea?

Don’t know how often I read about a TCP offload mechanism which doesn’t really do the job. If a network stack worked like these chips, the Internet would be a lot more frustrating a place.

But I doubt Cisco cares, even though their engineers do. With China’s massive Huawei on the bottom end and ruthless Broadcom moving in, don’t bet on Cisco to defend their turf. After all, they’re too big to beat, right? And if a company lets things go long enough, figuring that “someone else I like better will invent it”, they take the risk someone they don’t like will do it.

Reliable Wireless and Link Layers

It is often the case that a “different” puzzle presents an opportunity for a young scientist to say “Oh, I can solve this problem because it’s different”. Well, sure… but is it simpler, or simply different?

The question begs in a discussion of using reliable link layers in wireless to solve the problem of retransmissions and poor QOS. The problem is that artifact of retransmission distorts the use of the medium, because too many retransmits / congestion events occur, biasing the statistics and becoming unfair. The solution for the wireless approach is to use the same thing that causes artifacts of noise as an architectural solution.

In the wired case, the solution is smaller packet sizes, so that when congestion occurs, congestion recovery impacts fewer events. But the increase in the number of packets than distorts the results of reliable link layer, so you get the same problem, only it’s less obvious, which is why I mention the wireless case first.

In the first case, it’s a first order effect. In the second case, it’s a second order effect dominating the first. But in neither case is it a “different” effect that can be solved independent of the other. And there’s where the delusion lies.

An old physics trick — look at the really lossy case first, and then once you’ve figured out what’s the problem, ask if a similar problem hides in the less lossy case. Given the scope of the Internet, little problems become big fast.

Of course, I got told last year that this wasn’t a problem in wireless. Oh, and we’ll eventually find weapons of mass destruction, too. Are you holding your breath?

Flamethrowers and Memories

Alex Cannera dropped an interesting paper on my desktop discussing congestion control in grid networks. And it’s results confirm what I and others have seen over the years; Vint Cerf seriously saw in 1998 that hop-by-hop reliability preserving end-to-end semantics in the routers was the real key to handling this issue. Vint also is a renowned wine expert, and treated me and William to a wonderful tour of fine wines at the Rubicon Restaurant in San Francisco where we had a memorable discussion on exactly this issue.

Of course, their terms-of-art are different from mine, since we all seem to invent new terms. So their “network of queues” is my bucket brigade mechanism. And their test demo is similar to one at InterProphet we called FlameThrower devised by Senior Hardware Engineer Todd Lawson and Software Engineer Madan Musuvathi to literally flood the other side with packets and see if it falls over.

Todd and Madan built a wirewrap version of SiliconTCP on a DEC PAM card with a NIC wired on (and that’s exciting with 100MHz logic). We demo’d this to Microsoft, venture, and lots of other companies back in Summer 1998. I have the wirewrap on my wall alongside a production board.

But the solution presented in this paper to “back pressure” the plug by disabling TCP congestion control selectively is where we part company. Herbert and Blanc/Primet quite correctly point out some of the barriers to FAST, Highspeed TCP / Scalable TCP, and XCP, but then fall back on the old link layer solution approach (which we diddle in the stack software). If only it were that simple.

Reliable link layer isn’t enough, and Vint (in looking back) clearly knew this. That’s why he saw SiliconTCP as fitting best here. This was the key reason he joined the board of InterProphet so many years ago.

Many people have made reliable link layers. We’ve done it with the boards we have here right now. But no one else made a reliable network and transport layer that spans many hops, maximizing the capacity of the aggregate network. Our boards also do this. So we did demo Vint’s vision in practice.

It’s only now that people are starting to thrash the problem that Vint saw many years ago. But they lack his insight as to the real nature of the problem. It isn’t turning off congestion control — it’s using it effectively.

So long as engineers think the answer is a simple “stack hack” instead of rethinking how to more effectively meet the protocol demands — not new protocols, not turning off the congestion, not cheating by biasing fairness — but really simply doing our job better, we’ll continue to run into this problem.

That’s a lot of video and stills

From an article entitled Dotcoms a hot domain in the SF Chronicle today:
“One of the standouts was Sunnyvale’s Sandisk, the world’s largest maker of flash-memory cards, which are used in digital cameras. During the past year, its revenue doubled to nearly $1.1 billion, and its market value nearly quadrupled.”

That’s a lot of video clips and stills. This is the revenue driver for new PCs and disk drives that’s been picking up in the consumer sector. Gotta put those pics and vids somewhere, right? Or has anyone thought about this yet?

Oh where Oh where did my protocol engine go?

In a walk down memory lane, Craig Partridge and Alex Cannara discussed Craig’s mention of an XCP meeting and Greg Chesson, Alex saying “But, we still have suboptimal network design, insofar as we depend on TCP from the ’80s and a glacial IETF process — all this while now having complete web servers on a chip inside an RJ45 jack! So maybe his ideas for SiProts were something to consider, even if they weren’t right on target?”

For those not in the know, Greg Chesson stepped on a lot of “TOEs” (hee hee) first in the early 1990’s with filing a lot of patents with protocol engines (PEI – backed by HP at the time).

I have a slide from a presentation that I did for Intel back in 1997 explaining why he failed — simply put, preprocessing likely conditions based on heuristics always failed in the general case, with the preprocessor commonly falling behind the processor even though it was put there to speed up the processing — so the software stack on average was usually faster.

This same process in FEPs has been repeatedly repatented in network processors — I reviewed several — but they never got the methods that allow for completion of the processing without falling behind (esp. on checksum, but there are also other conditions). I always thought Greg could sue a number of network processor companies for infringement, but since they all fail in the same way, who the hell cares.

Greg made his money in SGI, by the way, and look how that company eventually turned out — lots of “throw code over the fence” to linux, which undermined their own sales of systems. Very self-destructive company.

Your paper is silly, smells, and is ugly, ugly…

Jonathan M. Smith has an interesting idea on how to avoid blackballing in tech paper reviews.

For those not clued in (or fortunate enough to have avoided academic paper submission follies), in order to have an academic paper accepted, one must submit to double-blind review by anonymous experts in the field to evaluate whether a paper is interesting and appropriate to the conference venue without being dazzled (or tainted) by knowledge of who actually wrote it.

While in theory this approach seems quite reasonable, in practice one tends to find that papers which push the envelope, contain ideas not within the accepted compact, or even radically new treatise often meet with less-than, shall we say, open-minded and even-handed analysis?

And since it’s pretty easy to guess who’s paper it is anyways, or even find out using a google search on the keywords, which everyone does anyway to figure out if “someone else wrote something like this before, so I can use their results in my analysis”, the “double” in double-blind doesn’t really work.

So Mr. Miller has proposed (at SIGCOMM in the OO session) a simple process: 1) That all reviews be public, and 2) signed by the reviewer. According to Mr. Miller, “That way history gets to see who was right – and who was wrong.”

Sounds good to me – I’m willing to take on the judgement of history in my work, since that’s only rational. Any other takers?

Excuse my Bios, but…

Check out the Phoenix Technologies announcement on their BIOS hardware authentication scheme.

People like Bruce Schneier lecture us all on how hard it is to create a trusted network verification model that holds up under a variety of conditions and needs. And who needs this (very few)?

Some of you may recall how Microsoft had to pull back on their extra evil hardware authentication when it turned out XP didn’t work when you added a device like a DVD. Everybody complained, vendors got really annoyed with customer support, and Microsoft got blamed (well, it was their fault). They’re trying to moderate this in practice, but it’s not a deterministic strategy for a host of technical and practical reasons.

But Microsoft has an OS monopoly and can get away with failure. Does Phoenix have a monopoly on the BIOS? Don’t think so. The customer hates this process of verification – it makes him feel like a criminal. So the vendor notices the customer hates the product, buys another BIOS from AMI or any of the hundreds of others, gets rid of this nonsense for the customer, and the problem is self-correcting.

And now Phoenix is going to disallow network access from people who don’t match this same faulty hardware profile? Great – it didn’t work before, so let’s make it even bigger this time. Sounds like lunacy? It is. Phoenix has been trying this trick for years and years – and it won’t work unless you keep people from upgrading / fixing their PCs.

Can you imagine how people would react if you told them they couldn’t change the oil filter on their cars, or take it to the local oil changers and buy a standard filter? You’d have to throw the car away – sorry Charlie.

As the real security guys will tell you, verification of trusted networked sources is actually a very difficult game, even using hardware and secure links. You fool yourself into believing that you can live in a black and white universe. In reality, with security this brings more problems, because too narrow a window results, so exceptions are made to the rules and soon you’re back to the same corner case issues as before. The real success comes down to knowing the character of the individuals and the use / practice of measures, such as the friend/foe rules of engagement process used in the military.

The last thing everyone needs is an Internet where people are refused access for arbitrary reasons, subverting the entire point of TCP/IP and networked communications – to exchange information – by preying on people’s fears of the “bad guys” gaining access. Maybe if Microsoft would just secure their OS better, we wouldn’t have this fear in the first place.

Now that I’ve pointed out the problem from the perspective of an Internet expert, we could all use a crafty rebuttal from a security expert as to why this is just another brand of snake oil. Hey Bruce, where are you?

Sometimes a Legend

And once again, an interesting item in the postel.org end-to-end group – “An interesting version of TCP was created a few years ago at a large data-storage-system company here — essentially, the TCP receive window was reduced to Go/No-Go, and startup was modified, so the sending Unix box would blast to its mirror at full wire rate from the get go. ACKs would have meaningless Window values, excepting 0, because sender and receiver had similar processing/buffering capability. Loss produced replacement, via repeated ACKs. Being LAN-based system overall made all these mods workable. But clearly, the engineers involved found normal TCP wanting in the ability to deliver data on high-speed links.”

Interesting how legends develop. This project was called the flamethrower demo done with a wirewrap version of SiliconTCP on a DEC PAM card with a NIC wired on (and that’s exciting with 100MHz logic).

We demo’d this to Microsoft, venture, and lots of other companies back in Summer 1998. One Microsoft exec (Peter Ford) noted that we were so overloading the standard NICs that an “etherlock” condition was likely to occur. Etherlock, for those who don’t know, occurs when all of the bandwidth is consumed and nothing else can communicate because there is no idle time effectively. And yes, we saw this occur.

One of the more interesting things we found is that many “standard” NICs were not standard compliant. I still have the wirewrap on my wall alongside a production board.

The Power of TCP is in its Completeness

Interesting line of discussion passed through my email regarding the future of TCP. In particular, Alex Cannara decided to take on a few of the more “conservative” elements on dealing with end-end flows by interior management of links.

As Alex puts it: “Apparently, a great bias has existed against this sort of design, which is actually very successful in other contexts”. Even a very “big old name in Internet Land” liked this type of approach, for the “…reason it [TCP] requires the opposite of backoff is because it doesn’t have the visibility to determine which algorithm to choose differently as it navigates the network at any point in time. But if you can do it hop by hop you can make these rules work in all places and vary the algorithm knowing your working on a deterministic small segment instead of the big wide Internet.”

Let’s take this further.

In math we deal with continuous functions differently than discontinous ones, and TCP algorithms know this – they have different strategies for each approach – but when you get a mixture across the network you’re limited to statistics. If we limit the inhomogeneity, then the end points of TCP can then optimize the remaining result. In this case, the gross aspects limiting the performance no longer dominate the equation.

So you can’t overtransmit or overcommit a link if you’re disciplined – you only fill in the idealized link of the puzzle from the perspective of what you know.

Has the hobgoblin of statistics ruined any ability to do a deterministic job (with metrics and cost value) of improving loss ratios and understanding what is really happening at any point along the way? If so, this would in turn validate / prove a statistical model. But think of all the projects that wouldn’t fly.

At InterProphet we proposed that for every hop we get the best possible effect –basically the same level of end-to-end principle in each segment instead of viewing all hops as one end-to-end segment — by deploying low latency TCP processing as a bucket brigade throughout the infrastructure. Now, the pushback from the manufacturers was cost, but we met all cost constraints with our dataflow design (that works, by the way, and is proven).

The power of this approach is amazing. Instead of simplistically thinking of end-to-end as just two cans and a string, we can apply end-to-end completeness on every segment.

Very few people have understood this — looks like Alex does. And I know Vint Cerf, the Father of the Internet, does. He joined InterProphet’s Board of Directors on the strength of the idea alone. Of course, he’s also a visionary and gentleman in every sense of the word. We should all be so gifted.