The Power of TCP is in its Completeness

Interesting line of discussion passed through my email regarding the future of TCP. In particular, Alex Cannara decided to take on a few of the more "conservative" elements on dealing with end-end flows by interior management of links.

As Alex puts it: "Apparently, a great bias has existed against this sort of design, which is actually very successful in other contexts". Even a very "big old name in Internet Land" liked this type of approach, for the "...reason it [TCP] requires the opposite of backoff is because it doesn't have the visibility to determine which algorithm to choose differently as it navigates the network at any point in time. But if you can do it hop by hop you can make these rules work in all places and vary the algorithm knowing your working on a deterministic small segment instead of the big wide Internet."

Let's take this further.

In math we deal with continuous functions differently than discontinous ones, and TCP algorithms know this - they have different strategies for each approach - but when you get a mixture across the network you're limited to statistics. If we limit the inhomogeneity, then the end points of TCP can then optimize the remaining result. In this case, the gross aspects limiting the performance no longer dominate the equation.

So you can't overtransmit or overcommit a link if you're disciplined - you only fill in the idealized link of the puzzle from the perspective of what you know.

Has the hobgoblin of statistics ruined any ability to do a deterministic job (with metrics and cost value) of improving loss ratios and understanding what is really happening at any point along the way? If so, this would in turn validate / prove a statistical model. But think of all the projects that wouldn't fly.

At InterProphet we proposed that for every hop we get the best possible effect - basically the same level of end-to-end principle in each segment instead of viewing all hops as one end-to-end segment - by deploying low latency TCP processing as a bucket brigade throughout the infrastructure. Now, the pushback from the manufacturers was cost, but we met all cost constraints with our dataflow design (that works, by the way, and is proven).

The power of this approach is amazing. Instead of simplistically thinking of end-to-end as just two cans and a string, we can apply end-to-end completeness on *every* segment.

Very few people have understood this - looks like Alex does. And I know Vint Cerf, the Father of the Internet, does. He joined InterProphet's Board of Directors on the strength of the idea alone. Of course, he's also a visionary and gentleman in every sense of the word. We should all be so gifted.

Sometimes a Legend

And once again, an interesting item in the postel.org end-to-end group - "An interesting version of TCP was created a few years ago at a large data-storage-system company here -- essentially, the TCP receive window was reduced to Go/No-Go, and startup was modified, so the sending Unix box would blast to its mirror at full wire rate from the get go. ACKs would have meaningless Window values, excepting 0, because sender and receiver had similar processing/buffering capability. Loss produced replacement, via repeated ACKs. Being LAN-based system overall made all these mods workable. But clearly, the engineers involved found normal TCP wanting in the ability to deliver data on high-speed links."

Interesting how legends develop. This project was called the "flamethrower" demo done with a wirewrap version of SiliconTCP on a DEC PAM card with a NIC wired on (and that's exciting with 100MHz logic).

We demo'd this to Microsoft, venture, and lots of other companies back in Summer 1998. One Microsoft exec (Peter Ford) noted that we were so overloading the standard NICs that an "etherlock" condition was likely to occur. Etherlock, for those who don't know, occurs when all of the bandwidth is consumed and nothing else can communicate because there is no idle time effectively. And yes, we saw this occur.

One of the more interesting things we found is that many "standard" NICs were not standard compliant. I still have the wirewrap on my wall alongside a production board.

Flamethrowers and Memories

Alex Cannera dropped an interesting paper on my desktop discussing congestion control in grid networks. And it's results confirm what I and others have seen over the years - Vint Cerf seriously saw in 1998 that hop-by-hop reliability preserving end-to-end semantics in the routers was the real key to handling this issue. Vint also is a renowned wine expert, and treated me and William to a wonderful tour of fine wines at the Rubicon in San Francisco where we had a memorable discussion on exactly this issue.

Of course, their terms-of-art are different from mine, since we all seem to invent new terms. So their "network of queues" is my bucket brigade mechanism. And their test demo is similar to one at
InterProphet we called "FlameThrower" devised by Senior Engineer Todd Lawson and Software Engineer Madan Musuvathi to literally flood the other side with packets and see if it falls over. Todd and Madan built a wirewrap version of SiliconTCP on a DEC PAM card with a NIC wired on (and that's exciting with 100MHz logic). We demo'd this to Microsoft, venture, and lots of other companies back in Summer 1998. I have the wirewrap on my wall alongside a production board.

But the solution presented in this paper to "back pressure" the plug by disabling TCP congestion control selectively is where we part company. Herbert and Blanc/Primet quite correctly point out some of the barriers to FAST, Highspeed TCP / Scalable TCP, and XCP, but then fall back on the old link layer solution approach (which we diddle in the stack software). If only it were that simple.

Reliable link layer isn't enough, and Vint (in looking back) clearly knew this. That's why he saw SiliconTCP as fitting best here. This was the key reason he joined the board of InterProphet so many years ago.

Many people have made reliable link layers. We've done it with the boards we have here right now. But no one else made a reliable network and transport layer that spans many hops, maximizing the capacity of the aggregate network. Our boards also do this. So we did demo Vint's vision in practice.

It's only now that people are starting to thrash the problem that Vint saw many years ago. But they lack his insight as to the real nature of the problem. It isn't turning off congestion control - it's using it effectively. So long as engineers think the answer is a simple "stack hack" instead of rethinking how to more effectively meet the protocol demands - not new protocols, not turning off the congestion, not cheating by biasing fairness - but really simply doing our job better, we'll continue to run into this problem.

What, me worry?

This little article just in from a dedicated Cisco engineer. Looks like Cisco is taking a "broadside" from Broadcom in the "TCP offload" universe.

Of course, notice the weasel words of ""selected network streams". In contrast, at InterProphet we showed a 10x advantage of all network streams on NT at Microsoft in their offices in Redmond in 1998 with a patented design. So it's taken Broadcom and Microsoft working together about 6 years to kind of make something work but not really. Not very impressive.

And it works for Unix and other OS's as well. You don't have to use Windows to get the benefit. In a world of "choice", shouldn't a chip be OS-agnostic? Or do they think choice is a bad idea?

Don't know how often I read about a TCP offload mechanism which doesn't really do the job. If a network stack worked like these chips, the Internet would be a lot more frustrating a place.

But I doubt Cisco cares, even though their engineers do. With China's massive Huawei on the bottom end and ruthless Broadcom moving in, don't bet on Cisco to defend their turf. After all, they're too big to beat, right? And if a company lets things go long enough, figuring that "someone else I like better will invent it", they take the risk someone they don't like will do it.

Term Memory Patent Parchment Looks Lovely and Feels Great!

OK. I know the patent attorney said I got the patent grant ("Term Addressable Memory of an Accelerator System and Method") a while ago, but it really is different when you actually hold it in your hand! I was so excited that I told Vint Cerf about it and ever gracious, he said "congratulations, Lynne - persistence counts!" Means a great deal to me to hear that from the "Father of the Internet".

It's my 2nd parchment but there's more in the queue. This patent relates to the limits found in the original design for InterProphet discussed in the SiliconTCP paper I put together earlier this year. Work on this patent was done after InterProphet went into a low-key mode because of a lack of commitment to it as private venture. But just because it's easy to bet against someone knowing that life isn't fair, that doesn't mean it's right. Karma rules!

I'm happy to keep my word and execute it well. It just takes a bit more time to make it to shore when the winds are set against you. But winds shift, and so do trends.

SiliconTCP, EtherSAN, and Scalability

Everyone says I was amazingly ahead of my time. As Rick Merritt, EE Times writes about the possibility of using storage interconnects concluding "Competitors such as Broadcom Corp., which have existing 1-Gbit R-NICs, will not be able to scale to the greater bandwidth because they lack the ASIC state machine architecture...".

Well, now I'm pleased that I wrote a paper for the global storage network workshop last MayAll You Need is TCP: EtherSAN and Storage Networks, and even more grateful for the feedback I received from people like Jim Grey of Microsoft, John Wakerly of Cisco, and Greg Pfister of IBM. Gordon Bell was an earlier advocate of the InterProphet technology and urged Chuck Thacker to take a look at it several years ago. So it appears this is finally becoming a topic of serious consideration - although I've been seeing it coming for many years.

The fundamental scalable state machine architecture patent ("TCP/IP network accelerator system and method which identifies classes of packet traffic for predictable protocols") was filed in 1997 and granted 2000. A term memory patent ("Term Addressable Memory of an Accelerator System and Method was independently filed and granted July of 2004. It's a better memory approach that hand-in-glove with a state machine architecture that deals with certain flaws.

While these guys are hoping to use an interconnect to scalably use RDMA over TCP (something we discussed doing in 1999), they're not there yet. Why? Simple - if you read the other patents in this area, the "expert" opinion is you can't use a traditional CAM or a ternary CAM. That's what they're using. So you don't have to believe me - you can believe the other guys.

I've been told I should be getting a parchment shortly for my work on SiliconTCP and low power in cellular / WiFi. That got me thinking ahead of where we'd be this time next year, or maybe the year after. Four years ago when I was in a meeting at Cisco, I was told by the M&A group that low power wasn't important, and neither was storage. Now Cisco is moving heavily into storage. Is low power far behind?

Perhaps the toughest audience any technologist in a complex business has to face is the impatient investor. I've met VCs who expect to "get it" in one minute missing seconds, claiming they are such geniuses they understand everything. It's important to speak to someone who really understand the area, and is willing to do their work. These one-minute wonders are definitely bad news to a serious business.

It's been suggested that I just write a new EtherSAN paper that drills down more. Well, as more people validate this area, I believe I will. You see, being first isn't alway an advantage if others can't see the crest of the wave in the distance. If I did use the scalable state machine approach, which I shied away before from as a description (except in patents of course) because "nobody gets it", do you think they might finally get it now? Because that's what I see the other guys starting to say. All I can say in response is "Better late than never".

A ROSE is a ROSE - Reordering Segment Engine

Ashlee Vance wrote that Intel will be introducting "I/O Acceleration Technology" to "attack greedy TCP/IP stack" consumption - in other words, latency through the stack. "Customers often find that their servers spend an inordinate amount of time dealing with network traffic when they should be hammering away on application data." This sounds very familiar - we told them years ago that "all processors wait at the same speed".

Back in 1997, when I filed a provisional patent on just such an approach, I had an interesting meeting with Intel's processor side. We called the technique ROSE then, for Reordering Segment Engine for a product we envisioned called the Network Accelerator - and yes, this was before Adaptec and Alacritech and all those other TOE guys. It was the first in a series of parallel processing refinements, which dealt with the layer 2-7 issues of TCP/IP (the discussion was under NDA).

The ultimate low-cost solution was to build it into the southbridge. For high-end apps (going at a faster rate then the bus), you'd put it in the processor / memory interface itself. We even had a hardware socket interface. But in the meantime, as a first step, we could build Network Accelerator cards, just as the graphics accelerator cards changed the face of the graphics industry.

That same year, Dr. Vint Cerf (co-creator of TCP/IP) joined the Board of Directors of InterProphet, a funded company based in Silicon Valley which completed the patent filing based on a working prototype of the Network Accelator (see TCP/IP network accelerator system and method which identifies classes of packet traffic for predictable protocols, patent granted 2001). He joined because we'd solved the Internet TCP bottleneck problem that everyone said "couldn't be solved". Interestingly enough, companies like Alacritech formed *after* our company was formed, and reference us in their early patents as well. However, their TOE designs are not as efficient (high-cost) nor scalable.

We received the first patents in this area. Unlike other attempts to simply turn the stack code into Verilog (like Iready and others), we did a completely novel state machine implementation. It is an entirely scalable dedicated stream-oriented protocol processing mechanism. Everyone admitted we'd done it. We even on the $1M dollars funded not only did the prototype but also did a product and demo'd it to every major player in Silicon Valley (1998-1999).

There have been more patents since based on this technology, as well as papers and other work. At that time, stack latency wasn't considered as big a deal - big enterprise datacenter solutions with high-cost staff and technicians and big servers was still the norm. A low-cost, low-power chip wasn't considered important, as margins were so high and "would always be there".

Then came the dot-com bust, outsourcing, and the implosion of the enterprise market.

But Dr. Cerf even then spoke of the time when we'd see millions of Internet gadgets on the Internet, and doing older TOE or processor-intensive enterprise solutions would not be viable anymore. After all, how would we be able to power / control / manage 100 million transistor processors, each one embedded in a low-power sensor net? Or in your dress shirt for that matter?

Come On Charlie Brown, Just Kick the Football

Love the "Lucy holding the football for Charlie Brown" quote by Nathan Brookwood in CNET about the Dell / AMD relationship. It always seems so close, and then it slips away. Intel always holds the relationship.

Nathan may be right in saying last year it came the closest ever because of Intel's slips in so many areas. But instead of running with the ball, AMD fumbled by assuming they could rely solely on their 64-bit advantage for the sale. That isn't enough I was told. One exec who's negotiated agreements with Intel and AMD and companies like Dell told me that AMD needs to get "the whiphand" on Intel in some way to get the Dell close. AMD doesn't have that whiphand. And I know why. It came up chatting with an editor who wanted to know the background on Intel's preannounced new product. You see, he knew I'd been there, so he wanted the story again. So here it is...

Back in 1997, when I filed a preliminary patent on just such an approach, I had an interesting meeting with Intel's processor side. We called the technique ROSE then (see A ROSE is a ROSE - Reordering Segment Engine) as part of the Network Accelerator (see TCP/IP network accelerator system and method which identifies classes of packet traffic for predictable protocols). It was the first in a series of parallel processing refinements, which dealt with the layer 2-7 issues of TCP/IP (the discussion was under NDA). It is now called SiliconTCP (see What is Silicon TCP "really"?.

The ultimate low-cost solution was to build it into the southbridge. For high-end apps (going at a faster rate then the bus), you'd put it in the processor / memory interface itself. We even had a hardware socket interface. But in the meantime, as a first step, we could build Network Accelerator cards using SiliconTCP called EtherSAN to move the market, just as the graphics accelerator cards changed the face of the graphics industry. So that's what we built.

That same year, Dr. Cerf (co-creator of TCP/IP) joined the Board of Directors of InterProphet, a funded company which completed the patent filing based on a working prototype of the Network Accelator (patent granted 2001). He joined because we'd solved the Internet TCP bottleneck problem that everyone said "couldn't be solved". Interestingly enough, companies like Alacritech formed after our company was formed, and reference us in their early patents as well. However, their TOE designs are not as efficient (high-cost) nor scalable as ours, and I suppose filing first does help to keep the wolves outside the door.

We met with a VP at AMD as well in 1998, arranged by a USVP VC who had an investment interest in InterProphet. But AMD couldn't get why Vint said this was important. They didn't see the strategic value like Intel did. And they like to lead - not follow.

We received the first patents in this area. Unlike other attempts to simply turn the stack code into Verilog (e.g. Iready and others), we did a completely novel state machine implementation. It is an entirely scalable dedicated stream-oriented protocol processing mechanism. Everyone admitted we'd done it. We even on the $1M dollars funded not only did the prototype but also did a product and demo'd it to every major player in Silicon Valley (1998-1999). There have been more patents since, based on this technology, as well as papers and other work, so work continues.

But at that time, stack latency wasn't considered as big a deal - big enterprise datacenter solutions with high-cost staff and technicians and big servers was still the norm. A low-cost, low-power chip wasn't considered important, as margins were so high and "would always be there". Then came the dot-com bust, outsourcing, and the implosion of the enterprise market. It was all over.

But Dr. Cerf even then spoke of the time when we'd see millions of Internet gadgets on the Internet, and doing older TOE or processor-intensive enterprise solutions would not be viable anymore. After all, how would we be able to power / control / manage 100 million transistor processors, each one embedded in a low-power sensor net? Or in your dress shirt for that matter?

What's the upshot? This same exec last week said that AMD always screws up on getting the whiphand on Intel - not to win against them, but to make the sale to their exclusives. I'm just an inventor - not a saleswoman - but I think he's on to something from what I saw at AMD back then.

Do you think they'll ever learn the lesson and grab the football before Lucy does?

No, it's not "High Power", it's now "Low Power"

You may have heard that servers at Google have been packed so tight they catch fire in the datacenter. Turns out power dissipation is the key - even laptops get hot, and servers stacked are fire hazards. It's the power, everybody - the limiting factor to communications is power (according to Broadcom). What a difference a few years make. When I was invited to a meeting about InterProphet and SiliconTCP over at a major infrastructure company back then, the gal assigned to evaluate my work laughed at me when I seriously brought up the issue of low power and TCP. Of course, she also wanted to boast about some thesis she'd done on TCP about twenty years after everyone else. One (male) engineer, in hearing the description of the meeting said it was the first all-woman "pissing competition" on record. I told him just because I'm assigned a woman for due diligence because I'm a woman too doesn't mean I get a free ride - frequently it's the opposite. Alas, there's no secret sisterhood in business, but envy is universal.

So, how do you swap hot hotswap servers? The key is low power TCP - you can't burn out the stack anymore adding more and more processors (not to mention the management overhead) even in the datacenter (sorry Intel). You need to get the lowest power TCP stack possible. And that means a no-processor design.

The nontech "what does a low-power TCP hardware implementation do for me" (what a mouthful) is you get real full video on cellphones without burning out the battery as quickly. Since I'm doing automated full video production these days, that's kind of my interest too. Paul Baran wrote the basic patents on cellular communications for audio/video. But he couldn't achieve it fully because of this limit. He was so far ahead of his time it's amazing. I wonder what he'd think now?

So Silicon Valley isn't really dead technologically, despite what some people like to say. There are a lot of technology problems still to be solved, discussed in black and white in some legendary patents and papers from people like Baran that created entire industries. It's all right here. When I read patents, it's pretty clear to me that "everything *hasn't* been invented yet".

Unix was invented when I was in high school. The Internet - gee, it was old hat at Berkeley when I got there. Packet-switching? Baran. Cellular. Baran et. al. But no one man or woman could anticipate *everything* because a lot of the pieces just weren't in place. So the inventors carefully outlined why things couldn't work, or came up with nonviable solutions because of these missing pieces. It's all there - if anyone wants to take the time to read it.