While we’re all oohing and ahhing over CalTech’s FastTCP bulk transfers and record busting using their new TCP congestion control – interesting paper (finally) by Jin/Wei/Low – contrast this with friendly rival Stanford’s protocol high-speed TCP that changes the fairness (I find it interesting and provides some new ideas). Are either likely to impact anyone’s use of the Internet in the next decade, anymore than studying cold fusion?
I’m struck by how all this “record busting” may be a mere sideshow in the scope of real Internet usage, especially given Microsoft Research’s own Jim Gray’s economic arguments against bulk transfers at Stanford a few months back.
Jim said that it is cheaper to send a disk drive via FedEx overnight than any of these contests could provide of benefit to ordinary users. Could the CalTech and Stanford work be too early given that hard reality? I leave that to CalTech and Stanford to battle out which is better a decade down the line. But what about what we can study now?
Maybe dealing with that long latency network issue that Beck etal finds makes storage jitter intractable in the first place is the real challenge of the decade.
Recently a few database experts were suggesting that end-to-end principle might be applied to databases. Beck, Clark, Jacobson, … don’t address this. The question “Are database commits end-to-end – do they satisfy the end-to-end principle?” such as that described in the simplest case (akin to a chaotic strange attractor in physics).
Another thing that came up was “When does latency and jitter combine in a chaotic way such that reliability is injured in database transactions?”
Doyle at CalTech speaks of fragility vs complexity, and uses a combination of control theory, dynamical systems, algebraic geometry and operator theory to connect problem fragility to computational complexity, such that “dual complexity implies primal fragility”, in an NP vs coNP way.
It could be that robust yet fragile (RYF) is effective in defining what’s necessary to prove a viable global storage system. EtherSAN approaches the problem by idealizing the simplest end-to-end mechanism, TCP, with fundamental remedies – not increased complexity. RYF would indicate that this radically improves this by removing primal fragility.
All this seems very similar to the old fusion sustained power burst we had in physics a decade ago. Kept everyone busy until the SSC debacle killed everything in the field. Plasma research is only now beginning to recover.
Let’s go back to fundamentals with Clark etal on end-to-end and simply considering Beck’s well-done arguments for small transactions per storage, cleaving to those goals only and not creating new ones. Reexamining definitions, and understanding them better, ala Bohr and mass, but not changing them.