AI – Lynne's Take on Tech

Thoughtful Thursday: Data Center Marauders and SV Villains

Today as our country spins further into self-induced insanity, I thought I’d talk a bit about (drumroll) data centers! This is ordinarily a rather dull topic to most, consisting of equipment and deployments and connectivity, hardware and software and systems, processors and networks, and of course, operating systems.

Since it is a dull but necessary topic that keeps things running, I could not resist writing about them back in the day. I even put together a book for Wiley entitled Inside the Internet Data Center in the early 2000s which predicted many Internet issues from the perspective of the data center administrator and how to deal with them. It was an enjoyable exercise. Unfortunately, the publisher decided it was an uninteresting field of study and canceled the book. I got to keep the advance and soon moved on to other projects. But data centers will always hold a fond place in my technology heart.

Now, everyone is talking about data centers. But not fondly. Not as an intellectual exercise. Nope, they are talking about them as technological marauders, stealing power and water and land from ordinary citizens in small towns. These entities are placed without any consultation or approval of the citizens in their communities. And the deals for taking the land and power and water are too often under nondisclosure, meaning nobody knows until it’s too late.

The big players in AI – Nvidia, Meta, Google, Amazon, OpenAI, and Microsoft – are now seen as the faces of villains. It’s a PR disaster for Silicon Valley. And yet, this is the endgame move. There are lots of pieces and players – and billions of dollars involved.

At the core, a data center is simply a processor receiving a request, accessing a database entry, and sending a result back. The devil is in the details. And there are a lot of devils, ranging from the actual hardware management and maintenance, to connectivity requirements, to database management, update and integrity — not to mention the software and operating systems that superintend these functions.

These operations require power. Lots of power. And they produce heat. Lots of heat. Hence the massive power and water requirements. And the increased demands of AI mean even more power and water for those AI chatterboxes telling folks that they can do no wrong, even though the actual interactivity to the end user is essentially meaningless.

According to McKinsey, almost $7 trillion will be invested in data center acquisition and development by 2030, $3 trillion in real estate and power infrastructure and $4 trillion in “computing hardware infrastructure” ($3.5T in servers and $.8T in storage). This is a mind-boggling number of processors.

Nvidia as the leading vendor of GPUs is beyond excited. At GTC this week, Nvida CEO Jensen Huang hailed these numbers, claiming he sees “at least $1 trillion” in revenue from Nvidia products through 2027. “I am certain computing demand will be much higher than that.” As Motley Fool noted, “This is startling — in a positive way — as it comes only five months after his earlier forecast, suggesting enormous momentum in orders in recent months. Also, Huang’s words imply that, from what he’s seen so far, he thinks revenue could surpass the level of $1 trillion.” If so, McKinsey’s predicted spend is underestimated.“

The real estate side of the equation is equally compelling. Data center REITs are all the rage. While partners in these ventures are often carefully shielded from local scrutiny, huge funds are usually involved in the acquisition of land and construction of essentially a bunch of big concrete shells. Rural locations are targeted. Land is cheaper in rural locations. Political resistance is diluted. And oftentimes water available for agriculture can be recaptured for equipment cooling. In addition, it is easy to buy a few local officials (and relatively cheap) and obtain through strong-arm tactics power easements to the datacenter.

When large corporations come into a community, they usually attempt to co-opt it by selling folks on jobs. However, once the datacenter shells are put together, there really isn’t much of a need for unskilled or semi-skilled labor. A large data center requires very few on-site personnel. Most software and systems administration is done remotely by engineers. Hardware failover means that a lot of CPUs and GPUs can burn out without having to swap out the damaged equipment.

The modern data center isn’t a factory with people working in shifts and products trucked to customers. It’s a factory for analyzing your most personal secrets, foibles, and thoughts, circumscribing every purchase you make and every comment you write, and monetizing it.

If fund managers believe that $4T in real estate is worthwhile, think of what value they place on your personal data. To get a 10-fold investment return, youre looking at the data value of literally hundreds of trillions of dollars estimated. These numbers rival the GDPs of major countries and blocks, with the US, China and the European Union in the $20 trillion economy group. That’s two countries and one economic block.

Now, these numbers are incredible, but also quite ephemeral. The same value of data of citizens incorporated implicitly in GDP as a complex factor is often claimed as a pure monetary value by AI enthusiasts. Nope, this isn’t the ROI. It’s about controlling the mindspace of investors and securing current investments.

What is real and tangible is the resistance of data center construction in communities throughout the US. And that antipathy has gone hand-in-hand with increased public fear of AI used for government surveillance. And most certainly some of the biggest advocates for its use have been companies like Palantir. People don’t like being reminded they’re being watched, even though that’s what the “free Internet” has been all about.

AI advocates have touted AI as the means for companies to find and fire people because they’re no longer needed. Lots of people. Huge numbers of people. This doesn’t make folks worried about their next paycheck feel real good about it. Especially when they open up their power bill and find it’s gone way up because a datacenter in their area has raised costs for everyone.

And finally, the debacles of AI misinformation from overeager AI advisors have become a joke, and a mean one at that. AI for the masses is not a trusted source.

This is how the standing of Silicon Valley companies has plummeted — not in terms of the stock market, but in terms of their trustworthiness.

William and I discussed the benefits of AI and data centers many years ago. Those benefits haven’t changed. But the mishandling, greed, and sheer contempt of major players in Silicon Valley and large investment funds have frittered away the goodwill and obscured the benefits.

It’s too bad. I really do like data center design. I like the software. I like the systems. I like the hardware. I like the benefits of AI.

But I don’t like the companies and pundits. Nobody does.

Fun Friday: Generative AI, Infected LLMs and Breaking Tulips

It’s a sunny and pleasant day here in Silicon Valley. A perfect day to chat about the imperfections of AI and the oddities of Breaking Tulips.

The extinct Semper Augustus Tulip (Norton Simon Museum)

Several weeks ago, so long now that everyone has likely forgotten, MIT put out a business survey of generative AI effectiveness, and found that hardly anyone is happy, except the consultants. “Just 5% of AI pilot programs from enterprises contributed “rapid revenue acceleration,” while the majority stalled and offered little financial impact”.

Of course, companies like Nvidia took a brief hit and then promptly went back up again, proof of the resiliancy of this latest Silicon Valley AI Bubble.

Given the wackiness of our current business climate and government, it is no surprise that people are clinging to generative AI as a beacon in the looming economic darkness. There are few opportunities for launching other technology startups as a few huge AI companies continue to suck in the majority of investment dollars and press hype.

New technologies are also a very hard sell in a risk-adverse market. Customers will only buy from extremely well-funded “startups” or well-established companies — if they purchase anything at all. And with the ever-present fears of economic upheaval, nobody wants to issue a PO, unless it’s for a sure thing.

And that sure thing is, of course, generative AI! Why? Because all the press and buzz says it is.

Behind the curtain there is great concern about results. That is why companies like IBM are actually doing a bit better. The assumption is there will be a need for high priced consultants to make generative AI a positive and lucrative business success. This confidence in the consultant pipeline and the belief that generative AI will eventually “get there” is what’s keeping the investment bubble going.

Oddly enough, contrarians are saying generative AI is just another “tulip bubble”. The actual Netherlands tulip bubble occurred from the early 1600s to the 1630’s. After a long period of fascination and investment in tulips by the aristocracy, the merchant class got into the action in a mass of speculation and eventual catastrophic collapse. It’s one of those “case studies” economics students love to pontificate about. So yeah, everything is a “tulip bubble” to some folks.

I’d like to say right now that our AI bubble is not a tulip bubble. If you want to go pick flowers, try crypto.

But there is one little piece of trivia about the tulip bubble that I do find applicable, particularly to broad-based LLMs. And that is the investor fascination with Broken Tulips.

Broken Tulips are a stunning flower. Induced by a virus, tulips will unpredictably turn from a humdrum bland solid color to an exciting streaked flaming riot of colors. These tulips were sought after by investors and caught the attention of the masses. Everybody wanted one. Artists painted them. Investors purchased them at huge sums. They were the height of tulip mania.

But a virus riddled plant is not a strong plant. And so it was with these tulips. They all died out very quickly — and so did the investment. According to the Amsterdam Tulip Museum, “Over time, the virus weakens the bulb and inhibits proper reproduction. With each new generation, the bulb grows weaker and weaker, until it has no strength left to flower and withers away.“ Not a good bet for seasonal returns. And since tulips are supposed to bloom every year, a dead plant is a dead loss.

Which brings us to another recent study, in the long line of studies, relating to generative AI LLMs and “hallucinations”, in this case by researchers at OpenAI. In Why Language Models Hallucinate, Kalai, et al. argue that language models are essentially encouraged to “guess” if they don’t know the answer, leading to absurd and false statements. They also state that this problem is persistent and pervasive due to the training and evaluation processes inherent in the process.

Current LLM models are akin to breaking tulips. They’re infected with incorrect assumptions that make them appear smart and decisive. Like a breaking tulip, this confidence inspires the customer to want more and more. In an age of complexity, nothing is more seductive than a decisive answer. And when a model seems to know more than you do, you can stop worrying and thinking and just defer to it. Even if it’s wrong.

Hallucinations popping up again and again are not a good basis for business decisions. Hence the MIT survey results.

Like that lovely infected tulip, the infection is persistent and insidious — to the point the LLM model may become too damaged to rely upon. Kalai, et al. offer no satisfactory cure for this infection. They state, “This ‘epidemic’ of penalizing uncertain responses can only be addressed through a socio-technical mitigation: modifying the scoring of existing benchmarks that are misaligned but dominate leaderboards, rather than introducing additional hallucination evaluations. This change may steer the field toward more trustworthy AI systems.”

In other words, businesses should go back to the drawing board and create their own LLM results based on binary true/false statements, carefully run, so as to mitigate the attempts to provide any non-verifiable answers. Which is how earlier models were typically done.

When a solution is sold as “simple”, telling the customer they now must do a lot of complicated work to make it trustworthy isn’t the best answer. But it’s the only path forward for these customers who are betting their future on generative AI.

Generative AI is not a tulip bubble. But some aspects of it are bubble-like. Given the promise it offers to business, I expect to see curated trustworthy business LLMs which are held proprietary.

However, for those heavily infected models used by the public, we can continue expect nonsense spewed out confidently and wrongly. I doubt most people will notice.

Sedate Sunday: AI Spending is Never Enough. Vine Revived?

It’s a pleasant morning here in the Santa Cruz mountains overlooking Silicon Valley and Monterey Bay. So what better time to think of AI and video?

According to the Guardian, our beloved tech overlords – Meta, Microsoft, Amazon and Google – have spent a whopping $155B on AI development this year alone. As the Guardian notes, this is “more than the US government has spent on education, training, employment and social services” the same period. Oddly enough, these are also the very areas where AI is expected to take over from all that expensive and pesky “skinware” (that’s people, in case you didn’t know). Such a coincidence.

Most of this money falls under the capital expenditures category, which is not an expense but an investment in infrastructure, R&D, semiconductors, and so forth, and hence runs under very different tax rules. Most of these investments I would say fall under the AI rubric only tangentially, but are categorized as “AI” to make investors happy. And happy they are. Big Tech (sans tariff goblins) have lots of money to spend on themselves.

From a Silicon Valley perspective, many of these investments are long-overdue, particularly in semiconductor design and datacenter platforms, along with requirements for low-latency low-loss networking. So this is actually a good thing for a broad swathe of hardware and software engineers and designers.

On a more amusing note, Musk announced yesterday that the Vine archive will be restored after seven years in cold storage after Twitter dumped it. I’d hate to be on the restore team. The bit rot coupled with mind rot would be formidable. But if you’re wishing you’d grabbed Aunt Martha’s six second blowing out the candles on her birthday cake over and over, now’s your chance, if it’s really still there. Industry and customers have long moved on to other video pastures, so this seems just another fair weather balloon soaring on bluster.

It’s still a beautiful day. Take a walk. Breathe the free air. Remember that nothing lasts forever. So take a chance today.

AI Trends for 2024: What’s Old is New Again

There’s a lot of shock and awe in the AI space these days. And lots of money on the table. But through the sturm und drang, some trends are emerging. To level-set, I went back and reread our last published article together, Moving Forward in 2020: Technology Investment in ML, AI, and Big Data (William F Jolitz & Lynne G Jolitz, Cutter Business Journal, 7 April 2020).

Four years ago, AI was at a crossroads. When we looked a traditional value propositions in technology, where one went from a specific technology to a target customer in a high value sector to a broadened sector and use, AI was doing miserably. 70% of companies said their AI projects provided little to no benefit to their company. Only 40% of companies said they had made a significant investment in AI. The frustration lay with “products sold with ill-defined benefits” which led to “unsustainable revenue that plummets when customers become disillusioned from a tactical lack of sales focus”. We stated the key problem was “the startup’s sales focus no longer aligns with the customer’s strategic focus”. In tech speak, they couldn’t figure out what to do with it and got disappointed.

We suggested what we called an “axiomatic” approach: “Instead of moving from technology to key customers with an abstracted TAM (Total Available Market), we must instead quantify AI and ML benefits where they specifically fit within business strategies across segment industries”. We then highlighted three areas to watch: surveillance, entertainment, and whitespace, while also discussing the issues with ad hoc architectures which potentially disrupt the cloud services costs and security. In terms of architectures, there is now more focus on data ownership and control, as well as reducing costs in the cloud. But it’s still very much the same as four years ago for most customers.

But the key prediction where we were literally “on the money” was our analysis of chaotic disruption of the market forwarded and funded by “super angels”. This was how companies like OpenAI spawned and spurred tremendous disruption in a very short timeframe:

“Venture capital (VC) investments in ML/AI fixate on a startup’s ability to obtain go-to-market sales by disintermediating other vendors and to lock-up highly profitable (yet elusive) opportunities. The VC’s intent is startup validation and gauging threats to other vendors’ uncompetitive businesses that will drive the startup’s ability to gain partnerships and revenue shares. However, sometimes, the result is not what VCs would wholly desire but rather more like paralysis with no clear “win” — because the startup only partially engages the customers and does not succeed in displacing other vendors. To force the win, tactical deconstructing/reconstructing of AI/ML solutions around existing layers of edge and cloud platforms as an investment category is akin to desperately reshuffling poker chips on the poker table. This is best avoided. Industry disruption is inherently unstable. Like an ouroboros, it can abruptly turn from obvious low-hanging fruit targets to feeding off earlier successful targets undergoing a state of change.
The potential for radically greater opportunities is more interesting than patiently maintaining course or re-navigating the rough waters to see existing ventures through to a reasonable conclusion. This potential is the realm of super angels, self-funders, and leading edge “winners.” These individuals and groups see no disadvantage to riding a chaotic wave because they’ve gotten accustomed to being so out in front of theirthe self-competition within their newly chosen, ever-shifting “whitespace path.”
However, the traditional VC process is disadvantaged by these groups because venture capitalists’ gut instincts based on the feel of the deal get whipsawed by the loss of bragging rights to ROI, limiting them from getting too far out beyond their headlights. Thus, chaotic disruption is a no-go zone for most. For those who decide to enter these perilous waters, the tendency to share risk across many partners leads to a kind of groupthink at odds with the fast moves and flexibility required of the super angels.

As open source investments demonstrated, it’s a risky business consuming your own potential customers. In the AI chaotic disruption, all potential customers are considered targets: media, artists, writers, businesses.

In consuming the “long tail” of literature, art, whitepapers, business databases, and personal information and opinion on the Internet and then regurgitating it as facsimiles stripped of authorship and authority, companies like OpenAI and Google whipsawed established players. As we have seen, the rush of businesses and consumers to magnify this effect was phenomenal — and dangerous.

The intent was to rapidly drive paniced companies to sign exclusive agreements and become the dominant company in AI for the next half century. If it sound unbelievable, note we now have only a few companies which dominate search, content, and connection due to brand recognition and addictive use. It takes a lot of money to maintain an addition, or establish a new one.

As the investment space is still suppressed due to poor conditions despite all the dry powder, there are a few bright spots. Battery investments continue to spark interest. Climate change companies surge and storm. Crypto was actually legalized by the SEC, because you can’t play with GameStop forever — so it’s time to jump into the big scams, kids. Space investments are, well, vast. AI plays a role in all of these.

But because of the chaotic disruption strategy that our billionaires strategized in Silicon Valley, AI now has the attention of everyone, from governments and military and NGOs to plain ordinary users. It doesn’t matter if AI is “lazy”. Even the IMF is jumping in.

Will AI benefit humanity? That’s out of my paygrade. William and I saw it had unique potential in many areas in 2020. That’s still true in 2024. I hope the chaotic disruption doesn’t prevent us from seeing some real benefits for the better.

Merry Monday: Venture Gets Antsy, Boom or Bust? Everyone goes Buggy over AI

Another Monday, another week of business excitement.

Venture investment firms are getting nervous as the projections for a slow IPO market in 2024 make fund exit profitability iffy as each fund matures. So what to do? Simple – roll the money into another fund that has no exit date and call it a day. And so, continuation funds, and their more desparate cousins, strip sales, are generating increased interest (subscription required, and I’m really sorry about that).

It’s hard to support a wealthy lifestyle before one is truly wealthy, but somehow VCs manage with their fees. But they want more. And they’re going to get it, by hook (clawback) or by crook (IPO).

It may be hard to believe, given most of these funds have ten year lifetimes, but even though we’ve gone through a lot of “booms” over the last decade, apparently they’ve chosen so poorly that they don’t have the exit they promised their Limiteds. In a more cynical moment, one might also wonder if the carry was so good they held off on distributions, hoping for more.

And then the pandemic hit. And then we had inflation grow. And now we are watching a world grow hotter in terms of climate and strife.

There are still optimists, however.

UBS Wealth Management just released a case study claiming that the 2020’s will look more like the 1990’s (without the early 1990’s recession) instead of some Roaring 20’s stock market insanity before folks jump off of buildings. For that alone, I must confess I’d rather consider a Clinton-style economic “Americans for a Bigger America” boom, as I don’t like heights.

In terms of technology investment, the 1990s was a good era for us personally.

We started off in 1991 (after two prior years of work with Berkeley with no real support) with the introduction of Porting Unix to the 386 in Dr. Dobbs Journal. Over the next two years, we painstakingly described our 386BSD port of Berkeley Unix from design to execution to distribution over the Internet.

386BSD made manifest Stallman’s concept of “open source” as a means to encourage innovative software development.
386BSD demonstrated that the Internet could be a viable mechanism for software distribution and updates instead of CDROMs and other hard media.
386BSD provided universities and research groups all over the world the economic means to finally conduct OS and software research using Berkeley Unix on inexpensive 386 PCs instead of minicomputers and mainframes.
386BSD spurred a plethera of new funded startups launched with a focus on open source online tools and support.

In sum, 386BSD broke the logjam on university research encumbered by proprietary agreements and spurred the growth of a new industry in Silicon Valley.

Not bad for a research OS project that was disliked by Berkeley’s CS department and essentially moribund by 1989.

In 1997, we observed that Internet traffic wasn’t well-optimized. We launched and obtained funding for InterProphet, a low-latency TCP processor dataflow engine. In 1998 we went from concept to patents to prototype. We proved that dataflow architectures, a non-starter in the 1980s with floating point processing, was a viable means to effectively offloading TCP processing from the kernel to a dedicated processor, just as graphics was offloaded through use of dedicated graphics processors. We did this on a million-dollar handshake investment and a handful of creative engineers. Silicon Valley can be an amazing place.

Global strife and pain isn’t usually good for business. It was an age of “renovation, not innovation” the prior decade — hyper-focused on strip-mining extant technologies and vending rent-seeking fads — unequipped to deal with these unsettled times.

But such times often spur interest in non-conventional problem-solving, opening a door to new technologies and risky solutions. Given the huge issues with climate change and what it spawns, we do need more “innovation, not renovation”.

I guess I’m just an optimist.

Speaking of technology fads, as the smoke clears and the mirrors crack, will generative AI still be the savior Silicon Valley hopes it to be? As with all answers, it depends.

If one owns the datasets from which one mines the answers, likely “Yes”. Security and privacy issues are moot inside your datacenter, for the most part, assuming you actually invest in security. Cost reduction is a viable metric that businesses can use to determine the efficacy of AI independent of fads. Expect to see use cases that focus on support and customer effectiveness. We also should expect new solutions crafted out of analysis of highly complex areas, from drug development to climate modeling.

However, generative AI has limits. The latest cut of a thousand critics, courtesy of Google, demonstrated that one could overload the AI generative variation responses to the point it begins to spew out actual training data using Internet data with a single word. Using an extraction technique that relied on an infinite request (do something “forever”), they achieved immediate results:

“After running similar queries again and again, the researchers had used just $200 to get more than 10,000 examples of ChatGPT spitting out memorized training data, they wrote. This included verbatim paragraphs from novels, the personal information of dozens of people, snippets of research papers and “NSFW content” from dating sites, according to the paper.”

I’m sure a new set of lawsuits based on infrigement are already in the works. Maybe even using ChapGPT to generate them. Who knows?

So let us send our thoughts and prayers to the poor VCs and happy lawyers. I haven’t seen this much eagerness for technology infringement lawsuits since the USL v UCB and Java v Everybody years.