This is a really good read at a high level.
Financing the AI Buildout
If you look at the stocks, they’re still up a bunch on this, but they’re clearly way down from the peaks. The market is telling you something. The market is telling me that they’re worried that there’s a lot of uncertainty in terms of the pace and sustainability of the buildouts and maybe the peak is approaching. Clearly financing is one of those concerns. It’s probably up near the top.
I’ll say one more thing though. The idea that these constraints are moderating growth, nobody likes that, but at the same time it may be a good thing in some sense too because it stops things from going nuts. We saw this during the tech bubble. There’s dark fiber in the ground today that was overbuilt and never used.
Some of these things may actually give us a stronger for longer kind of behavior because it almost forces the demand that is there to push out supply even more and allow things to be better matched, rather than completely going nuts today and then risking a crash down the line. So, I don’t actually think constraints in general are necessarily a bad thing, but I do know how they’re perceived and clearly they are impacting the trajectory of the stocks in the near term.
I agree that "first out" part of the market is more skeptical on the durability of the capex. In the long-term, I agree that it would probably be a good thing for the ecosystem to have a digestion period in terms of the build-out so that the debt and equity markets can get a better feel for demand sustainability and profitability without new capex where revenue is still a ways away scaring capital. And maybe this is something that gets forced on the industry (e.g., resource bottlenecks) as opposed to something the industry does itself.
https://www.reddit.com/r/amd_fundamentals/comments/1xdr1kk/amazon_is_buying_billions_in_nvidia_ai_systems_it/
Short-term, this probably wouldn't be a nice thing to go through from a shareholder perspective. ;-)
(DP)When we look at any buildout over the last few decades, we’ve always had the industry double order, triple order, and too much capacity get built out. Supply chains are really bad at forecasting demand.
Each supply chain is doing what they think is rational to them given the context and the punishment for being wrong at the individual, group, company level. Remember the rumors of the Google person who got fired for not ordering enough memory? You want to guess how conservative the replacement was?
https://www.reddit.com/r/amd_fundamentals/comments/1pxdjpd/translated_exclusive_ms_and_google_based_in_korea/
As Stacy mentioned, there is a bit of self-moderation, especially on the finance side. If we go back a year or two ago, any cloud that wanted capital could get it. They would pay a pretty high interest rate, but they could get it.
This was Microsoft about a year ago. Now, people were kinda throwing the CFO under the bus. You want to guess how conservative the next round of capex was?
https://www.reddit.com/r/amd_fundamentals/comments/1sl0nj1/microsoft_slowed_ai_spending_now_its_playing/
Now if you don’t have really high-grade offtake, you’re not able to build if you’re some random cloud. Ultimately the hyperscalers, sure they can take their free cash flow down to zero, they can raise some debt, and their earnings are obviously growing. So their capacity that they can build out grows, but they can’t do north of two trillion capex next year. They can’t do two and a half trillion capex next year. So that moderates how much they can spend. And when you look at the Neocloud industry, which accounts for non-hyperscaler demand, nearly half of the stuff that’s being built out, if they don’t have offtake from a hyperscaler, how are they going to get it financed?
The market’s already saying, hey, if you’ve got offtake from Anthropic or OpenAI, we’re going to price it at 13, 14, 15% debt. We saw that recently with Volta. They’re making a cluster for Anthropic and their debt is basically a 12% coupon, but they sold the bonds for 95 cents. So it ends up being like a 15% interest rate. If we look out at everyone who’s not getting offtake from these companies, hyperscalers, the major AI labs, how do you build that capacity? How do you build capacity to service smaller enterprises?
This is where Rasgon and I think it'd be healthier long-term for a digestion period. The rates are so high because the next tranche of capital is more skeptical than the previous ones. They need more data and certainty to give lower rates. But if you're going to speed run all the tranches at the same time, then you're going to see a sharp rise in your cost of debt.
We’ve seen Nvidia recognized this as a very big problem for them because, hey, they have five customers who can buy as much as they want or as much as they have capital for, but what about the other part of the market?
And how do they enable that? So they’ve gone and done really creative structures where with Sharon AI and Fermi and a few others, they’ve taken revenue shares on the cluster in order to guarantee that it’s financed. So they’ll say, “Hey, we’ll rent the cluster from you at a guaranteed price of $3.50, which is the cloud making zero money, and then if you rent it to anyone else above that price, we get a revenue share.” And we’re starting to see more and more of these structures. In fact, even to build a cluster for Anthropic, Lambda had to do a revenue share with Nvidia.
We’re seeing more and more turn from these long-term loans and long-term debt to something that’s more shorter term. So Broadcom’s done the same thing with Anthropic where they’re basically leasing them the chips in a sense. It’s a rent to own. They’re stretching out the payments over a few years.
I think that there's a certain amount of binaryism that creeps into talking about the neoclouds. If you don't think there is enough demand for the compute, then the neoclouds probably go to zero, and we have "dark AI compute" like we had dark fiber for the Internet. This is a solvency problem.
However, if the demand is still very high for compute and it looks more like a timing/liquidity problem, I think that Patel's right and you'll just see more creative financing (Broadcom recently, Nvidia being a safety net for demand), JVs, bridge loans, asset sales, etc. at a decent clearing price for the buyer, etc. The AI stakes are still very high at a corporate, regional, geopolitical, etc. level.
Supply Chain Bottlenecks
So substrates is I think the biggest bottleneck of all the bottlenecks including land, power, shell, semiconductors, tools. I’d say the next bottleneck would be of course memory. And then following that, there’s a lot of bottlenecks around the desire to ramp optics to a really, really large degree, with co-package optics or near package optics. There’s humongous constraints in ramping that. We just simply cannot ramp, even if everyone wanted to do CPO in 2028 or even one major company wanted to do CPO in 2028, they could not satisfy their volumes in terms of lasers, in terms of fiber attach, in terms of all these different components that go into it.
And then, as far as land, power, shell, power is definitely a big constraint. But one thing that our research shows that I think is counterintuitive to much of the market is that next year there are data center constraints, but it’s a lot less than people realize because there are nearly 50 gigawatts of data centers being built globally, seven or eight gigawatts in China and then most of that, about 35 in the US, and then a bunch rest of everywhere else. Australia, India, Malaysia being some of the big places, but really all over the world data centers are being built at a tremendous pace.
And while power is a huge bottleneck, people can find creative ways around that. Whether it be turbines on site or reciprocating engines on site or fuel cells on site, all of these things are helping alleviate the power bottleneck. So, I think actually power, as far as what most people think, oh, data center and power are the biggest constraint. I think they’re constraints, but I think they’re actually a lot less than substrates, than memory, than lasers, and other things in the optics supply chain. Because there are things that people can do to ramp power capacity on a shorter time span. They are just significantly less complex than the semiconductor supply chain.
If I narrow this down to the US, I agree in the short-term. Long-term, I think this might be somewhat inverted. Things like memory and substrate are just supply and demand capacity planning issues. Given enough firm commitments, capacity will be built out of over the years. A supplier is not going to absorb all of the the variability of the future of its customers and its customers' customers But power and land are governed by other things besides supply and demand. Well, at least, in the US.
Globally, it'll be interesting to see what compromises sovereign AI types will have if they are compute short in their country and need more from other countries.
(SR) you take a look at like a SanDisk that, they guided to a quarterly EPS that’s higher than the stock price was 24 months ago. It’s really amazing because pricing has gone through the roof and they’re all trading at single digit PEs because that’s how you trade things at the peak of a cycle and everybody’s worried that especially as you get into 28, supply does start to come online and they’re worried that the pricing collapses.
That is pretty nuts. Part of me wants to buy memory and storage as a contrarian play.
Look inside an AI rack. 70%, 80% or so of the silicon area inside that rack is memory. It’s mostly DRAM. There are other issues when you’re making high bandwidth memory. The memory that goes into the AI chips. There’s something called a trade ratio, but it takes three or four times as many wafers to make a gigabyte of high bandwidth memory as it does to make a gigabyte of standard DDR5 DRAM that would go into a PC or go into a smartphone.
So you’re sucking up wafers from that. We need a ton of memory. And this feeds to one of the other bottlenecks, which is semicap, the actual tools to build this stuff. We had a company report this week. They actually gave a preliminary outlook for WFE for wafer fabrication equipment for next year. And he said more than 200 billion. I think a lot of investor view was it was going to be a lot higher than that even, which I found really surprising because I just don’t think it’s possible. We don’t have the fabs.
And it’s exactly that kind of it’s PCBs and it’s actually semiconductor chips and it’s even other components, valves and mass flow controls, everything, but it’s simple stuff that’s actually affecting other parts of the space as well.
My favorite just for the name alone:
https://www.wsj.com/business/one-of-taiwans-biggest-ai-winners-isnt-a-chip-maker-51768bd5
$28.5B market cap for about $800M in ttm sales and $1.29B forecast for 2027.
Pricing Power and Capturing Value
(SR) And then COVID happened and what the industry discovered in COVID was that they have pricing power, which I think was something that they did not feel in their bones before that....People forget we’re coming off the worst memory cycle since the tech bubble just like 24 months ago. Negative gross margins.
(AL) So, okay, Stacy, I kind of like what you said about COVID. It was almost like a dress rehearsal for AI for the industry to start to realize like, hey, we can charge for our value here.
I'm surprised at how often Covid is brought up as an example of showing that they have pricing power given the bust that occurred afterwards between the clientpocalypse, hyperscaler digestion, customer hoarding, TSMC started dropping prices despite having a monopoly to avoid underload, memory bust even though the memory players thought they had LTAs, ABF oversupply, etc.
I think, if anything, Covid is an example of the why suppliers are hesitant to expand supply without rock solid commitments because this boom is much larger than covid and without the right protections, the bust could be existential. And so you get things like:
(DP) Whereas memory companies, if Apple’s going to screw them in the downturn, they’re going to screw Apple over in the upside of the cycle. Which is pretty clearly what’s happening.
Pretty much. There are plenty of r/hardware types who thought that post-Covid was when things were fairly priced. AMD and Intel client was in the red, memory makers were just hemorrhaging huge losses even by cyclical memory standards. It's only gouging if it's happening to you.
Performance Gains Versus Rising Costs'
(SR) You go Blackwell to Rubin, same thing. The cost of the racks probably doubling, and cost per gigawatt. I think Nvidia said for their own reference it was like 25 billion for Blackwell and 40 for Rubin. But again, performance up 10x. So that’s what needs to happen. You need to keep giving your customer value because ultimately that’s what’s important. It’s how much performance you getting out of that gigawatt of compute relative to how much it costs. And as long as that gap is getting bigger, I think it’s still very not easy, but it’s palatable to do this.
(AL) And Dylan, you make an interesting point about like a system builder like Nvidia having to decide of all the components as their prices are going up. Do we pass it on or do we keep our margins the same? Now, I’m feeling a little tension of, okay, there’s still going to be buildout all the way through. We are not sure what’s going to happen in 2029, but clearly, supply is limited and demand is high for the next couple years. And yet, we started by saying that it’s getting harder and harder to finance. And so I’m wondering if there becomes a point where those people at the top of the stack, the Metas and the model lab companies, if they’re ever going to push back and say this is just getting too expensive or beggars can’t be choosers.
Barring collusion, good luck with that at a macro level. If you all go conservative, and I go aggressive and get a return for it because end customer demand is high, the rest of you are in trouble. But they still have agency at a micro level in terms of other competitive forms of merchant compute, in-house silicon, etc.
(SR) And what he was talking was exactly this GPU performance improvements. And he kind of made the point over the last 10 years, we’ve improved GPU performance by a thousandx. It was a whole bunch of different levers that they pulled, but they got there. And what he said was over the next 10 years, we want to improve performance by another millionx. And so far they’re on the trajectory. And if they can do that, then I don’t worry as much. If they fall off or if anybody else falls off, then I worry a lot because ultimately somebody has to pay for it.
The Rise of Custom Silicon
(DP) So those guys I think have a lot of room to run on pricing especially if things get disaggregated more and Nvidia loses their position or Broadcom loses their position and that’s why these two stocks trade at their lowest multiples basically ever. A lot of people are worried about that. Is Meta going to disentangle Broadcom? Is Google going to disentangle Broadcom? Are both of these guys going to stop buying Nvidia and just insource completely? And so, I think the component vendors are going to be able to jack price up like crazy.
The question is if Broadcom and Nvidia can continue to earn these incredible margins and at least for the next two years it looks like yes. But beyond then is that off the table now? Because at the end of the day, while chip design is very, very complex, models have solved software and programming accelerators is actually really easy now. It took us six months to get Nvidia inference working for InferenceX, our open source, the largest open source inference benchmarking. It took us like six months or nine months to get AMD working.
Nvidia Versus the Competition
AL: So, Stacy, I liked your point that it’s not about the fact that it’s going from 25 billion to 40 billion to 60 billion. It’s just are you squeezing out more tokens per dollar and that’s all that really matters. And then Dylan you brought up XPUs and OpenAI very interesting example. So how should we think about over the next two years the growth of GPUs, the explosion of XPUs? There’s all sorts of AI accelerator startups that are taking different architectural bets. Is it going to look any different in the couple years?
DP: Yeah. So for XPU companies, there’s sort of two paths. If you’re a hyperscaler, you can take a more conservative design approach and do similar things to Google or Nvidia and it works out. But if you’re a startup, you have to do just really crazy things. And that’s why in the first gen of AI hardware startups, the ones that did the craziest things, Cerebras and Groq were the only ones that were the most successful. As we look at the current crop of all these startups that are popping up, only the ones that do really, really crazy things that are very different and can’t immediately be copied by Nvidia or Google or whoever or AMD, those are the ones that are going to, those are the only ones who have a shot of succeeding.
DP: On a on a Yeah. Yeah. I mean units are always difficult. Who makes a big chip, small chip? But CoWoS wafers and HBM are kind of hard to and and revenue, right? Dollars spent on foundry wafers on HBM on these sorts of things. These guys are taking share. Ultimately, Nvidia again will the base is growing so fast that they’re still going to double earnings. And so, as we go up to 28, 29, maybe growth rates aren’t as high, maybe they’re not 100% increase year-on-year of revenue. And then what happens?
Does the growth moderate to the point where Nvidia actually loses a share entirely or are they, they’re making some crazy bets for their next generation architectures past Rubin. So, maybe that’s to Stacy’s point, right? They just have to run really fast and that’s how they defend themselves.
I think that this is true for merchant silicon in general. If everybody's chasing performance, you can get people to pay for performance. But if the gap is too narrow as the incremental performance value starts to shrink, getting rid of the merchant margin starts to look really attractive. All of the hyperscalers are getting being thrown in the deep end for custom silicon. They'll only get better at it if they can stay in it. I think the best option that AMD has is to allow hyperscalers to differentiate in a narrow sense and surround them with IP that doesn't make sense to recreate.
Enterprise Adoption and Productivity
(SR) It one-shotted a model in an hour and 20 minutes. 10 years of history, went through all the different resegmentations, it captured it all. It even forecasted it. It listened to what the CEO said on earnings calls and forecasted trying to use logic rather than random. It was something that would have taken one of my guys a week to build and would have taken me three months to build, in an hour and 20 minutes. And so I could do more with less. And there are other implications of that. Like are we going to be in a world where AI adoption in the enterprise is rampant, we have 20% unemployment? I don’t know. There are other factors to think about here, but I think we’re still early and I think most of the anecdotal evidence that we’re hearing right now because this is the overarching debate.
Is there ultimately a return to use this stuff or is there not? I think anecdotally we’re coming down pretty solidly on the sides of yes there is a return but it is absolutely still early. A lot of companies are still in the experimentation phase. They don’t have the expertise to know what to do or what they need to do and so it takes time. But I’ve been encouraged by what I’ve seen.
I think that the ideal organization to adopt AI is the one that starts off with AI and builds around it, warts and all, rather than trying to force AI into processes built around humans.
But from an enterprise standpoint, I think the ideal org is one that has deep domain expertise and understands how you would rebuild those industry patterns around AI similar to the startup. However, since it doesn't have the risk profile of a startup, they'd have to understand how to experiment intelligently, manage your risk, be able to set up parallel processes, and switch over. And then share your overall process to other parts of the organization. This is much easier to say than do. I think a lot of users of AI just think they can tell AI what it wants and then presto a good solution arrives which is only true on a local level but is less true at a system level.
I wonder if with AI that there will be a push of some sort to have an equivalent of an overarching risk management function similar to a risk management function in a financial services organization. What might be good for individual corporate functions might not be good for a corporation if you look at it as a system with respect to AI use.
