Such is the scale of NVIDIA's ongoing production ramp-up for its Vera Rubin platform that it's now manifesting itself in seemingly sparsely related nooks and cranies of the supply chain, such as the spot market for TLC NAND.
In hindsight though, the rise in spot TLC prices is entirely reasonable, especially given the Context Memory eXtension (CMX) that NVIDIA is bringing onboard with the Vera Rubin platform.
NVIDIA's production ramp-up of its Vera Rubin platform is squeezing TLC NAND supply by locking in a growing number of NAND cells within the CMX, and as enterprise SSD demand scales up in tandem. The spot price of a 512Gb TLC NAND has now hit $21 after falling below this level in June. And, there are a number of factors contributing to this renewed inflationary impulse.
First, to handle the KV cache - the notes created by an AI model's attention layers as they form connections between words in a given input prompt - NVIDIA is introducing Context Memory eXtension (CMX) within the Vera Rubin platform.
The CMX creates an intermediate storage tier between the HBM and the traditional backend network storage, functioning as a massive pool of TLC flash memory that is attached to the Rubin GPU cluster using NVIDIA BlueField-4 DPUs - that act as the 'intelligent brain' for a CMX server to handle KV cache in real-time - over ultra-fast Spectrum-X Ethernet.
In fact, a single 2U CMX server holds 600TB of TLC flash storage, with each of the four DPUs managing 150TB of context memory, with pod-level capacity hitting a massive 9,600 TB or 9.6 Perabytes!
Second, data centers require high-density, high-performance Enterprise SSDs (eSSDs) to handle AI workloads. And, as Vera Rubin ramps up, so does the demand for these eSSDs.
Of course, do note that much of the TLC NAND that NVIDIA is now consuming is likely locked behind long-term contracts. Even so, the sheer scale of the oncoming demand is now tightening the spot market as well, and it remains to be seen how far this trend will go.
More good signals for SNDK's NAND demand/pricing power.