Role of Memory in the data center buildout

We continue the component series of the AI data center buildout with Memory, today's single biggest AI infrastructure bottleneck.

What is memory? Every computer needs somewhere to keep data, and the work is split across three technologies. DRAM is the fastest of these three technologies and provides working memory for processors. NAND flash is slower but keeps its data when the power is off. HDD, built from spinning magnetic disks, is the slowest and the cheapest. The three form a trade-off triangle: the faster the memory, the more each gigabyte costs.

A simple analogy. Think of a classroom. DRAM is the blackboard. The teacher can write, rub out and rewrite in seconds, but the board is wiped clean before the next class begins, just as DRAM loses everything when the power goes off. NAND flash is the student's notebook. Copying notes into it takes longer, but they are still there next week. HDD is the school library. It is by far the cheapest place to keep large volumes of material, and more shelves can always be added, but the librarian (the spinning disk) has to walk to the right shelf to fetch each book, so every request takes longer.

DRAM: DRAM (Dynamic Random Access Memory) is volatile, meaning it forgets everything the moment power is cut. It is also the only memory fast enough to sit beside a GPU and keep it supplied with data at the speed it needs. AI uses DRAM in very large quantities, and makers are prioritising AI and server products. TrendForce reported that conventional DRAM contract prices rose approximately 93–98% in January–March 2026. Its subsequent forecasts projected further increases of 58–63% in April–June, 13–18% in July–September and 10–15% in October–December. Phones and laptops rely on DRAM for everyday operation too, which goes a long way to explaining why new devices have become noticeably more expensive this year. Samsung, SK Hynix and Micron together account for c.88% of revenue, making DRAM one of the most concentrated markets in semiconductors.

NAND: NAND is non-volatile flash memory. It keeps its data without power but is slower than DRAM. AI data centres depend on it because they hold enormous amounts of data: training datasets, the checkpoints a model saves as it learns (much like saving progress in a video game), and the finished models, which have to be loaded from fast SSDs before a GPU can use them. It is the same memory found in phones and laptops. A buyer choosing between 256GB, 512GB and 1TB of storage is really choosing how much NAND the device contains, and it is where photos, apps and files stay after the battery runs out. AI server demand is also putting pressure on NAND supply. TrendForce’s revised forecast projected NAND contract-price increases of 85–90% in January - March 2026, with subsequent forecasts pointing to increases of 70-75% in April–June, 10–15% in July–September and 15–20% in October - December. The main players are Samsung, SK Hynix, Micron, Kioxia and SanDisk (through a joint venture), and China's YMTC, which sells largely within China.

HDD: HDDs (Hard Disk Drives) store data on spinning magnetic platters. The technology is decades old and has survived for one reason: its cost per terabyte is a fraction of NAND's. In data centres, HDDs hold "cold" data such as old backups, logs and AI training datasets, which are rarely read but too large and too important to delete. Western Digital (c.42%), Seagate (c.40%) and Toshiba (c.18%) account for virtually all of the world's HDD supply. Ironically, AI's need for cheap cold storage has revived demand for what was widely written off as a legacy product.

The structural shift: DRAM is morphing into HBM. HBM (High Bandwidth Memory) is not a new material. It is ordinary DRAM, sliced thin and stacked into vertical towers that sit right next to the GPU, with thousands of microscopic connections running through the stack so it can deliver data far faster than standard memory. Because HBM is made on the same production lines as the DRAM in phones and laptops, every wafer a factory allocates to HBM is one fewer wafer for everyday DRAM. The two products now compete directly for the same limited capacity, and Samsung expects HBM to take nearly 30% of the DRAM industry's wafer capacity in 2027, up from c.20% today.

So why are memory prices going up so rapidly? Memory has always been a cyclical industry. When prices are high, every maker adds capacity at the same time, supply overshoots and prices collapse. The last downturn ran through 2023-24. Micron lost c.$5.8bn in FY23, Samsung cut production and capex was slashed across the industry, so memory makers met the AI wave with very little spare capacity. They are now racing to build, but a new fab takes 2-3 years to become fully operational. Until then, lead times stay long and existing capacity commands a premium.

Just how big is the market? Reports estimate that DRAM makers earned about $149bn in Q2 2026, up c.56% on the previous quarter and more than 5x on a y/y basis. Most of that growth has come from pricing. Shipments rose c.35% y/y while average selling prices nearly quadrupled and servers (the computers in data centres that run cloud services and AI workloads) now account for more than half of all DRAM demand. NAND has followed a similar path, with c.$84bn of revenue in Q2 (+72% q/q) and reports expecting around $337bn for the full year, close to 5x the c.$71bn of 2025. HDDs are smaller and growing more slowly by revenue, but still ship well over 100 million units a year for cold storage. A single quarter of DRAM sales now exceeds Nvidia's entire quarterly revenue.

HBM is heading to 30% of DRAM wafers, so where does the rest go? Samsung expects HBM to take nearly 30% of the DRAM industry's wafer capacity next year, up from about 20% currently. The rest is split between:

Conventional DRAM. The standard DDR5/LPDDR5X chips used in every PC, phone and server. This segment is currently under the greatest pressure on supply.

Specialty/legacy DRAM. Older, lower-margin chips used in cars, industrial equipment and budget devices. These are typically the first to lose capacity.

The shift costs more than the percentages suggest, because HBM needs three to four times the wafer area of regular DRAM for the same capacity. TrendForce estimates that by end-2027 HBM will take about 30% of DRAM wafers but contribute only about 13% of total DRAM supply.

Why memory, not GPUs, is now the industry's biggest bottleneck. Nvidia's advantage in GPUs rests largely on software (CUDA). The constraint in memory is more fundamental: it comes down to physical manufacturing capacity. There are three major DRAM makers and a small number of NAND and HDD makers globally, and a new fab requires tens of billions of dollars and multiple years to bring online. So when AI needs more HBM, the extra supply can only come out of memory meant for everyday devices. Conventional DRAM contract prices rose c.90-95% q/q in Q1 26 alone, and device makers' inventories fell from c.17 weeks of supply to about 4. Analysts have begun referring to the situation as "RAMageddon."

Current key debates. We highlight the current key debates on memory:

Has AI fundamentally changed the nature of the memory industry? Memory has always been cyclical. The case for a structural shift is that every new data centre and GPU generation needs more memory than the last, GPUs are replaced every few years, and buyers are now locking in supply through long-term agreements. The risk is the usual one: makers are adding capacity quickly, and JPM expects DRAM wafer capacity to grow by about a third between 2026 and 2028.

When will pricing stabilise? Supply is expected to stay tight through 2027, with reports estimating DRAM supply will cover only 70-80% of demand that year, before easing somewhat in 2028. However, demand continues to grow exponentially and its yet to be seen when the prices ease down.

What happens to margins over time? Memory makers' margins have expanded sharply, helped by aggravated price premiums on AI and server memory. The question is how much of that holds once new capacity comes online.

Share wars. SK Hynix still holds c.50% of HBM but Samsung is narrowing the gap and has reportedly secured over 30% of Nvidia's HBM4 supply for 2026 and Micron is closing in as a close competitor.

Ionic View

In our view, AI has positively transformed the memory cycle and there is further scope for this to continue. Current trends point to supply lagging demand through at least 2028. Every GPU needs more and faster memory beside it, and HBM's appetite for wafers pulls supply away from everyday devices. New fabs take years to come online and analysts estimate DRAM supply will cover only 70-80% of demand in 2027, with the shortage starting to ease from 2028. While that gap persists, the pace of price increases is likely to slow, but prices should remain elevated, with limited downside risk over this period. Companies with strong HBM positions appear best placed as demand is least price-sensitive and long-term contracts offer visibility. SK Hynix and Samsung lead this market with Micron a credible third.

***

Missed our previous deep dives? Catch it here:

TTT Edition 77 - Role of GPUs in the data center buildout

TTT Edition 78 - Role of CPUs in the data center buildout