What is high-bandwidth storage and why is the US blocking China?

What is high-bandwidth storage and why is the US blocking China?

The US government has imposed new export controls on the sale of high-tech memory chips used in artificial intelligence (AI) applications to China. These rules affect both U.S.-made high-bandwidth memory (HBM) technologies and foreign-made chips. Here's everything you need to know about these cutting-edge semiconductors, whose demand has skyrocketed alongside the global hype around AI.

What is High Bandwidth Storage (HBM)?

High-bandwidth memory (HBM) basically consists of a stack of memory chips, small components that store data. HBM can store significantly more information and transfer data much faster than the older technology called DRAM (Dynamic Random Access Memory).

HBM chips are widely used in graphics cards, high-performance computing systems, data centers and autonomous vehicles. Most importantly, they are essential for increasingly popular AI applications, including generative AI. These are powered by AI processors, such as graphics processing units (GPUs) manufactured by companies such as Nvidia and Advanced Micro Devices (AMD).

"The processor and memory are two essential components for AI. Without memory, it's like having a brain with logic but no memory," said G. Dan Hutcheson, vice president of TechInsights, a research organization that specializes in chips.

How do the restrictions affect China?

The youngest Export restrictions, announced Dec. 2, follow two previous rounds of enactments related to advanced chips by the Biden administration over the past three years. The aim is to block China's access to critical technology that could give it a military advantage.

In retaliation, Beijing has imposed new restrictions on exports of germanium, gallium and other materials necessary to make semiconductors and other high-tech devices. Experts agree that the new export restrictions will slow China's development of AI chips and, at best, stall access to HBM. While China's ability to produce HBM currently lags behind that of South Korea's SK Hynix and Samsung as well as America's Micron, the country is developing its own capabilities in this area.

“The US export restrictions would limit China’s access to high-quality HBM in the short term,” said Jeffery Chiu, CEO of Ansforce, an expert network consultancy specializing in technology. “In the long term, however, China will continue to be able to produce HBM independently, even if it works with less advanced technologies.” In China, Yangtze Memory Technologies and Changxin Memory Technologies are the leading memory chip manufacturers and appear to be expanding HBM line production capacity to achieve their strategic goal of technological self-sufficiency.

Why is HBM so important?

HBM chips are so powerful compared to conventional memory chips primarily because of their larger storage space and significantly faster data transfer. Since AI applications require many complex calculations, these features ensure that these applications work smoothly, without delays or glitches.

Greater storage space means more data can be stored, transferred and processed, improving the performance of AI applications as large language models (LLM) are able to work with more parameters. Think of the higher speed of data transfer, or the higher bandwidth in chip language, like a highway: the more lanes a highway has, the less likely it is to congestion, and the more vehicles it can accommodate.

"It's like the difference between a two-lane highway and a hundred-lane highway. You just don't have congestion," Hutcheson said.

Who are the leading manufacturers?

Currently only three companies dominate the global HBM market. In 2022, Hynix accounted for 50% of the total market share for HBM, followed by Samsung at 40% and Micron at 10%, according to one Research note, released by Taipei-based market research agency TrendForce. Both South Korean companies are expected to hold similar shares in the HBM market in 2023 and 2024, which would give them a total of around 95%.

Micron aims to increase its HBM market share to between 20% and 25% by 2025, Taiwan's official news agency reports, citing Praveen Vaidyanatha, a Micron executive. HBM's high value has led all manufacturers to focus a significant portion of their production capacity on the more advanced memory chip. According to Avril Wu, senior research vice president at TrendForce, HBM is expected to account for more than 20% of the total commodity memory chip market by value starting in 2024 and may rise above 30% next year.

How is HBM produced?

Imagine several standard memory chips stacked on top of each other in layers, similar to a hamburger. This is basically what HBM's structure looks like. At first glance, this sounds quite simple, but it is anything but easy to implement, which is also reflected in the price. The selling price per unit of HBM is several times higher than that of conventional memory chips.

This is because the height of an HBM chip is approximately the same as six human hairs. This means that each layer of standard memory chips stacked on top of each other must be extremely thin, requiring a high level of manufacturing expertise known as advanced packaging.

"Each of these memory chips needs to be ground to half a hair's thickness before stacking them on top of each other, which is very difficult," Chiu said. Additionally, holes are drilled into these memory chips before they are mounted one on top of the other so that electrical wires can pass through. The position and size of these holes must be extremely precise.

"You have a lot more failure points when you're trying to make these devices. It's almost like building a house of cards," Hutcheson noted.