The father of HBM: The essence of AI is memory; GPUs only work for 10% of the time.

Professor Kim Jung-ho of the Korea Advanced Institute of Science and Technology (KAIST), known as the "Father of HBM," has made a groundbreaking assertion: the essence of AI is memory, not GPUs. Recently, Professor Kim Jung-ho of the Department of Electrical Engineering at KAIST gave a video interview, systematically elaborating on the evolution of HBM technology, the landscape of AI computing power, and future semiconductor architecture. Known as the "Father of HBM," Kim Jung-ho collaborated with SK Hynix on the development of HBM1 in the early 2010s and has since led a series of underlying architecture research projects. This interview has been widely circulated in technology and investment circles, with its core viewpoint directly addressing the structural contradictions in the current AI computing power race. In an interview, Kim Jung-ho directly presented a shocking figure: "Even with 1 million GPUs installed, they only operate for 10% of the time." He explained that whenever ChatGPT outputs a word, the system needs to read data from HBM, perform calculations, and write it back to memory. "Reading and writing take up almost all the time, while the GPU just sits there waiting." Even with algorithm optimization, GPU utilization is difficult to exceed 30%. This is precisely the practical basis for his core argument that he has adhered to for many years: "AI equals memory." I. Why GPUs Encounter an "External Communication Deadlock" Kim Jung-ho's assessment of NVIDIA's current situation is sharp. He said that Huang Renxun's recent frequent visits to South Korea, appearances on variety shows, eating fried chicken and drinking beer, and meetings with various people indicate that "so many meetings show his unease." "The technological growth of GPUs has almost stopped; that's my judgment. The evolution of artificial intelligence computers is in the hands of memory." His logical chain is clear: if GPUs want to improve performance, they can only increase chip area and stack more computing units; however, GPUs get too hot and must have cooling devices installed on the back, so they cannot be stacked vertically like memory. "GPUs are trapped in a dead end (external communication dead end)." In contrast, as we shift from the training era to the inference era, the importance of memory is being repriced. Kim Jung-ho says, "In the inference era, what matters more is how much data is crammed into AI, and the semiconductor that determines this is memory." He further points out that the competition for AI capabilities is ultimately a competition for memory capabilities: "Google Gemini, OpenAI, Anthropic Claude—who is stronger is determined by memory—that's my opinion." II. The Two Core Elements of HBM: Capacity and Bandwidth Kim Jung-ho summarizes the value of HBM into two dimensions. The first is capacity. With the advent of context engineering, multimodal input, and Agentic AI, memory demand is doubling every year—"1000 times in 10 years." Traditional methods rely on shrinking transistors to increase capacity, but we're now approaching the boundaries of quantum mechanics, making further shrinking almost impossible; therefore, we must "stack upwards." The second is bandwidth. Kim Jung-ho used an analogy: "If traditional memory is an 8-lane highway, HBM is 1024 lanes; now it's 2048 lanes; and in a few years it might reach 1 million lanes." Only by transmitting massive amounts of data simultaneously through parallel channels can we match the speed demands of AI computing. Third, HBF: The Stacking Era of NAND Flash HBM solved the speed problem, but capacity still has a ceiling. In the interview, Kim Jung-ho elaborated on what he believes is the next technological path—HBF (High Bandwidth Flash). Simply put, HBF is like vertically stacking NAND flash memory, similar to HBM. DRAM is fast but has limited capacity, while NAND flash memory has large capacity and can store data for a long time. Although it is slower, it is sufficient to meet the storage needs of "cold data" in inference scenarios. Kim Jung-ho believes that in the future, HBM and HBF will coexist, similar to urban planning: "Just like there are department stores, duplex apartments, and ordinary residences around them, various forms of HBM and HBF are combined to form a complex, supplying data to the GPU." He made a clear long-term prediction: "Now is the era of HBM, but in 10 years, the market demand for NAND flash memory and HBF will exceed that of HBM. Samsung and SK Hynix must prepare for the HBF era." He pointed out that companies currently developing HBF include SK Hynix, SanDisk, Samsung Electronics, and Japan's Kioxia. Kioxia's market capitalization recently surpassed Toyota's, becoming the top-ranked company on the Japanese stock market. SanDisk's stock price continues to rise, while Samsung and SK Hynix maintain their leading market capitalization positions in the South Korean market.

IV. HBS: A More Advanced Third Path

Kim Jung-ho also proposed a concept that is still at the forefront of technology—HBS (High Bandwidth SRAM).

SRAM (Static Random Access Memory) is about 1000 times faster than DRAM, but it has low density and high cost, and traditionally can only be used as a small-capacity cache within a chip. Kim Jung-ho's idea is to make the entire 12-inch wafer into SRAM, and then vertically stack 12 to 16 layers, which would expand the capacity from 100GB to 1600GB. "If it's 1000 times faster and has enough capacity, then it makes sense." He described the ultimate AI chip form as a "100-story 3D building": "HBM, HBF, and HBS each constitute multiple layers of the building, with the GPU placed on the top floor to handle heat dissipation and cooling. This is the inevitable 3D semiconductor structure of future AI computers—that's my current assessment." He also frankly admitted that the biggest engineering challenge of this path is not computation, but power supply and heat dissipation: "Providing the GPU and stacked memory with thousands of amperes of power, the design of the power supply network will be the most difficult technology, and this will become the real core competitiveness among enterprises." V. Customized HBM: The Relationship Between Clients and Suppliers is Reversing. Kim Jung-ho specifically discussed the changes in the supply and demand structure brought about by HBM4. In the past, memory was a standardized product. Manufacturers produced it first, and customers then purchased it. Buyers dictated prices, and memory manufacturers bore the inventory risk. This was the essence of the "memory cycle." However, starting with HBM4, because it required custom design based on the accelerator architectures of customers like NVIDIA, Google, and AMD (i.e., "custom HBM"), memory manufacturers had to obtain quantity commitments from customers at the beginning of R&D before starting development—this is the so-called "long-term agreement." "AI companies desperately need high-performance HBM, so they're lining up. Suppliers are starting to set prices; this is a paradigm shift." He also anticipates that future HBM chips will integrate communication functions, enabling "HBMs to communicate with each other," forming an alliance-like structure: "We communicate with each other, and whoever treats us better gets more memory; disobedient GPUs won't be allocated any." This further elevates the systemic status of memory manufacturers. VI. Samsung and SK Hynix are the only companies capable of doing both simultaneously. Kim Jung-ho repeatedly emphasized in the interview that globally, only Samsung Electronics and SK Hynix are currently capable of simultaneously mass-producing DRAM (HBM) and NAND flash memory (HBF). "SanDisk and Kioxia, despite their soaring stock prices, can only produce HBF (Heat-Free Fluid), not HBM (Heat-Free Microelectronics). Samsung and SK Hynix possess the most powerful tools to lead the future." When asked if the prediction of Samsung and SK Hynix's combined operating profit of 500 to 600 trillion won this year was realistic, Kim Jung-ho replied, "Realistic." He added that he frequently engages in technical exchanges with executives from both companies, "and their eyes are getting brighter and brighter." However, he also pointed out that competitive pressure is real, with Micron and SanDisk receiving orders from Nvidia and Google. VII. AI PCs and AI Phones: Memory Determines Device Prices. Kim Jung-ho further extended the narrative of memory demand to terminal devices. He predicts that for future AI PCs to truly achieve personal AI computing, the required memory will make "the price of a PC reach 10 million won, with memory prices determining the PC price." Of the 3 million to 5 million won price tag for AI smartphones, 2 million to 3 million won will be for memory. "The continuous evolution of AI infrastructure and AI models requires increasingly more memory. AI PCs and AI phones are another main trend in this direction." VIII. Agentic AI and Physical AI: Memory Demand Will Increase 1000 Times Kim Jung-ho's assessment of the direction of AI evolution is also noteworthy. He believes that with the advent of Agentic AI and Physical AI, memory usage will be approximately 1000 times higher than it is now. "AI agents work 24/7, unlike humans who need to sleep. Their workload increases dramatically, naturally leading to an explosion in memory demand. That's when it won't be HBM anymore, but rather the era of 'Super HBM.'" IX. The Path of Research: 50 Years of Accumulation, and the "Luck" Theory At the end of the interview, Kim Jung-ho traced his academic path. He received his PhD in 1993, specializing in femtosecond-level ultrafast electrical signal measurement; his advisor had won the Nobel Prize in Physics several years prior. In 1994, he joined Samsung Electronics' memory division, returning to KAIST in 1996. He then dedicated himself to fundamental research in memory and HBM for approximately 10 years before developing commercial products. In 2015, he first heard the term "deep learning" at an on-campus conference and immediately realized that AI algorithms and HBM architectures used the same set of mathematics—linear algebra and matrix operations. "I was particularly fond of matrices in my sophomore year of college, and it just so happens that both sides use the same mathematics—that's just luck." He chuckled, saying that when he first developed HBM, he envisioned it being used in televisions to make the picture more vivid, never imagining it would become the infrastructure of the AI ​​era: "I didn't know then, so you could call it luck." The following is an abridged transcript of the interview (translated with AI assistance) Kim Jung-ho: HBM, HBF, and HBS will form a hundred-story building, with the GPU located on the top floor for heat dissipation, etc. I believe that this 3D semiconductor structure is an inevitable architecture for future AI computers. One of the most difficult technologies is power supply. It requires supplying thousands of amperes of current, so designing the power supply network will be the most challenging aspect. This will become a core technological competitive advantage. Host: Professor Kim Jung-ho of KAIST, known as the "Father of HBM," has joined our program. Hello! Kim Jung-ho: Hello, it's a pleasure to meet you. Thank you for your invitation. Host: Thank you for taking the time. Kim Jung-ho: You're welcome. (Laughter) Host: We should start by talking about HBM. Actually, HBM has only been in mass production and application for about two years, right? That's true for HBM3. As for HBM1, I've been involved with SK Hynix since the 2010s. At that time, NVIDIA and AMD were in the GPU field. So HBM1 started in the early 2010s, but at that time it was used for graphics cards. Host: Professor, you received your doctorate in the 1990s, right? Kim Jung-ho: Yes. Host: But you started your research early on, back in 2010 when HBM was first developed. Kim Jung-ho: Yes. I received my PhD in 1993, and my research at that time was more focused on physics. I built what was then the world's fastest oscilloscope that used lasers to measure electrical signals. My advisor won the Nobel Prize in Physics a few years ago. The device I built could observe extreme time phenomena at the femtosecond level (almost a standstill of light). Now, with the development of AI, the need to process massive amounts of data means that digital circuits operate at picosecond or even femtosecond speeds. So, my research from 30 years ago is now proving useful. However, my research at that time was very narrow and in-depth, while my personality is more inclined towards social interaction and communication. So, I thought that memory would become very important in the future. With this in mind, I joined Samsung Electronics' memory division in 1994. Since then, I have been studying and researching memory. I came to KAIST in 1996. Around 2010, the basic research on HBM continued for about 10 years before it was applied to products. The various technologies required for HBM, such as quantum mechanics, semiconductor physics, and mathematics, were actually subjects I studied in my second or third year of university. In particular, it requires a lot of linear algebra knowledge, which I learned in 1981 and can still apply today. HBM is constantly innovating; our lab even proposed a 30-year roadmap up to HBM8. So, from the initial research to now, it's been almost 50 years. Host: When you first researched and conceived the concept of HBM, did you anticipate the arrival of the artificial intelligence era and that HBM would become its core? Jin Zhenghao: No, at that time, AMD and NVIDIA intended to use it in graphics cards. The mathematics required for graphics cards is the same as the mathematics required for artificial intelligence. So HBM later became a core component of AI, but initially NVIDIA thought it would only be used in graphics cards. At the time, I thought that South Korea's television industry was very developed, so I wanted to put this chip into televisions to make the picture more gorgeous, vivid, and realistic. Therefore, I initially considered using it in televisions. Around 2015, during a meeting with some young professors at the university, they used the term "deep learning," which was in the early stages of AI. At the time, I just thought, "Oh, there's such a technology," and chatted half-jokingly, only I didn't understand. So from then on, around 2015, I actually shifted my professional focus to AI. Although ostensibly working in a lab researching HBM, I personally completely switched to AI research starting in 2015. After several years of research, I discovered that AI algorithms and HBM were a perfect match. I felt then that this would lead to explosive applications in the field of AI. Back then, it was mainly used in CNNs (camera object recognition), and later in reinforcement learning (like playing Go). These applications all require a lot of matrix operations, so HBM is needed. But the kind of explosive growth we see now probably started in the early 2020s with the emergence of ChatGPT. Future AI will evolve towards Agentic AI, and some will also move towards Physical AI. Algorithmically speaking, the memory usage of Agentic AI or Physical AI might increase by 1000 times compared to now. That would require an upgraded version of HBM, "Ultra HBM." So we also have some other ideas. In short, I didn't know it would turn out this way at first; you could say it was luck. Because I really liked linear algebra in my second year of university, and both use the same mathematics. Host: I understand HBM to be like stacking multiple DRAMs together. Is my understanding correct? Kim Jung-ho: Yes, correct. Whether it's a graphics card or AI, computation requires rapidly reading data from memory. HBM is necessary for two reasons. First, it needs to have a large capacity. Especially as AI evolves towards contextual engineering, multimodal computing, and Physical AI, the amount of data accumulating in memory is increasing exponentially. It might double every year, resulting in a 1000-fold increase in ten years. Increasing memory capacity requires continuously shrinking transistors or storage cells, but due to interference and leakage between cells, we've approached the limits of quantum mechanics, making further shrinking difficult. Therefore, increasing capacity is challenging. Therefore, in the early 2000s, I believed that future memory must be stacked. From then on, we advocated for "stacked" rather than "planar." At the time, most people designed single-layer semiconductors, while our design direction was stacked. Of course, we focused on design, while Samsung and SK Hynix handled the implementation, but the final product was HBM. The second reason is that even with large capacity, data must be transferred to the GPU quickly. This is necessary for rapid response, processing documents, text, and even, more recently, film production. To increase speed, parallel data transfer technology is needed. Just like a highway that went from 8 lanes to 1024 lanes, and recently to 2048 lanes, and may become a million lanes in a few years. So the core of HBM is: increasing capacity through stacking, while simultaneously transmitting data at the speed of light (thousands or millions of times faster than traditional memory) by installing "elevators" and "highways" structures—this is the so-called parallel structure. Host: When mentioning HBM, we often hear about HBF. What is HBF, and how is it different from HBM? Jin Zhenghao: There are two main types of general-purpose memory: DRAM and NAND Flash. DRAM is fast but cannot be stored for long periods; while NAND Flash has a large capacity (about 10 times that of DRAM), is slower, but can be stored for long periods, and is mainly used in devices such as cameras. However, even though HBM stacks, its capacity is still insufficient. Recently, due to context engineering, input to AI is not only text, but also includes reference documents, YouTube videos, etc., resulting in a surge in video and image files, requiring even more memory capacity than it does now. Intermediate results (KV Cache) during the computation process also need to be fully stored. In the era of Agentic AI, I might hire 10 or 100 AIs to work for me. The workload of AI is 100 times that of me, and they work 24 hours a day, unlike us who sleep and rest. Therefore, the workload increases dramatically, and memory demand increases accordingly. Even with stacked DRAM, the capacity is still insufficient, so the idea of ​​stacking NAND Flash was conceived—this is HBF. Currently, companies developing HBF include SK Hynix, Sandisk, and Samsung Electronics. Japan's Kioxia may also be developing it. Recently, Kioxia's market capitalization even surpassed Toyota's, becoming the top stock in the Japanese stock market. The stock prices of Micron and Sandisk, which manufacture NAND Flash or HBF in the US, have also continued to rise, while Samsung and SK Hynix, which manufacture these in South Korea, rank among the top in market capitalization. There are two types of memory adjacent to the GPU: HBM and HBF, also called "hot memory"; while devices used for long-term recording of AI information about users are called "cold memory," and the demand for both is increasing. In the long run, about 10 years from now, the market demand growth for NAND Flash and HBF may surpass that for HBM. Therefore, although we are currently in the HBM era, Samsung and SK Hynix should also prepare for the HBF era; this is my opinion. Host: You mentioned that HBM might reach its eighth generation around 2038. Kim Jung-ho: Yes. Host: At that time, both HBM and HBF will enter the commercialization stage. Will they be complementary or competitive? Kim Jung-ho: They are complementary. HBM4 will be launched this year, and HBM5 will come out in a few years. There will be a generation change approximately every three years, and in 10 years, we will reach HBM8. At that time, HBM and HBF will be used together. HBM has a smaller capacity but is faster, while HBF is slightly slower and has some physical limitations, but has a huge capacity. If HBM capacity is insufficient, it will be paired with HBF. The two don't exist independently, but rather resemble an apartment complex: a department store (HBM) at the center, surrounded by apartment buildings (HBF). Various forms of HBM and HBF will form a complex, interconnected, providing data to users. In terms of total capacity, HBF may be larger than HBM. Host: Ultimately, it comes down to the difference between stacked DRAM and NAND Flash; both are indispensable. Kim Jung-ho: Yes, globally, only Samsung Electronics and SK Hynix can produce both. While Sandisk and Kioxia's stock prices have soared, they can only produce HBF (or ESSD technology for stacked NAND), not HBM. Therefore, I believe Samsung Electronics and SK Hynix possess the most powerful tools to lead the future. Host: So, can we say that Samsung Electronics and SK Hynix have an absolute leading advantage? Jin Zhenghao: You could say that. Didn't the stock price break through 9000 this morning? Although predicting stock prices isn't my area of ​​expertise, fundamentally speaking, the world is moving towards an era of AI hegemony, and I believe that AI's capabilities are determined by memory capacity. Until last year, I thought AI capabilities originated from mathematics (such as attention mechanisms), but achieving them is inseparable from memory. Ultimately, memory performance is AI performance. Therefore, I define "AI = Memory." AI companies, AI nations, or even those building data centers using semiconductors, must all rely on memory companies. This is an era of shifting power. Even more astonishingly, HBM and HBF, used to build AI data centers, are now also called "AI factories"—factories that manufacture AI. I call them "memory factories." The core of an AI factory is memory; the amount of memory a nation possesses determines its AI hegemony and the competitiveness of AI companies. Which is better: Google, Gemini, OpenAI, or Anthropic Claude? My argument is that it's determined by memory. Recently, to protect personal information, there has been a trend of computing AI directly on one's own computer, called AIPC. NVIDIA also wants to do this, collaborating with TSMC to manufacture PCs with 128GB of LPDDR memory and other massive amounts of RAM. To truly perfect it, it might require terabytes of RAM, making the PC cost 10 million Korean won. The price of RAM determines the price of the PC. Future smartphones will also become AI smartphones, with only one window on the screen, and everything else handled by AI. There might even be AI glasses. I believe that more than half the price of an AI phone will be the RAM cost; for example, in a 3 million or 5 million Korean won phone, 2 million or 3 million will be RAM costs. The more AI infrastructure and AI models develop, the greater the demand for RAM will be, and AI PCs and AI phones represent another major growth axis. Host: Among the current global tech giants, NVIDIA demonstrates overwhelming performance. What is the biggest secret to its continued dominance? Jin Zhenghao: Until last year, AI's "learning" (training) was more important; learning ability was AI capability. In learning, the encoder part of the Transformer model mainly performs backpropagation calculations, involving differentiation, which is done well by GPUs. Therefore, the training era was the era of GPUs, because doing AI requires GPUs, so everyone rushed to buy them at high prices. But since last summer, "inference" has become more important. Training alone cannot overcome the "illusion" problem; if it gives absurdly wrong answers, it becomes unusable. To achieve personalized AI, inference becomes important, and the semiconductor that is even more important for inference is memory. Therefore, in the inference era, memory will be more expensive and in greater demand than GPUs. Another reason is that to improve GPU performance, the GPU area must be increased (to fit more calculators). One way is like Cerebras, making the entire 12-inch wafer a GPU. But this is difficult to manufacture; a single defect would require discarding the entire wafer, which is uneconomical and limits its applications. Even so, Cerebras still relies on HBM and HBF; without memory, it would be very weak in the inference era. So, can NVIDIA stack GPUs? No, because it gets too hot; a cooler would be needed, making stacking impossible. Therefore, GPUs seem somewhat constrained. Recently, Jensen Huang has been restless, appearing on television in South Korea, throwing baseballs, eating fried chicken and drinking beer, and meeting with many people, indicating he's not at ease. One reason is that I believe GPU technological growth has almost stagnated. Conversely, the growth and evolution of AI computers depends on memory. Host: There's a saying that only 10% of GPUs are actually running? Kim Jung-ho: Yes. Even with a million GPUs installed, the actual working time might only be 20%, or even 10%. Why? Because GPUs need to retrieve data from memory to perform calculations and return results, but data cannot be transferred from memory (HBM/HBF). When ChatGPT rapidly outputs words, it needs to read data from HBM/HBF, perform calculations, and write it back every instant. Almost all the time is spent on reading and writing, while the GPU waits. Therefore, the key is the speed and quantity of data read, which is why HBM and HBF are needed. No matter how the algorithm is improved, the GPU may only be working at most 30% of the time, with the rest idling. Host: So, Professor, you advocate that in the future, GPU functionality will be integrated into HBM or HBF, ushering in a new era? Jin Zhenghao: Yes. Since the GPU is waiting for data from HBM/HBF, why not perform the calculations ourselves? It's like installing a GPU on the first floor of an apartment building; the data is processed down in the elevator, solving everything within the entire building, saving time and effort. Therefore, I advocate putting CPU/GPU functionality into HBM, even allowing the GPU to "take a backseat." Of course, the GPU shouldn't be completely idle; it needs appropriate division of labor to keep it "always eager." This is what I call "Memory-Centric Computing." We've been working in this direction since HBM4. Host: Even though GPU functionality is integrated into HBM/HBF, since multiple GPUs aren't stacked, shouldn't there be no heat dissipation issues? Kim Jung-ho: There will still be some heat dissipation issues. So, starting with HBM4, the performance of products manufactured by SK Hynix and Samsung might differ, and this is related to heat dissipation—the ability to effectively dissipate heat. Because some GPU functionality is integrated on the first floor (memory layer), it gets too hot there. The memory is like sitting on a heated kang (a traditional heated platform), and its performance will decrease; the kang must be cooled down. Who can cool it better will determine the performance differences in HBM4 and later products, and the same applies to GPUs. So, our lab's idea is that since the first floor is too hot, why not move some functionality to the "roof" (top floor) and install cooling towers there for direct cooling from the top? This is one of our core architectures, and currently, master's and doctoral students are conducting this research in HBM5-related studies, hoping for great success. After we publish these papers, NVIDIA, AMD, Samsung, and SK Hynix will see them. Initially, they might resist, but finding no other way, they will eventually adopt them. Host: If the future of integrated GPUs within HBM/HBF, as the professor mentioned, arrives, and even CPUs are integrated later, then Samsung Electronics and SK Hynix should develop even better, right? Kim Jung-ho: Yes, the opportunity is coming. "Developing better" means gaining more control, and possibly even surpassing NVIDIA. But to achieve this, we need technological development, investment, talent cultivation, as well as sound policy judgment and open-minded and correct judgment from management. Management's judgment is the most important. Host: The professor advocates that we are about to enter the memory era rather than the GPU era, and this seems to have already begun. Also, while GPUs are gaining momentum recently, NPUs have also emerged. What is an NPU? Jin Zhenghao: They are all processors used for matrix calculations and AI. GPUs were originally GPGPUs, and TPUs also contain HBM, so they all rely on HBM and memory. Gemini can write articles, process language models, and draw, offering diverse functions; while some chips are only good at writing articles, simplified for specific purposes, which is the NPU. Some people also call it LPU. They are all calculators needed for AI, made smaller, with lower power consumption, and lower cost according to special uses. There are companies like Rebellions, FuriosaAI, and HyperExcel in China, and about a dozen companies worldwide that make NPUs, but whether it's Rebellions or FuriosaAI, they must use HBM for high performance. Host: Recently, FuriosaAI and Rebellions received large-scale investment from the National Growth Fund, which is meant to allow them to truly compete with NVIDIA. Are these two companies truly globally competitive? Jin Zhenghao: I was one of the review committee members at the time. This decision was based on the following considerations: NVIDIA cannot control all sectors globally; there are definitely niche markets for NPUs, TPUs, etc. For example, if Saudi Arabia were to build a data center and rely entirely on American products, the dependence would be too high. Therefore, it's possible that 10% of the components would use alternative solutions, and South Korean NPU companies could be candidates. Additionally, if South Korea were to build AI data centers (potentially requiring millions of devices), and rely 100% on NVIDIA chips, our dependence on overseas markets would be too high; we need to cultivate local companies. Therefore, the decision was made to invest in nurturing domestic companies. That's the general overview. It also has its technological advantages.

Host: Professor, in your recent research, you proposed the concept of "High Bandwidth SRAM (HBS)"?

Kim Jung-ho: Yes, this is a new concept I recently proposed. As mentioned before, I propose concepts, but realizing them requires significant effort from companies like Samsung and SK Hynix. These concepts often have a major impact 10 or 20 years from now. I've mentioned Cerebras, which has huge GPUs, and the US also has chips called LPUs.

To maintain their image or reduce reliance on HBM, they integrated SRAM as memory within the GPU. SRAM is about 1000 times faster than DRAM, but has a smaller capacity. I researched it, and both Cerebras and LPUs face the problem of insufficient SRAM capacity. From what I understand, a Cerebras chip made entirely from a 12-inch wafer only has 44GB of SRAM, while I think at least 400 to 440GB would be necessary. So my idea is: to create a chip that fills an entire 12-inch wafer with SRAM, and then stack it 10, 12, or 16 layers. This way, 100GB becomes 1600GB, an astonishing capacity. Then, place the GPU on this wafer-level SRAM stack. A thousand times faster, with sufficient capacity—this idea sounds feasible. Therefore, I call this wafer-level SRAM HBS. My future dream is that HBM, HBF, and HBS will all become 100-story buildings, with the GPU on the top floor and the cooling system integrated together. This 3D semiconductor structure will inevitably become the architecture of future AI computers. This may take 10, 20, or even 30 years. One of the most difficult technologies is power supply. Stacking GPUs on top of HBS and HBM requires supplying thousands of amperes of current. The design of the power supply network will be the most difficult part, and this will become a core competitive advantage. SK Hynix, Samsung, Micron, and TSMC are all in the same boat. Secondly, there's the issue of heat dissipation, which is an obstacle in the implementation process. Currently, people are focused on who does better in the nanometer process and what their yield rate is, but in the future, for 3D AI computers, including HBS, how to supply power and how to cool them will determine the survival of companies. Host: HBS is practically the "Hwang Jung-min" of the memory semiconductor field (a metaphor for a big name). Kim Jung-ho: Yes, it's Hwang Jung-min. I heard about Cerebras using 12-inch wafers for GPUs 10 years ago, and I thought, "What? Where can this be used?" Probably for defense AI. I was quite arrogant back then. But two weeks ago, the company went public on Nasdaq, which changed my mind. It does have its uses. Since the biggest weakness of Cerebras chips is insufficient memory, let's stack it up too. I had this idea one morning and had my students draw the diagrams. Recently, we've started talking about HBF, and when the new master's students enroll this year, I plan to have them start using HBS as their master's and doctoral thesis research direction. Host: Who manufactures the SRAM? Kim Jung-ho: It's manufactured by foundries; TSMC and Samsung Electronics both do it. Host: Samsung and SK Hynix's combined operating profit this year is said to be between 500 and 600 trillion won. Is this a realistic target or an overly optimistic forecast? Kim Jung-ho: I think it's realistic. I often have technical meetings with executives from Samsung and SK Hynix, and I feel their eyes are getting brighter and brighter. Although they don't talk to me about specific sales figures. A key feature of HBM and HBF now is "customized HBM." Previously, it was about manufacturing standardized products, mass production, and price fluctuations depending on how much customers bought—this was called a "cycle." Memory manufacturers didn't have a say; it was CPU manufacturers, Microsoft, or computer manufacturers who decided the purchase quantity. We could only produce more and wait and see. If customers didn't buy, the inventory pressure would fall on us—that's the "memory cycle." But starting with HBM4, not only is GPU functionality integrated, but another important function is that HBMs can communicate with each other. Previously, we only handled GPU instructions; now, we advocate communication between them as well. In the future, HBMs can compete with each other, allocating more memory to the better-performing HBMs. In other words, they will form internal combinations, preventing the less performing HBMs from transferring data to the GPU. In short, with the addition of these algorithms, communication functions, and GPU functions, each company (Google, AMD, NVIDIA) has different design requirements for HBMs, resulting in customized HBMs. This leads to long-term supply agreements (LTAs) being signed early in development; development doesn't begin without orders. Currently, AI companies desperately need high-performance HBMs, so they are queuing up to buy them, turning the market into a seller's market where suppliers set prices. This is a paradigm shift. Host: So far, we have had a dialogue with Professor Jin Zhenghao of KAIST about the semiconductor ecosystem. Thank you for sharing today. Jin Zhenghao: Thank you.