Semiconductors are materials that have the ability to conduct electricity under certain conditions. They are an essential component of modern electronic devices, and their importance to the world cannot be overstated. And it has only risen over the past years. And AMD Chiplets will be one of the keys to unlock the future of compute.
Some people say that semiconductors are the modern equivalent of oil in terms of importance and potential implications for geopolitics. Well, let’s ask ChatGPT what does it think:
What is more important? Oil or Semiconductors?
It is difficult to say which is more important, oil or semiconductors. Both are essential for the global economy and have significant implications for geopolitics.
Oil is a vital source of energy and is used in a wide range of industries, from transportation and manufacturing to heating and electricity generation.
Semiconductors, on the other hand, are essential for modern electronic devices, and are the foundation of the global digital economy. Without semiconductors, electronic devices such as computers and smartphones would not be able to function, and the global economy would be severely impacted.
ChatGPT
In terms of importance, the relative importance of oil and semiconductors may vary depending on the context. For example, in countries with large oil reserves and industries, oil may be considered more important, while in countries with a strong digital economy and technology sector, semiconductors may be considered more important. Ultimately, both oil and semiconductors are essential for the global economy and play important roles in the world today.
We hope to never see the day wars will be fought over semiconductors, but it is not out of the realm of possibilities.
Why are semiconductors important?
One of the most important reasons why semiconductors are so important is that they are the foundation of the global digital economy. Almost every electronic device that we use today, from computers and smartphones to televisions and household appliances, relies on semiconductors in some way. Without semiconductors, these devices would not be able to function, and the global economy would grind to a halt.
Another reason why semiconductors are so important is that they enable the development of new technologies. Because they can be used to create efficient, small, and fast electronic components, semiconductors are the foundation of many emerging technologies, such as solar panels and medical devices. These technologies have the potential to improve our lives and reduce our impact on the environment, making semiconductors an essential part of the world’s future.
Semiconductors are also important because they are the driving force behind many of the world’s most innovative companies. Many of the world’s largest and most successful technology companies, such as AMD, Intel, Qualcomm, and TSMC, are in the semiconductor industry. These companies not only drive economic growth but also provide jobs and opportunities for people around the world.
In short, semiconductors are essential for the world because they are the foundation of modern electronic devices, they enable the development of new technologies, and they are the driving force behind many of the world’s most innovative companies. Without semiconductors, the global economy and the world’s technological progress would be severely hampered.
Is Moore’s Law Still Applicable?
Moore’s law is a prediction made by Intel co-founder Gordon Moore in 1965. He observed that the number of transistors on a microchip was doubling every year, and he predicted that this trend would continue for the foreseeable future. This prediction has largely held true, and as a result, Moore’s law has become a guiding principle for the semiconductor industry. It states that the number of transistors on a microchip will double approximately every two years, resulting in a corresponding increase in the performance and capabilities of electronic devices. This law has driven the rapid pace of technological advancement and has played a crucial role in the development of the global digital economy.
As the semiconductor industry moves beyond the 5nm domain, the complexity of chip production is increasing rapidly. Due to the way semiconductors are made, the industry is reaching the limit of what can be achieved with current lithographic techniques, making it exponentially more expensive and even physically impossible to produce cutting-edge chips. This dynamic has been quietly reshaping the industry for almost a decade, and the broader market is only now starting to realize the implications.
Are AMD Chiplets the Correct Path?
The goal in the semiconductor industry is to provide more computing power at a lower cost and with less energy consumption. To achieve this, the industry has traditionally focused on creating monolithic chips that pack more computing power into a single chip. However, AMD has been pursuing a different approach known as the “chiplet” route for the last decade. This approach, which involves using multiple smaller chips instead of one large chip, was initially seen as a silly idea by many, including Intel. But as the industry approaches the limits of what can be achieved with monolithic chips, the chiplet approach is starting to gain more recognition and acceptance.
In simple terms, Chiplets are small, modular chips that can be combined to create a larger, more powerful chip. This approach allows companies to use smaller, more specialized chips instead of one large, monolithic chip. Chiplet technology can provide a more efficient and cost-effective solution for the semiconductor industry as it continues to advance, as it allows companies to combine the benefits of small, specialized chips with the performance of a larger, monolithic chip.
There are several advantages to using chiplets in the semiconductor industry. Firstly, chiplets allow companies to create more specialized and efficient chips, as they can be designed to perform specific tasks. This can provide a performance boost compared to using one large, monolithic chip. Additionally, AMD chiplets can be more cost-effective than monolithic chips, as they can be produced using existing manufacturing processes, which can be less expensive than developing new processes for larger chips. Chiplet technology can also provide flexibility and scalability, as different chiplets can be easily combined and rearranged to create different configurations, allowing companies to quickly adapt to changing market conditions. Overall, the use of chiplets can provide a more efficient and cost-effective solution for the semiconductor industry as it continues to advance.
AMD chiplets can deliver almost the same performance as the best monolithic chips, but at a 40% lower cost. This is because if something goes wrong during production, you don’t have to throw away the entire chip; you can just replace the faulty chiplet(s). This leads to higher yields and greater organizational agility. As a result, AMD has been giving Intel a tough time in the CPU market, and the company has been reaching new highs on a regular basis.
The corporate culture and DNA required to produce chiplets successfully is different from what is needed for monolithic chips. It not only demands, but also enables a company to be more agile than monolithic chip producers. In today’s rapidly changing world, this agility is a valuable quality, and it will become even more important as the industry moves toward gen 4 data centers and beyond the 5nm domain.
Both Intel and Nvidia are now moving toward chiplets, but AMD has been working on them for about a decade. The experience and expertise that AMD has developed during this time will give it a significant advantage going forward. This is a textbook example of the “innovator’s dilemma,” where a non-incremental innovation gradually disrupts an industry, with incumbent players initially dismissing it before pivoting when it becomes a dominant force.
AMD had to overcome significant technical challenges to make chiplets a reality, which shows that the company has excellent management and culture. Intel and Nvidia are also great companies, but in the semiconductor industry, success depends on a combination of strong corporate properties and a successful technology roadmap. At its core, a company like AMD is just a group of people working together to design new chips. All they do is process information (electrons) to unlock novel atomic configurations that offer some kind of benefit, but someone has to decide on the company’s strategy for the next five years. The company has to make the right technological bets.
In fact, the main reason AMD struggled so much before Su took over was that it wasn’t making the right bets. The success we’ve seen so far is the result of the roadmap that Su and others put in place years ago. It shows an ability to allocate capital effectively, which is perhaps the most important attribute of the company. Pursuing the chiplet route was a great, contrarian decision. Now we have the same capital allocators, with more resources at their disposal, leading the way around Moore’s limit. The future is still very much open for AMD.
AI: Gradually, Then Suddenly
Many people are puzzled by Su’s decision to spend $49 billion in stock to acquire Xilinx and nearly $2 billion in cash to acquire Pensando. This is because the market typically views semiconductor companies as focused primarily on CPUs and GPUs, without considering computation at a fundamental level.
But the point of computation is to move electrons around and uncover insights. As more and more devices connect to the internet, CPUs and GPUs must be part of a highly connected and self-optimizing environments, such as a stateful and smart data center, in order to drive incremental productivity.
As we move into the future, many problems will become networking problems or will depend on how effectively we can manage electrons. Over the next few decades, much of the global economy will involve collecting data from endpoints and processing it to train AI systems and uncover insights that can drive value. This will transform the economy, as we move from being poor at making predictions to being excellent at it. The Xilinx and Pensando acquisitions position AMD well for this future.
Pensando Power
Pensando specializes in making data centers stateful, which is essential for AMD’s future because it provides the environment for their products to self-optimize and for the rest of the world to run Industry 4.0 infrastructures. Statefulness simplifies the complexity that arose with gen 3 data centers. To explain, data centers have been evolving for decades, and until 2010, traffic was mostly north-south, running on bare metal.
A request would come into the data center, and it would respond with a package of information. Since 2010, with the advent of gen 3 data centers, applications and servers have been virtualized, breaking them down into microservices. This has introduced a new kind of traffic, with information flowing among different components of the data center, dramatically increasing the complexity of managing them.
Statefulness eliminates this complexity, making it possible for “anyone” to become a hyperscaler.
When a data center is stateful, it means that information about itself flows freely, without using CPUs or GPUs. This makes their job easier. As the data center handles north-south and east-west workloads, it generates data that can be used to train machine learning algorithms to run various applications, such as XDR, analytics, and microsegmentation. This allows AMD to provide the environment that their computing units need to operate in for the next decade or two.
This is facilitated by Pensando’s hardware, specifically the DPU (data processing unit), but it is mostly enabled by Pensando’s Stateful Software Services.
Xilinx Conundrum
Xilinx and FPGAs are important because they can be reconfigured on the fly, unlike ASICs. This means that one FPGA can become any ASIC you want with just a bit of code, rather than requiring a full manufacturing process. ASICs and FPGAs are better than GPUs at making inferences from AI models, so you can add FPGAs to other computing units to give them inference capabilities.
Every deep learning (AI) model involves linear algebra during the forward pass (inference) and multivariate calculus during the backward pass (learning). While an FPGA may not be as good as a GPU at performing multivariate calculus, it excels at accelerating linear algebra. Different deep learning models perform different linear algebra computations, so you want a computing unit that can adapt at a low cost as you move from one model to another. Only FPGAs can do this, and Xilinx is the clear leader in the field.
Accelerating linear algebra in an automated way at a low cost is important because it will be a big part of the economy in the coming decades. Once we train an AI model with data, it can only be useful if we can make inferences with it at a low cost. FPGAs are poised to replace GPUs and ASICs for inference.
Synergies of CPUs and GPUs
AMD’s acquisitions of Xilinx and Pensando have positioned the company to provide not just the best CPUs and GPUs, but also the environment in which they can excel and be mixed and matched to address the needs of their customers. This includes the ability to become an inference giant and offer highly tailored products, such as infusing its CPU portfolio with Xilinx’s FPGA-powered AI inference engine. The company’s experience with connecting different computing units through chiplets and the AMD Infinity Fabric will be crucial in this next chapter.
In addition, the explosion of unstructured data and the extension of data center intelligence to the edge will require different types of computing units to be connected in the same device. AMD is well-positioned to address these developments with its technology and expertise.
AMD’s chiplet strategy has allowed the company to take market share from Intel. Although Intel may eventually catch up, it may take some time for them to do so. Investing in Intel’s upcoming foundries may be a good hedge in case things go wrong for AMD in Taiwan. This would provide exposure to semiconductor production and CPUs. Otherwise, investing in both companies could be a win-win situation, particularly at current prices.
The GPU space is becoming more competitive as players move towards a chiplet architecture similar to AMD’s strategy in the CPU space. AMD has not yet released GPUs based on this architecture, but the company has been integrating its CPU and GPU divisions for some time. AMD’s first chiplet-based GPUs are expected to be released in early 2023. If the company’s GPU division is managed as effectively as the CPU division, it is likely to see similar success in the market.
Angle of Tesla and NVidia
Tesla itself has adopted a chiplet architecture to train its AI systems. The company’s Dojo system is designed to maximize the runtime of its D1 GPU chips when running neural networks, allowing for better performance than existing GPUs. This is achieved through an effective method of connecting the tiles and spreading workloads, similar to AMD’s Infinity Fabric. This approach has produced impressive results, although it should be noted that it is only applicable to AI workloads, and not to other markets such as gaming and cryptocurrency.
Nvidia has been working on creating “super-chips” by combining multiple large chips together. This approach can provide strong performance, but as technology continues to advance and move down the nanometer curve, it may not be sustainable. Nvidia’s team is capable of finding a solution, but it will likely require a shift to a more modular approach. NVLink is Nvidia’s technology for connecting computing units, and it will be interesting to see if it can compete with AMD’s Infinity Fabric in terms of scalability.
Geopolitics Risk and Future of Compute
I do not share concerns about the export ban to China. Sharing too much technology with China while the CCP is in power is not a good long-term strategy, as they will use it to impose their own values and lifestyles on us. I am willing to sacrifice short-term gains in order to avoid this. Additionally, the demand for computing in 10 years will be significantly higher than it is today, regardless of the state of the economy. This will be true with or without China.
In the next 20 years, most things will be a matter of manipulating electrons and making inferences. We live in a universe with no material scarcity, just a lack of insights into how to access the abundance that exists. This is a game for AI, and it is likely that it will continue to run on AMD’s computing units because the company is making the right bets. Currently, the world’s total computing power is not enough to satisfy even 0.1% of its potential. This will lift humanity out of material scarcity and redefine how the world is run.
Performance per Watt
If AMD’s products continue to improve their computing performance per watt, the company is likely to do well financially. AMD focuses on the high-performance computing (HPC) space, so the performance of its products is more important to its customers than their price. However, total cost of ownership is still a key selling point in HPC, and providing more computing power for less power is the way to achieve this.
To provide some context, RDNA2 had more than 50% higher performance per watt than its predecessor RDNA, both of which ran on the same 7nm nodes. AMD is now targeting another 50%+ increase in performance per watt for RDNA3 (on 5nm nodes). By continuing this trajectory ahead of its competitors, AMD will be able to deliver exponentially more computation at a reasonable price in the near future. This rate of improvement is evident across its product portfolio, including CPUs and GPUs.
Fundamentally, AMD’s ability to connect different computing units makes it more likely that the company will continue to be successful with this metric in the future. This is because, as discussed in section 2.0, improving performance per watt over time is now not only about improving CPUs and GPUs, but also about being able to incorporate accelerators (FPGAs) and creating an environment where these units can learn to optimize themselves better than any human chip designer could. This agility enables AMD to provide the right solutions for the right markets at the right time.
The accelerators themselves must be able to keep up with the pace of learning and adapt accordingly, which is why the Xilinx acquisition is key. It allows AMD to generate ASICs on the fly at a low cost, as smart infrastructures based on Pensando’s DPU and stateful software services increasingly turn the management of data centers and the edge into an electron management problem. This will create an added moat that competitors will have a difficult time catching up with unless they develop similar ecosystems. The financials we see today do not yet reflect this future.
Conclusion: Is AMD Ready?
Semiconductors are essential for modern electronic devices and the global digital economy. They enable the development of new technologies and are the driving force behind many innovative companies.
Moore’s law, a guiding principle for the semiconductor industry, predicts that the number of transistors on a microchip will double every two years, leading to increases in the performance and capabilities of electronic devices.
However, as the industry moves beyond 5nm, the complexity of chip production is increasing rapidly, which may challenge the industry’s ability to continue following Moore’s law. To overcome this challenge, some companies, such as AMD, are pursuing the “chiplet” approach, which uses multiple smaller chips instead of one large chip. This approach may provide a more efficient and cost-effective solution for the industry as it continues to advance.
Overall, the world needs more semiconductors to support the growth of the global digital economy, enable the development of new technologies, and support the success of the semiconductor industry.
AMD is well positioned to get the most out of the world’s need for more computational power and is likely to benefit more than their competitors.
