Photo from Reuters
Ho Chi Minh city, 30 May 2023 8:am
Google's CEO has drawn a parallel between the artificial intelligence revolution and humanity's discovery of fire, emphasizing their comparable transformative impact. However, there is a pressing need for the industry to acquire the necessary resources to sustain this revolution, much like securing the fuel to keep a fire burning.
The scarcity of advanced chips, crucial for powering innovative generative AI systems, has triggered a competitive pursuit to secure computing power and explore alternative solutions. Nvidia predominantly dominates the market for graphics chips (GPUs) used in AI, but the overwhelming surge in demand, fueled by the viral popularity of ChatGPT—a chatbot capable of humanlike question responses—has far exceeded the available supply.
Sharon Zhou, co-founder, and CEO of Lamini, a startup specializing in assisting companies with AI model development, highlighted the significance of connections in navigating the chip shortage. Zhou compared the situation to the scarcity of toilet paper during the pandemic, emphasizing the importance of having the right contacts to secure the necessary resources.
The chip shortage has limited the processing capabilities available to cloud-service providers like Amazon.com and Microsoft, affecting their ability to offer optimal resources to clients such as OpenAI, the organization responsible for ChatGPT. AI developers require ample server capacity to develop and operate their progressively intricate models, as well as to assist other companies in constructing AI services.
Tech entrepreneurs with extensive connections are encountering difficulties in obtaining sufficient capacity. At a congressional hearing on AI held on May 16, OpenAI CEO Sam Altman acknowledged the challenges posed by the processor bottleneck. Altman expressed the view that it would be more advantageous if there were fewer users of ChatGPT to alleviate the strain on processors.
Elon Musk, speaking at the Wall Street Journal CEO Council Summit on May 23, made a comparison, stating that acquiring GPUs at present is significantly more challenging than obtaining drugs. However, being Elon Musk has its advantages. Earlier this year, several startups seeking Oracle computing capacity were surprised to learn that a buyer had already acquired a significant portion of Oracle's available server space. The startups were informed that the buyer in question was Elon Musk, who is reportedly developing his own OpenAI competitor named X.AI.
For companies involved in training extensive AI models capable of generating original text and analysis, having access to tens of thousands of advanced graphics chips is of utmost importance. The founders emphasize that the absence of these chips significantly slows down the progress of developing the large language models that drive AI advancements. Nvidia's high-performance graphics chips are particularly adept at simultaneously executing numerous computations, a vital requirement for efficient AI tasks.
According to UBS analysts, a previous iteration of ChatGPT was estimated to have utilized approximately 10,000 graphics chips. Elon Musk, the prominent figure behind Tesla, SpaceX, and Twitter, estimates that an updated version of ChatGPT would require three to five times more of Nvidia's advanced processors. Elon Musk engaged in a discussion about technology with Thorold Barker from The Wall Street Journal.
In response to the increasing demand, Nvidia recently announced plans to expand its chip supplies. Jensen Huang, the CEO of Nvidia, stated that the company has significantly increased production of the H100, their latest and most advanced chip designed for generative AI applications. Speaking at a trade show in Taiwan, Huang mentioned that the demand for Nvidia's chips is coming from all parts of the globe. He also unveiled several new AI-related initiatives during the event.
The high demand for Nvidia's products has resulted in a significant increase in the company's stock value, approximately rising by 167%. While the costs of chips can vary, Nvidia's advanced AI chips are typically priced around $33,000 when sold by certain retailers. However, due to the intense demand, these chips can fetch even higher prices in secondary markets.
Certain investors are tapping into their networks to locate available computing power, while others are coordinating large-scale orders of processors and server capacity that can be shared among their AI startups. Startups are taking measures such as optimizing their AI models for greater efficiency, procuring their own physical servers equipped with suitable graphics chips, or temporarily shifting to less popular cloud providers like Oracle until the chip shortage is resolved. These observations are based on insights provided by AI investors and startups.
In some cases, founders are resorting to imploring salespeople at Amazon and Microsoft for increased computational power.
Zhou, co-founder of Lamini, a company she established alongside a former Nvidia engineer, stated that they have managed to acquire the necessary chips. However, when asked about the specifics of how they obtained them, Zhou and numerous other founders interviewed by the Journal opted not to disclose the precise details.
OpenAI's spokesperson acknowledged the robust demand for GPUs in the industry and emphasized the company's commitment to ensuring that its customers have the necessary capacity to meet their requirements.
Oracle and Elon Musk did not provide a response when requested for comment. Microsoft and Amazon declined to comment as well. Numerous AI founders anticipate the shortage will likely continue until at least the following year.
According to Adam Wenchel, CEO of Arthur, a company specializing in AI risk mitigation tools like data leak protection, certain companies are reserving cloud capacity due to concerns about future accessibility. Even if they currently don't have a need for it, these companies are choosing to continue paying for the reserved capacity as a precautionary measure.
Founders and investors highlight that even companies that successfully secure computing power may experience significant waiting periods before being able to utilize it. Aravind Srinivas, CEO of Perplexity AI, which develops AI-driven conversational search tools, emphasized that upfront payment does not guarantee immediate availability of GPUs. Instead, companies must exercise patience and wait for their turn to access the computing power, which may take weeks.
Server manufacturers and their direct customers are reporting delays of over six months in obtaining the latest graphics chips from Nvidia. The CEO of Supermicro, a prominent server manufacturer, stated that the company's back-orders for systems with graphics chips have reached an all-time high. As a result, Supermicro is making urgent efforts to expand its manufacturing capacity in response to the overwhelming demand.
As a result of the high demand and shortage of advanced chips, a secondary market has emerged. This market involves large cryptocurrency companies that initially purchased the chips for mining purposes during the boom but no longer require them due to the downturn in the digital currency market. Kanjun Qiu, the CEO of Generally Intelligent, an AI research company, has been procuring advanced graphic chips for her own servers since last year, enabling her to navigate through the current shortage. Recently, a venture capitalist approached Qiu, inquiring about the possibility of renting spare capacity from her chips for other startups. However, Qiu has yet to make a decision regarding whether she will part with her chips.
In the meantime, OpenAI's CEO Sam Altman and his colleagues have been handling grievances from companies that are constructing AI services using OpenAI's platform.
According to Alex Lebrun, CEO and founder of Nabla, a company that created an AI assistant for physicians, OpenAI's software sometimes takes as long as two minutes to provide responses to queries. Nabla utilizes AI technology to generate notes, referral letters, and clinical recommendations automatically, and their customers anticipate instantaneous production of these notes.
To address the issue, Nabla has implemented a solution by developing simpler models that can swiftly generate initial drafts of the material, as explained by Lebrun. They subsequently utilize the latest version of ChatGPT to make final adjustments. Lebrun emphasized that successful startups are the ones who adeptly navigate and find ways to overcome such limitations. He also mentioned that he has directly communicated these challenges to Altman, expressing the concerns to OpenAI's CEO.
Altman and other representatives from OpenAI have assured founders that the company is actively collaborating with Microsoft, its largest investor, and provider of data-center services, to tackle and resolve the issue.