News from the IT Home on October 24th: Andy Jassy of Amazon believes that the inference costs of AI are still too high. To enable AI to be widely used in enterprise software, the costs need to be reduced to about one-tenth of the current level. Amazon's self-developed chips are expected to help lower these costs.
In an interview released on the 23rd local time at the venture capital firm a16z, Jia Xi revealed: "In my opinion, to enable AI to be used as widely as possible, the inference cost is still an order of magnitude too high."
Amazon plans to invest $220 billion (Note from IT: the current exchange rate is approximately 1.47 trillion yuan) in capital expenditures this year, yet it still cannot build enough infrastructure to meet the demand of AI. Jassy revealed that the main capacity supply for 2026 is basically fully booked, most of 2027 has also been reserved by customers, and a considerable portion of 2028 has also been sold out. In his view, with such strong market demand, even if Amazon invests more substantial funds, it would not be considered a waste.
Jiaxi compared the AI requirement to a "barbell": one end is the AI laboratory, which consumes an unprecedented amount of computing power; the other end is corporate clients, who mainly use AI for tasks such as customer service and fraud detection.
What has truly not been fully developed yet is the intermediate market, namely the various software programs used by businesses on a daily basis. These software programs have not yet been widely integrated with AI, but Jia Xi believes that the greatest future demand for AI may come from this area.
The main factor that restricts enterprises from adopting AI is the reasoning cost. The AI model consumes computing power for each answer it provides and each task it completes. If the benefits brought by introducing AI are not sufficient to offset the higher operating costs, enterprises will have little incentive to restructure their existing software.
Jia Xi compares it to Amazon's experience from several years ago. At that time, Intel dominated the cloud server processor market. Amazon wanted to achieve higher computing performance at the same cost but found it difficult to persuade Intel to prioritize this need, so they ultimately decided to develop their own Graviton processors. In his view, "the same story is playing out again in the AI field."
The chip market is currently dominated by NVIDIA. Amazon, on the other hand, has launched its own Trainium chips in an effort to provide customers with a lower-cost option. Jiaxi did not directly mention NVIDIA in the interview, but emphasized that the dominant supplier in the market remains an important partner for Amazon.
According to the introduction, the cost-performance ratio of Trainium is about 30% to 40% higher than that of similar processors. Customers can obtain more computing power for the same amount of money. 'Our chip business plays an important role in reducing inference costs.'
Amazon previously disclosed that Trainium has received future revenue commitments of over $225 billion (approximately 1.5 trillion yuan at current exchange rates), which includes agreements reached with Anthropic and OpenAI.
