The affordability of DeepSeek is a myth: The revolutionary AI actually cost $1.6 billion to develop

DeepSeek's new chatbot boasts an impressive introduction: "Hi, I was created so you can ask anything and get an answer that might even surprise you." This AI, a product of the Chinese startup DeepSeek, has quickly become a major player, even contributing to a significant drop in NVIDIA's stock price.

DeepSeek's success stems from its innovative architecture and training methods. Key technologies include:

• Multi-token Prediction (MTP): Predicting multiple words simultaneously, boosting accuracy and efficiency.
• Mixture of Experts (MoE): Utilizing 256 neural networks (eight activated per token), accelerating training and improving performance.
• Multi-head Latent Attention (MLA): Repeatedly extracting key details from text, ensuring crucial information isn't missed.

While DeepSeek initially claimed a remarkably low training cost of $6 million for DeepSeek V3 using 2048 GPUs, SemiAnalysis revealed a far more extensive infrastructure: approximately 50,000 Nvidia Hopper GPUs (including 10,000 H800, 10,000 H100, and additional H20 GPUs) spread across multiple data centers. This translates to a total server investment of roughly $1.6 billion and operational expenses estimated at $944 million.

DeepSeek, a subsidiary of the Chinese hedge fund High-Flyer, owns its data centers, providing control over optimization and faster innovation implementation. Its self-funded nature enhances flexibility and decision-making speed. The company also attracts top talent, with some researchers earning over $1.3 million annually, primarily recruiting from leading Chinese universities.

DeepSeek's initial $6 million figure only represents pre-training GPU costs, excluding research, refinement, data processing, and infrastructure. The company's total investment in AI development exceeds $500 million. Its lean structure, however, allows for efficient innovation implementation compared to larger, more bureaucratic companies.

DeepSeek's success highlights the potential of well-funded independent AI companies to compete with industry giants. However, its achievements are rooted in substantial investment, technological breakthroughs, and a strong team. The "revolutionary budget" claim for model development is, therefore, an oversimplification. Nevertheless, its costs remain significantly lower than competitors; for example, DeepSeek's R1 model cost $5 million to train, compared to ChatGPT4's $100 million.