Plenty of us are excited about new AI models but many of them are not open source. DeepSeek V3 is a new open source model with 671B parameters, trained on 14.8T high quality tokens. It features a multi-head latent attention for efficient inference. It uses the DualPipe algorithm for efficient pipeline parallelism.
LiveBench reported by r/LocalLlama – DeepSeek v3 is the BEST open weight LLM AND SECOND BEST non-reasoning LLM after `gemini-exp-1206` 🔥 https://t.co/OQMeuIxlBm pic.twitter.com/rbWu9jDB25
— Vaibhav (VB) Srivastav (@reach_vb) December 25, 2024
As the company explains, this model is comparable to GPT-4o in educational and factuality benchmarks. It is trained on a cluster with 2048 NVIDIA H800 GPUs. As far as the API price, you pay $0.27 for a million tokens.
🎉 What’s new in V3?
🧠 671B MoE parameters
🚀 37B activated parameters
📚 Trained on 14.8T high-quality tokens🔗 Dive deeper here:
Model 👉 https://t.co/9iwEF6aLuk
Paper 👉 https://t.co/ruzwMFYAAH🐋 2/n
— DeepSeek (@deepseek_ai) December 26, 2024
For now, you can access DeepSeek V3 right away here. You can activate DeepThink and perform search to use this.
[HT]
