xAI is preparing to drop Grok 5 in mid-June, and the numbers being reported are staggering. According to multiple sources tracking the model's development, Grok 5 uses a mixture-of-experts (MoE) architecture with 6 trillion total parameters — making it one of the largest models ever trained. The release has been anticipated since early in the year, and Polymarket currently gives it 33% odds of arriving before June 30.

TL;DR: Grok 5 is a 6T-parameter multimodal model with 1.5M token context, trained on xAI's Colossus 2 supercluster. It's expected mid-June and xAI is claiming a lead on coding benchmarks.

The Architecture: Scale at a New Level

The 6 trillion parameter figure refers to the total pool of weights across all expert layers — in a MoE model, only a fraction of those parameters are active for any given token, keeping inference costs manageable while still benefiting from the representational capacity of a much larger model. The design follows the direction set by models like Mixtral and GPT-4, but at significantly larger scale.

Native context length is reported at 1.5 million tokens, which would comfortably hold the entire contents of a large codebase, a full novel, or many hours of transcribed conversation in a single prompt window.

Colossus 2: The Machine Behind the Model

Grok 5 was trained on Colossus 2, xAI's second-generation supercluster based in Memphis, Tennessee. The cluster runs 550,000 NVIDIA GPUs and draws approximately 1 gigawatt of power — figures that put it among the largest AI training installations in the world. xAI built Colossus 2 specifically to have the compute headroom to train models at this scale without being bottlenecked by shared cloud infrastructure.

Multimodal by Default

Unlike earlier Grok models that bolt on vision as an afterthought, Grok 5 is reported to be multimodal from the ground up — natively processing text, images, video, and audio in a unified architecture. This would put it in direct competition with GPT-4o and Gemini Ultra on the multimodal front, rather than trailing them.

The Coding Claim

xAI has been highlighting Grok 5's performance on coding benchmarks, with internal numbers suggesting it leads current frontier models on several standard evaluations. Independent verification hasn't happened yet — those results will come once the model is released and the research community can run their own tests. For now, the claim is plausible but unconfirmed.

Key Takeaways

  • Grok 5 is a 6 trillion parameter MoE model — one of the largest ever trained
  • Native 1.5 million token context window
  • Natively multimodal: text, images, video, audio
  • Trained on Colossus 2 (Memphis), xAI's 550,000-GPU, 1GW supercluster
  • Mid-June release expected; Polymarket gives 33% odds before June 30
  • xAI claims Grok 5 leads on coding benchmarks — independent verification pending

Conclusion

If the reported specs hold up, Grok 5 will be a genuine frontier model and not just a catch-up release. The combination of MoE scale, long context, and native multimodality would make it a serious contender against OpenAI's and Google's best. The mid-June window is almost here — expect the benchmarks and community reception to tell the full story.