Cosine reposted this
Building the UK’s first fully sovereign frontier AI model is no small feat, but the roadmap for Lumen Sovereign is locked in. We’re pushing the boundaries of architecture at Cosine to deliver a world-class LLM, and the next few months are critical. – Baseline & architecture validation (June-Aug 2026): We are currently testing our Mixture-of-Experts scaling pilots. Our lead configuration targets 192-256 experts with top-6 routing. We are promoting configurations based on rigorous loss analysis, SWE evals, long-context checks, and expert routing stability. – Model enlargement (Aug-Sept 2026): This is where the heavy lifting happens. We'll run the main expert-expansion continued pretraining phase to produce our 1 trillion-parameter intermediate checkpoint (based on the 256K source architecture) with 49.2B active parameters. – Agent post-training (Sept-Oct 2026): We'll run supervised fine-tuning on high-quality agentic coding demonstrations, followed by asynchronous reinforcement learning on long-horizon executable coding tasks. Building frontier AI requires pushing the absolute boundaries of model architecture. By Q4, our coalition partners will gain early access to these checkpoints, paving the way for a Q1 2027 commercial launch, available in the cloud, VPC, or fully air-gapped on your own GPUs. I am deeply proud of the relentless engineering going into this. It’s a big job, but we have the highly capable talent to do it. Keep an eye out over the coming weeks for an inside look into our lab.