Gradient

Just published a technical report on one of the most overlooked pain points in scaling RLHF: staleness. When you decouple rollout generation from policy optimization for throughput, your training becomes off-policy — and stale rollouts introduce a systematic bias that scales with S × η (staleness × learning rate). We derive a Two-Constraint Stability Rule that explains why training can suddenly collapse , and how to configure your async RLHF system to stay in the stable regime. Key takeaways: → Stale rollouts are not just noisy — they introduce a directional bias → Two training regimes: diffusive (stable) vs. ballistic (collapse) → Scaling law: η << min{ R_batch/(S·G), R_crit/(T·G) } Full technical report: https://lnkd.in/gfeDxuAi

Something we've been working on is now public. We're launching RL Commons — an open research initiative for the reinforcement learning era. The RL research community is facing real problems right now. Compute access. Most labs can't run the experiments they want to. R1-style post-training is out of reach for most university teams — not because the ideas aren't there, but because the resources aren't. Environments and benchmarks. RL progress depends on shared evaluation ground. Today, too much of that work gets rebuilt privately in every lab, and the best contributions don't travel. Visibility. Strong research gets done, then gets buried. There's no good connective tissue for the community to find, amplify, and build on each other's work. RL Commons is our attempt to address these together. Three things we're building: → A research platform connecting researchers with GPU providers, so teams can actually run the experiments they've been putting off. → An open library of RL environments, benchmarks, and rubrics — where contributors get credit, visibility, and a real audience for their work. → A connective effort for RL research — helping good work get seen, shared, and built upon across the community. Our first program is Project Aster — a 9-week founding cohort starting in late spring 2026. 5 to 10 selected teams. Free managed RL infrastructure. You focus on the research. We handle the rest. What each side gets: Researchers — free compute, a cohort of peers, full ownership of your work, and active promotion of your research across our channels. Compute partners — direct access to the researchers shaping the next five years of RL, founding-partner visibility across every program output, and the opportunity to support open science at a formative moment for the field. Contributors — credit, audience, and durable attribution for every environment, benchmark, and tool you share. We're early. The work only matters if the community builds it with us. If this resonates — apply, partner, or just follow along. rlcommons.org · @RLCommons (https://rlcommons.org/) on X

for those interested in distributed reinforcement learning I just published a ~1h tutorial on the echo2 framework by Gradient which is pushing hard on this direction. distributed RL is a very interesting research problem because it can create two very valuable economic benefits: - improvement for doing RL finetuning (i.e. cost less to fine tune) - allowing for participation in model training from consumers (i.e. access to more compute). both payoff having the opportunity of reinforcing themselves. however, it is one difficult problem to tackle since you pretty much have to have experts in compute infrastructure as part of the research team to tackle the very practical engineering considerations. this is why seeing the result the team at gradient managed to pull out is so satisfying (congrats folks! 👏 ). In this tutorial I've dived into: - how echo1 and echo2 implement distributed RL. - the infra split between rollout workers and centralized learner. - the different part of the stack they built and where they fit (parallax / P2P communication system between worker) - the role of off-policy RL finetuning in such an infrastructure. + we even did an interview with gradient cofounder eric yang himself! enjoy! 🌹

𝗣𝗿𝗼𝗺𝗽𝘁 𝗟𝗲𝗮𝗿𝗻𝗶𝗻𝗴 (ex. GEPA, ACE) improves accuracy by learning from experiences but it 𝗱𝗼𝗲𝘀 𝗻𝗼𝘁 𝘀𝗰𝗮𝗹𝗲 for 𝗽𝗮𝗿𝗮𝗹𝗹𝗲𝗹 𝗮𝗴𝗲𝗻𝘁𝘀. More parallel agents 🤖 = worse prompts 😭 Why? Processing too many trajectories concurrently damages the prompt update process 🐝 We fix this with 𝗖𝗼𝗺𝗯𝗲𝗲 : → preserves high-value signals → scales to more than 80 concurrent agents  → up to 17× speedup without quality drop The key idea is simple: - Separate the aggregation task into subgroups and process the rich information in two levels - Duplicate reflections to give each trajectory multiple chances of being considered We find that improved aggregation is the key for scaling prompt learning and it could be generalizable across popular prompt learning methods including GEPA and ACE. This prevents the "𝘤𝘰𝘯𝘵𝘦𝘹𝘵 𝘰𝘷𝘦𝘳𝘭𝘰𝘢𝘥" problem that we observed when naive scaling up the parallelism in previous methods, where context updates degrades from task-specific strategies to generic reminders. As usage accelerates for agents like Cursor, Claude Code from Anthropic, Codex from OpenAI, we believe prompt learning is entering a new era of scale.  Combee is a first step toward making that possible by optimizing for the aggregation of parallel agents. Paper: https://lnkd.in/gx4KTYxR Code (as PRs): https://lnkd.in/gA2QB-a6 + https://lnkd.in/gYw-wkPK   #LLM #Agent #ContextEngineering #AI #GenAI #ParallelAgents

New on the Anthropic Engineering Blog: How we use a multi-agent harness to push Claude further in frontend design and long-running autonomous software engineering. Read more: https://lnkd.in/gBi8Q6wt

AI companies rely on reinforcement learning (RL) to embed reasoning capabilities in their models. However, it's expensive, slow, and largely gatekept by labs with access to massive GPU clusters. Gradient is taking a different approach with Echo-2, a distributed RL framework built on top of their Open Intelligence Stack, a network that puts idle hardware to work across a global mesh of heterogeneous machines. Instead of forcing the same GPUs to juggle generation and model updates (and sit idle in between), Echo-2 splits the workload so consumer GPUs generate outputs in parallel while a centralized cluster handles learning. Both jobs run simultaneously. The efficiency gains are striking: -$425 vs. $4,490 for the same Qwen3-30B training run (a 90.5% cost reduction) -9.5 hours vs. 124 hours (92.3% faster) -Average model performance that exceeds ByteDance's verl across math reasoning benchmarks Gradient is productizing this through Logits, an RL-as-a-Service platform. The waitlist is now open to university students and researchers. For more information, read the full report (link in comments below).

We just published a new profile on Gradient, the AI lab building tools for sovereign intelligence. They're open-sourcing the infrastructure to run models anywhere. Our latest operator profile covers: - Parallax: Distributed inference across heterogeneous hardware (3-5x faster than alternatives) - CommonStack: API routing with up to 25% cost savings and built-in fault tolerance - Echo-2: Reinforcement learning framework cutting training costs from $4,490 to $425 Backed by Pantera, Multicoin, and HSG. Early commercial traction. RL-as-a-service launching later this year. Full breakdown on what they've built, why it matters, and what's still unproven. Read the full report: https://lnkd.in/eJPTuD8u