Picture this: you ask a flagship model to build a complete code repository from a natural-language description. It interacts with terminals, tests, and file systems for hundreds of rounds, burns roughly 700,000 tokens per rollout, and runs for hours. Then you want to push pass rates higher with online reinforcement learning, only to watch a large chunk of your GPU fleet sit idle, waiting for the slowest trajectories to finish. Wait or don't wait, either way the training budget bleeds out through the long tail.
QwenGyre, newly released by Alibaba's Qwen team, takes this exact problem as its target. The answer comes in two halves: one manages compute scheduling, letting idle GPUs switch between sampling and training on demand; the other manages trajectory processing, pruning the massive redundancy produced by branching agent executions into a shape training can efficiently digest.
What the Paper Claims, and What This Article Is Not
First, the source. The paper, "QwenGyre: An Elastic Reinforcement Learning Framework for Training xLong-Horizon Agents," was submitted to arXiv on September 27, 2026 (number 2609.33848, see https://arxiv.org/abs/2609.33848 ), with 12 authors including Weiqi Wang, Yuxin Zhou, and Mouxiang Chen. Chinese tech outlet ai-bot.cn picked up the release on October 13 under the headline that Alibaba's Qwen team launched an online RL framework. The model being trained is Qwen3.8-2.4T.
Now the editorial positioning. This site has previously covered Xiaomi's MiMo-V2.6 agentic RL route and its verl-based training infrastructure, as well as a comparison review of open-source RL training frameworks. So this piece will not rehash framework selection or run another side-by-side comparison. The angle here is narrower: when a flagship model runs 700K-token-class long-horizon agent tasks, what does engineering-grade scheduling and trajectory handling for online RL actually look like? On this specific question, QwenGyre is the most systematic public material available.
Two Walls in Long-Horizon Online RL
According to the paper's abstract, QwenGyre targets two pain points.
The first wall: high execution variance, long rollout latency, and massively idle GPUs. Long-horizon tasks have an unpleasant property: trajectories sampled from the same policy can differ in completion time by several-fold. Some rollouts converge in half an hour; others run for hours stuck in a loop of environment errors. Synchronous training waits for the slowest trajectory to finish while fast GPUs tag along doing nothing. Asynchronous training stops waiting, but once parameters update, the value of in-flight old trajectories is discounted, and poor scheduling can waste even more. The longer the task, the more variance gets amplified, and the more hopeless idling becomes.
The second wall: non-linear forking produces massive trajectory redundancy and drags down training efficiency. Agent tasks are not a straight line. Different results from one tool call lead to entirely different downstream paths; a half-built repository diverges into different worlds depending on whether tests pass or fail. After hundreds of interactions, trajectory counts balloon, yet much of the content consists of duplicated shared prefixes. If every trajectory is priced in full and pushes gradients in full, the same prefix gets computed and weighted repeatedly, and the training signal gets diluted by redundancy.
Neither pain point is fatal in ordinary-length RLHF. But once tasks stretch to the paper's definition of an xlong task, meaning a single execution approaching one million tokens, hundreds of model-environment interactions, and hours of runtime, both are amplified exponentially. QwenGyre's two core mechanisms each take aim at one wall.
The Elastic Scheduler: GPUs Changing Jobs Between Sampling and Training
The first pillar is the Elastic Scheduler. The problem it solves: the compute demand of rollout and the compute demand of training are mismatched in time, while conventional setups statically pin GPUs to a single role.
Homogeneous cells with no preset role
QwenGyre partitions GPUs into homogeneous cells, where each cell can either run sampling through SGLang or host a training replica through Megatron. Homogeneity is the precondition: if cells differed in hardware, job-swapping could not flow freely. A cell's role is no longer dictated by deployment topology; the scheduler assigns it on demand. When sampling falls behind, more cells get dispatched to sampling; when training has slack, they come back to compute gradients.
Seconds-level switching with three requirements
The headline claim is dynamic switching without interrupting running tasks. At switch time, three things happen: requests are re-routed, the KV cache migrates over RDMA, and workspace and tool state are preserved. Per the paper, the whole switch takes only seconds.
It is worth unpacking all three. Request re-routing keeps in-flight requests alive. KV cache migration over RDMA means the sampling context does not need a fresh prefill pass, which for a trajectory already hundreds of thousands of tokens deep would be prohibitively expensive. Workspace and tool state preservation is the most easily overlooked yet most critical piece: in agent RL, environment state is not just the model's context. File systems, terminal sessions, and test environments are state too. If switching sampling nodes required resetting the environment, a long-horizon task would effectively restart from zero. Folding that layer into the migration is what makes "uninterrupted" credible.
Core and satellite cells: hot-swappable compute
Another mechanism worth expanding on is the two-tier split between core and satellite cells. Core cells hold the authoritative parameters. Satellite cells pull a parameter snapshot and can join the current batch's gradient accumulation mid-flight. This means compute is hot-swappable: a freshly freed GPU does not need to wait for the next complete training round; it plugs into the batch already in progress. For long-horizon workloads, this kind of "grab work whenever available" elasticity is precisely what eats up idle time.
In one sentence: the Elastic Scheduler decouples compute supply from task rhythm, so a 700K-token long-tail rollout no longer holds the entire cluster hostage.
The Trajectory Processor: TITO, Trajectory Trees, and Partial Scoring
The second pillar is the Trajectory Processor, aimed at the trajectory-explosion wall. Three mechanisms interlock.
TITO: every output must match the context it was generated under
TITO, short for Token-in Token-out, records the exact input and output of every call. That sounds unremarkable, but it hits a subtle trap in agent RL. After many rounds of interaction, the model's context has been rewritten repeatedly by system injections, tool results, and environment feedback. If you take the final-state arrangement as the training signal, a given output may correspond to a context that is not the one it was actually generated under, and the training signal becomes "false context": gradients correcting a decision that never existed at that moment. TITO's principle is that every output must map back to the original context at its generation time. A concrete example makes the trap vivid: suppose the agent runs a test command at step 50, receives a failure report, and makes a fix based on it. By step 300, that report in the context may have been summarized, compressed, or pushed out of shape by later tool outputs. If training treats the distorted report as seen at step 300 as the input for the step-51 decision, the causal relationship the model learns is simply wrong. TITO locks in the exact input and output of every step at recording time; no matter how the context evolves afterward, the training signal stays anchored to the moment of generation. For tasks with hundreds of interactions, this single property determines whether gradients fix the mistake that was actually made or an imaginary one.
Trajectory trees: shared prefixes counted once
Trajectories are organized into a tree by shared prefix. Branching trajectories naturally share prefixes, and the tree structure lets the system recognize that these trajectories are different continuations of the same history. This is the foundation for loss accounting: since the prefix is one and the same, its loss should not be counted once per fork.
Partial Scoring and Jmax: credit for timeouts, a cap on forking
Long tasks frequently fail to finish. Over hours of execution, timeouts are the norm, not the exception. Giving such runs zero credit is clearly wrong: a half-built repository still has completed, verifiable modules. Partial Scoring scores the verifiable partial results of timed-out tasks, so "half done" also carries a training signal.
The other half is a quota mechanism: at most Jmax trajectories are kept per execution, and shared-prefix loss is counted only once, decoupling each execution's weight from the number of forks. Plainly put, if an agent state forks into 10 downstream paths, its contribution to the gradient does not inflate tenfold. Without this, the policy learns that fork-happy behavior gets higher weight, and training gets hijacked by exploration itself, which is the sneakiest cost of trajectory redundancy.
Together, the three mechanisms ensure the signal is correct (TITO), the structure is clear (trajectory trees), and the weights are fair (Partial Scoring plus Jmax). Trajectories stop being a pile of equal-length conversation logs and become a tree priced on demand.
Reading the Numbers
All figures below come from the paper. The model is Qwen3.8-2.4T. A single rollout runs to roughly 700,000 tokens. Over 48 training steps, the NL2RepoBench pass rate rises from 52.5 percent to 58.5 percent, an absolute gain of 6.0 points. Training time for those 48 steps drops from 134.55 hours to 75.42 hours. At equal GPU budget, QwenGyre is 1.85 times faster than a Colocate setup and 1.78 times faster than an Async setup.
NL2RepoBench measures building a complete code repository from natural language. A move from 52.5 to 58.5 percent is not an order-of-magnitude jump, but what it proves matters: on this class of ultra-long tasks, online RL can still lift an already strong flagship model by 6 absolute points. With the baseline already at 52.5 percent, every further point is expensive, so this gain is not cheap.
The time figure carries even more engineering weight: 134.55 hours down to 75.42 hours, nearly 60 hours saved. In those saved hours, the model did not change, the data did not change, and the reward function did not change. Everything saved is scheduling and trajectory-processing gains. In other words, under long-horizon workloads, systems-level optimization is a big enough lever to compete on stage with algorithmic improvements.
The two baselines deserve a closer look, because they represent the dominant ways of organizing resources today. Colocate puts sampling and training on the same set of GPUs, with each card switching roles across training phases. It saves machines, but role switching is coarse-grained, and when long-tail tasks show up, every card gets dragged along. Async separates sampling from training completely and lets them run independently, so nobody waits for slow trajectories, but parameter updates drift out of sync with in-flight trajectories, whose value decays over time. The 1.85x and 1.78x speedups suggest that elastic scheduling genuinely outperforms both the "lockstep synchronous" and the "loosely asynchronous" traditions, and by a comfortable margin.
For teams without a 2.4T-parameter model, the takeaway is directional: once your agent tasks are counted in hundreds of thousands of tokens and in hours, scheduling and trajectory handling should move up the priority list. They are not nice-to-have engineering details; they are the foundation that decides whether training can run at all.
Use Cases, Boundaries, and What to Watch
Per the paper, the target scenarios fall into five categories: generating complete repositories from natural language, long-horizon software engineering on existing repositories, multi-turn terminal automation, large-scale refactoring and migration, and multi-agent collaboration. The common traits are clear: long execution, many interactions, and a verifiable endpoint. Any task shape meeting all three is within this framework's range.
One boundary must be stated plainly: QwenGyre is currently released as a paper only. Code and installation packages are not publicly available (per ai-bot.cn's report). Reproducing the mechanisms on open-source frameworks like verl means either waiting for an official code drop or implementing from the paper yourself. And the hard parts of elastic scheduling, including RDMA-based KV cache migration, preserving tool state across cell switches, and mid-batch joining of satellite cells, all carry substantial engineering cost. Budget accordingly.
Three things are worth watching. First, whether the code will be open-sourced, which determines whether the community can verify and reuse the approach. Second, whether the 1.85x speedup holds on smaller models, and how elastic-scheduling gains scale with task length. Third, whether Partial Scoring's definition of verifiable partial results can extend beyond code; how you write the scorer decides how wide a task surface this method can cover.
The RL arms race for long-horizon agents is just opening. QwenGyre's value is not one benchmark number but the fact that it lays out, for the first time in full public view, an answer to a question that used to live in scattered internal practice: how do you make training infrastructure worthy of the model's capability.