Scaling Agentic RL: 365,000+ Environments for SWE, Terminal, and Search

The open research ecosystem has produced many great datasets for the three main agentic domains - software engineering, terminal use, and web research - but every one of them ships with its own harness, its own image conventions, its own grading scripts, and its own failure modes.

We integrated them all. We ship first-party integrations for 23 agentic tasksets behind one taskset API, unified runtime/sandbox hooks, and one command: ~198,000 software engineering tasks across 20+ languages, ~28,600 terminal tasks, and ~137,600 search tasks - ~365,000 tasks in total, ready for evals and RL training on Prime Intellect infrastructure, with validated and cleaned dataset re-uploads where the originals needed fixing.

uv pip install "git+https://github.com/PrimeIntellect-ai/research-environments.git#subdirectory=environments/swe/scaleswe_v1"

# Example running ScaleSWE in Codex harness on Prime Sandboxes:
uv run eval scaleswe-v1 --harness.id codex --harness.runtime.type prime -n 3

What makes that single command possible is verifiers v1, which decomposes an environment into three independent layers: a taskset, a harness, and a runtime. This post is the taskset piece of that story.

One contract, three domains

Each upstream taskset made reasonable choices for its own harness - and those choices don't compose. SWE-bench applies test patches inside a generated eval script; R2E-Gym bakes tests into the image and compares against expected outputs; search benchmarks each invent their own judge. If you want to train one agent across all of them, you need those lifecycles normalized without breaking each taskset's own scoring semantics.

Our integrations keep every taskset's original grading path - upstream log parsers, upstream report generation, upstream test commands - and normalize everything around it:

• One API. Every taskset loads from a typed config (dataset, split, filters), provisions a sandbox from the task's image, and scores with the taskset's own logic. Adding filter_fn or swapping splits works the same everywhere.
• One image registry. Task images live in our own Prime image registry, co-located with the sandboxes - ~135,000 prebuilt open-source task images, to our knowledge the largest such catalog hosted by any sandbox provider. No flags, no namespace mapping, and no Docker Hub rate limits at rollout time - the ones you would otherwise hit immediately when running a thousand concurrent rollouts against Docker Hub. You can also pre-build and upload your own images (prime images push) or copy existing ones straight from Docker Hub (prime images transfer-bulk).
• One integrity standard. During a rollout the agent lives inside the same sandbox as the grading machinery, so anything readable in the container is fair game for a reward hack. Every integration therefore withholds grading material - test patches, expected outputs, grading scripts - until scoring time. Some original authors chose to keep it visible: R2E-Gym ships its grading tests readable inside the image at /r2e_tests, and Multi-SWE's containers carry the grading shell scripts and test.patch readable under /home. That is a fine choice for their attach-at-eval harnesses, but not for a live RL sandbox, so our integrations hide these artifacts and restore them only for scoring.
• One validation bar. Before a dataset earns a spot as default, we run gold-patch and no-op validation against it and re-upload cleaned versions with the exclusions preserved for transparency - the next section covers how.

A common shape for a taskset is a data schema plus a handful of hooks:

import verifiers.v1 as vf

class MyTaskData(vf.TaskData):
    base_commit: str      # state the sandbox resets to
    test_patch: str       # grading material - withheld until scoring
    gold_patch: str       # reference fix - used only by `validate`

class MyTask(vf.Task[MyTaskData]):
    async def setup(self, runtime: vf.Runtime):
        ...  # prepare the repo inside the task's image

    async def finalize(self, trace: vf.Trace, runtime: vf.Runtime):
        ...  # capture the agent's diff into the trace for posterity

    @vf.reward
    async def solved(self, runtime: vf.Runtime) -> float:
        ...  # restore tests, apply test_patch, run the upstream grading path

    async def apply_gold_patch(self, runtime: vf.Runtime):
        ...  # apply the reference fix

    async def validate(self, runtime: vf.Runtime) -> bool:
        ...  # gold patch must score 1.0; not calling `validate` must not

class MyTaskset(vf.Taskset[MyTask, vf.TasksetConfig]):
    def load(self) -> list[MyTask]:
        ...  # (dataset, split, filter_fn) -> tasks pinned to images

We constantly evaluate or train on these tasksets internally, which continuously validates their interplay with our sandbox infrastructure, and we update the datasets, images, and integrations on a regular basis.

Validated re-uploads

A taskset row is only useful if it can produce a clean reward signal: gold patch applied → tests pass; no patch → tests fail. A surprising fraction of open agentic task data fails this precondition - broken images, network-dependent tests, drifted expected outputs, and tasks solvable without touching the code at all.

So we validated. For each dataset we ran the gold patch through the full scoring path in fresh sandboxes, retried failures up to 10× to separate flaky from deterministically broken, ran independent second passes to catch noisy rows, and ran multiple no-edit passes to drop tasks that score 1.0 without any fix. The results are re-uploaded in the SWE RL collection with the dropped rows persisted, so you can audit every exclusion:

• R2E-Gym-Subset-Verified: 4,522 of 4,578 rows pass 10×-retry gold validation; the 56 drops are mostly network/timing-sensitive aiohttp/tornado tests.
• SWE-Lego-Real-Data-Verified: 4,323 of 4,432 rows, same methodology.
• Multi-SWE-RL-Verified: 2,232 rows surviving two gold-patch passes plus a no-edit filter that caught tasks gradeable as solved with zero edits.
• SWE-rebench-V2-Filtered-Verified: 6,275 rows from 32,079 raw - language-level drops for wholesale-broken images, two independent gold passes with flaky rows removed, and problem statements scrubbed of inline GitHub issue/PR references.
• SWE-Bench-Verified-Quick: 468 of 500 Verified instances, dropping the slowest examples for quick online-evals.

The tooling behind these passes ships with verifiers, so you can reproduce them yourself. uv run validate <taskset-id> is the model-free sibling of eval: for every task it runs the gold check (setup, apply the reference fix, tests must pass) and a setup-only no-op check (tests must not pass without a fix) in independent runtimes - --only-gold and --only-setup select one side. And uv run debug <taskset-id> --command "<cmd>" (or --script-path) sets up a task's sandbox, runs an arbitrary command or uploaded script inside it, and persists the traces - by far the most convenient way to poke at a container task's state without writing a harness.

Software Engineering Tasks

• SWE-bench Verified (Jimenez et al., Princeton - paper) is the canonical benchmark: real GitHub issues from major Python repos, resolved by writing a patch that makes hidden failing tests pass, human-filtered by OpenAI down to 500 instances confirmed to be fairly solvable. swebench_verified_v1 runs the Harbor Hub packaging against the official instance images.
• SWE-bench Multilingual (Khandpur et al., SWE-bench team - blog, paper, swebench_multilingual_v1) extends the canonical benchmark beyond Python: the official 300-instance test set across C, C++, Go, Java, JS/TS, PHP, Ruby, and Rust. Our integration wraps the Harbor packaging, and grades with the canonical verifier.
• SWE-bench Pro (Deng et al., Scale AI - repo) is the harder successor: 731 public tasks from license-friendly repos with large-scale diffs. Our integration (swebench_pro_v1) wraps the Harbor packaging and resolves each task's hosted image from its test config.
• SWE-smith (Yang et al., Stanford / Princeton - paper, swesmith_v1) inverts the data problem: instead of mining issues, it injects bugs into healthy repos and keeps the tests that catch them - 88,130 instances across eight languages. Task branches carry the bug with the failing tests removed from the head commit; our port keeps the agent at that head, restores the tests via HEAD~1 only inside scoring, and reuses upstream's grading and profile registry verbatim.
• R2E-Gym (Jain et al., UC Berkeley - paper) generates executable environments from real commits with synthesized issue descriptions - 4,578 instances in its curated subset. The author images keep the grading tests readable inside the container; r2e_gym_v1 restores /r2e_tests only for scoring. We publish a gold-validated re-upload - 4,522 of the 4,578 instances - which the taskset defaults to.
• Multi-SWE (Zan et al., ByteDance - paper, multiswe_v1) extends beyond Python: 4,703 containerized RL instances across C, C++, Go, Java, JS, Rust, and TypeScript, graded by the upstream report protocol. Our re-uploads - the RL set plus the 2,132-task Multi-SWE-bench eval set - fix the HF schema, and we hide the grading scripts and test.patch that the original containers leave readable.
• SWE-rebench-V2 (Badertdinov et al., Nebius - paper, swerebench_v2_v1) continuously mines fresh GitHub PRs into tasks - 32,079 of them across 20 languages, naturally decontaminated by recency. We vendor the upstream log parsers (~3.6k lines, one per language ecosystem) for grading fidelity, hide the test patch until scoring, and default to our filtered-and-verified re-upload.
• Scale-SWE (Zhao et al. - paper, scaleswe_v1) contributes 20,181 Python tasks with test patches and scripts applied just before evaluation, exactly as the authors specify. Our re-upload carries the 17,202 instances that survived validation.
• SWE-Lego (Tao et al., Huawei - paper) builds SWE-bench-style training data at scale; we host the 4,432 resolved real-data instances plus a gold-validated variant. swelego_v1 keeps the original authors' evaluation pattern: the failing tests stay hidden during the rollout and are applied only at scoring, after resetting any test files the agent touched back to base_commit.
• OpenSWE (Fu et al., GAIR - paper) pairs 45,320 tasks with per-task evaluation scripts stored as data. openswe_v1 keeps the eval script out of the sandbox until scoring writes and runs it.
• Senior SWE-Bench (Snorkel AI - repo, senior_swe_bench_v1) moves past issue-resolution: 50 public investigation and design tasks drawn from 12 production open-source repos (gitea, posthog, teleport, immich, and more), scored by the upstream verifier pipeline - native pytest/vitest checks plus optional LLM rubric judges. Every task runs from a prebuilt Prime image and is gold-validated against the upstream solutions.

All our filtered and verified SWE re-uploads live in the SWE RL collection on Hugging Face, each with a dataset card documenting exactly what changed vs upstream, the validation methodology, and exclusions preserved for audit.

Terminal Tasks

• Terminal-Lego (Yang et al., Huawei / HKU - paper, terminal_lego_v1) is ~13,800 Docker-verified Terminal-Bench-style tasks built by the SWE-Lego team from real StackOverflow issues - filtered, converted through cascaded LLM generation, and kept only if they survive Docker round-trip verification. Each task is a directory with instruction, hidden pytest grader, and reference solution, pinned to a prebuilt per-task image in the Prime registry.
• TMax (Ivison et al., Ai2 / UW - paper, tmax_v1) is 14,600 terminal tasks pinned in our versioned task registry, every task with a prebuilt Prime image. The full corpus went through a validate pass, all 14,600 sandboxes booting and setting up green, before the taskset shipped.
• Terminal-Bench 2 (Merrill et al., Stanford / Laude Institute - paper) is the community-standard terminal eval: 89 rigorously verified tasks, each with its own environment, human-written solution, and tests - the benchmark the corpora above report against. terminal_bench_2_v1 wraps it through the same Harbor taskset, so the eval suite and the training corpora share one task format and one scoring contract.
• OpenThoughts-TBLite (the OpenThoughts-Agent team, Snorkel AI & Bespoke Labs - blog, openthoughts_tblite_v1) is a high-signal 100-task benchmark for iterating on terminal agents, each task running in its own prebuilt container and scored by a hidden grader.

All terminal tasksets live under environments/terminal, with an overview README covering the group and per-taskset changelogs.

Search Tasks

The search tasksets share one design decision: they are harness-agnostic and tool-free. The taskset ships questions and scoring only - the harness brings its own search tool (the Codex harness's built-in web search, our harness's search skill, or your own). The same tasks train and evaluate any search-capable agent without the environment prescribing a retrieval pipeline.

• WideSeek (wideseek_v1) - WideSearch-style table compilation (Xu et al., RLinf / Tsinghua - paper): three overlapping 20k-example splits with 44,632 unique questions between them; the width split asks for a full Markdown table as the answer, scored by item-level cell F1.
• PaperSearchQA (papersearchqa_v1) - 54,907 train + 5,000 test biomedical deep-research questions (Burgess et al., Stanford - paper), judge-graded against acceptable answers.
• OpenSeeker (openseeker_v1) - 11,677 web-research QA tasks (Du et al. - paper) with the original judge prompt.
• DeepDive (deepdive_v1) - 2,234 RL questions plus a 1,016-question SFT split (Lu et al., Z.ai / Tsinghua - paper), hard multi-hop research graded by a strict boxed-answer judge.
• S1-DeepResearch (s1_deepresearch_v1) - (Dong et al. - paper): ~15,000 multi-hop resolution questions with gold answers, judge-graded.
• REDSearcher (redsearcher_v1) - 1,000 long-horizon web-research questions (Chu et al. - paper).
• BrowseComp (browsecomp_v1) - the 1,266-question browsing benchmark (Wei et al., OpenAI - paper) in its original Explanation/Exact Answer/Confidence format.
• BrowseComp-Plus (browsecomp_plus_v1) - 830 BrowseComp queries re-grounded in a fixed, human-verified corpus of 100,195 web documents (Chen et al., Waterloo - paper). The one exception to bring-your-own search: the taskset serves the benchmark's own BM25 search tool over the corpus, so the retriever is a controlled variable and runs are reproducible - graded by the benchmark's HLE-style judge, with evidence recall tracked alongside accuracy.

All search tasksets live under environments/search, with an overview README covering the group and per-taskset changelogs.

Training

Everything above plugs straight into prime-rl. This training config trains GLM-4.5-Air on scaleswe_v1 on 6 H200 nodes in 2 days.

What's next

The integrations, validation harnesses, and test visibility changes described here live in research-environments; the validated SWE datasets are in this Hugging Face collection. Point prime-rl at any taskset id and you're training. If you train on these tasksets, cite the original papers - BibTeX for all of them is collected in the SWE taskset README, and our re-uploads preserve upstream licenses and link back to the source datasets on every card.

A reward signal can lie in two directions. The false positive is the reward hack, and we've watched it happen in our own training runs: the agent shares a sandbox with the grading machinery, and a policy under RL pressure will eventually find whatever seam is left - a test file it can edit, grading state it can tamper with, an artifact that leaks the expected answer. Withholding grading material until scoring raises the bar significantly, but as long as grading runs where the agent lives, it's mitigation, not a guarantee. The structural fix is on our roadmap: grading in isolated sandboxes, so the environment the agent can touch and the environment that scores it are separate.

The converse failure class - the false negative - is one that validation, by construction, cannot catch. Tasks mined from merged PRs inherit that PR's tests, and those tests often assert implementation details rather than behavior - a specific error-message string, the name of a private helper, an exact return shape. An agent that fixes the underlying issue in a different but equally correct way still fails them, and the reward reads as a false negative. Gold-patch validation is blind to this: the original patch passes its own tests by definition. At RL scale these near-misses are pure noise in the training signal - the model gets punished for correct work. Our internal effort to mitigate this is Agentic Judging. More on that soon.

We're continuing to expand validation to the datasets that haven't had their pass yet and publishing filtered re-uploads through the same transparent pipeline. If you're building agents and something's missing from this list, tell us - or better, port it: the taskset contract is small, and our sandbox does the heavy lifting.

@article{primeintellect2026scalingagenticrl,
author = {Prime Intellect Team},
title = {Scaling Agentic RL: 365,000+ Environments for SWE, Terminal, and Search},
journal = {Prime Intellect Blog},
year = {2026},
month = {July},
note = {https://www.primeintellect.ai/blog/scaling-agentic-rl}
}