Prime Sandboxes: MicroVMs for Agentic RL Training at Scale

In agentic RL, the sandbox is the foundation of a training system. It creates the state that the agent acts on, enforces isolation boundaries and produces trajectories that feed directly into the reward signal. However, model training requires running thousands of concurrent sandboxes, which leads to complex configuration and can be very costly. Prime Sandboxes were built to solve this, responding to the needs of research teams like ours as we were unable to find adequate options on the market. With an architecture built for agentic training and pricing designed for thousands of concurrent instances, they are the most cost-effective sandboxes available today.

Today, Prime Sandboxes enter general availability. Each sandbox is a fully capable Linux virtual machine. Our researchers and a few select customers have been using them over the last few weeks, with ~30M sandboxes created so far. We are now making Prime Sandboxes available to everyone, both as standalone infrastructure through our CLI/SDK and as part of our RL suite.

For our users, this means:

• Full VM fidelity, supporting Docker Compose, background jobs, and kernel-dependent workloads in every sandbox.
• Elastic capacity at scale, with the capability to run tens of thousands of concurrent sandboxes.
• First-class RL support, including integrations with verifiers, prime-rl, and Prime Tunnels, with access to over 365,000 prebuilt environments.
• Bring your own environment, using Docker images without needing a new packaging format.
• Tier-free pricing built for scale, with no subscriptions or minimum spend; currently the most competitive on the market (3x cheaper than other providers) through December 22.

Getting started is simple:

prime sandbox create
prime sandbox run <sandbox-id> 'echo Hello, World!'

Teams that need to push the bounds of sandbox concurrency can contact us directly.

In the near future, we will expand Prime Sandboxes to offer GPU microVMs, state snapshotting, sandbox forking, and shared persistent workspaces. This foundation will enable autonomous research loops that can explore, recover, and compound progress over time.

The VM Advantage for RL

Container sandboxes are a proven default for short-lived code execution, and much of the industry has settled on gVisor containers. gVisor narrows the host-kernel attack surface by implementing a Linux-compatible user space kernel. That is a strong fit when a workload stays within the interface it implements. For workloads that reach beyond that interface, however, its compatibility boundaries begin to show.

In our experience, these boundaries matter in two ways: 1) agents need high-fidelity environments that reflect real systems, and 2) researchers need the flexibility to shape those environments without compromising the reward signal. A full Linux VM provides both by giving each sandbox its own kernel while keeping the isolation boundary outside the workload.

For agents in training, silent differences from production can be more dangerous than hard failures because they can reward behaviors that do not transfer into reality. Docker is the clearest example. In order for agents to develop software engineering tasks around Docker Compose like they would on a real machine, gVisor needs complex setup and even then, does not offer the full list of features. Without a full VM, the agents risk learning an imitation environment instead of a production one.

For researchers, system-level control makes it possible to shape the environment against reward hacking. For example, in one browser task, we used file system namespaces to hide the mock site’s backing data. This forced agents to find answers by navigating the browser and prevented them from bypassing the task by reading the files directly.

Together, these properties make VMs a more faithful sandboxing environment for agents and a more flexible foundation for the researchers who train them.

RL Training Without Infrastructure Bottlenecks

Popular sandbox benchmarks test how quickly a provider can start many copies of the same image. But this favors a single warm cache and says little about performance across the diverse environments used in agentic RL training.

Real agentic RL workloads combine both variety and concurrency. A training run may span thousands of tasks, each with its own repository, tools, and environment, while launching multiple attempts at each one.

At Prime Intellect, our researchers run tens of thousands of concurrent sandboxes across diverse environments every day. We designed the system around this workload from day one, caching images near the compute and preferentially scheduling sandboxes where their environments are already available to deploy thousands of sandboxes within seconds. In launching Prime Sandboxes, we aim to make this same experience accessible to every researcher, so that infrastructure is never the limiting factor for experimentation.

First-Class RL Support

Prime Sandboxes are a standalone product available through our CLI/SDK. They also integrate directly with our RL-native stack including verifiers, prime-rl, and Hosted Training, rather than sitting beside them as a generic execution service.

Alongside Prime Sandboxes, we also built Prime Tunnels. They make inference services running on a cluster node accessible to sandboxed agents without additional network setup. In verifiers, for example, an interception server on the researcher’s host logs each model call and forwards it to the underlying model. With a single command, researchers can expose that server through a public HTTPS URL and connect it to many concurrent rollouts. Prime Tunnels are a standalone service but pair naturally with Prime Sandboxes through verifiers. Together, these pieces form the training system we wanted for ourselves and a foundation for others to push agentic RL further.

Open Environment Registry

Prime Sandboxes are backed by a registry of more than 365,000 prebuilt environments spanning open-source software engineering, terminal use, and agentic tasks. To our knowledge, it is the largest catalog offered by a sandbox provider.

Its breadth lets researchers build diverse curricula without creating and maintaining every environment themselves. The same task environment can be reused across evaluations, training runs, teams, and research projects.

All images, regardless of whether they are public or private, are managed through the same system. Task images are stored immutably, so researchers can reproduce a run's exact environment months later without worrying about drift or disappearing upstream artifacts.

Bring Your Own Environment

Docker is the common language for defining environments in research workflows: tasks either ship as Dockerfiles or are referenced by image tags. We preserve this workflow by handling the conversion from Docker image to bootable VM beneath the surface. This way, researchers gain full VM capabilities without changing how they package their environments. Creating a sandbox begins with the same familiar artifact:

prime sandbox create python:3.11-slim
prime sandbox run <sandbox-id> -- python3 -c "print('hello')"
prime sandbox upload <sandbox-id> ./agent.py /root/agent.py

Additionally, custom environments use the same workflow researchers already follow locally. Write a Dockerfile, build the image, push it to the Prime registry, and reference it by name when creating a sandbox. Public images from Docker Hub can be used directly with no additional preparation.

The result is a container-shaped API in front of a fleet of microVMs. The ergonomics are ones researchers already know, with images, Dockerfiles, exec, and file transfer, while at the same time, every sandbox boots its own guest kernel with hardware-level isolation and unrestricted Docker inside. This equips users with all the benefits of VMs and none of the drawbacks of operating them.

Tier-Free Pricing

Prime Sandboxes use a straightforward usage-based pricing model with no tiers that scales predictably with the resources you request.

For our launch through December 22, we’re proud to offer the most competitive pricing on the market for sandboxes, at a third of the cost of other large sandbox providers.

On the Roadmap Ahead

In the next months, three releases will push sandboxes deeper into the continual learning loop itself: GPU sandboxes will extend agent workloads into full AI research, bringing teams one step closer to recursive self-improvement; snapshots will allow researchers to save, restore, and fork a sandbox mid-run, ensuring that a single mistake never erases hours of progress; persistent workspaces will let agents build on each other's work across rollouts instead of starting over.

If shipping this sounds interesting to you, join our Sandbox Platform team!

In the meantime, Prime Sandboxes already give researchers full VM fidelity, native Docker support, and the scale to run tens of thousands of concurrent environments. Try them today.