Reinforcement learning environments
Types of RL environments, and what each one trains
Last updated /Reinforcement learning environments
Types of RL environments differ by the action space, the observation the agent receives, and the grader that scores the ending. A tool-use environment exposes functions. A computer-use environment exposes a screen. A replica exposes a sandboxed business app. A simulated-user environment adds a second party who speaks. The training goal picks the type. A type does not, by itself, mean a vendor has that package ready for your domain.
This page is the comparison. The map it belongs to is reinforcement learning environments: what they are and what real work adds. Specimens of named tasks, which are illustrations and not customers, are concrete examples of environments drawn from business work. Why an agent that already speaks still needs a world to act in is why language-model agents need an environment.

How environment types differ: actions, observations, state
Ask three questions before you compare vendors. What actions exist? What does the agent observe after each action? What state is hidden, and what state does the grader read? Two products can both be called an RL gym and still answer those questions differently. One might let the agent read a database it should not see. Another might grade the words while the record underneath stayed wrong.
A single-task environment proves a loop. A many-task environment is what training usually needs: shared tools, varied prompts, and a spread of difficulty. The type decision is about the loop. The task-set decision is about coverage. Do not buy the second because the first demo looked smooth.

Tool-use and MCP environments
Tool-use environments give the agent a list of functions and a schema for each call. The application state sits behind the tools. A good one distinguishes reads from writes, resets between attempts, and grades the final records rather than the politeness of the function names. MCP-style servers are one way to expose those tools. The protocol is not the environment. The hidden state and the check still have to be specified. The longer buyer page is tool-use RL environments for agents that call APIs.
Business work fits here when the job is already a sequence of system updates: assign a ticket, post a payment, change a stage. The records that might later seed that state are a different choice, described in how a buyer chooses among business record types. Choosing a record type is not the same as choosing an environment type.
Computer-use and browser environments
Computer-use environments let the agent click, type, and read pixels, an accessibility tree, or a page's structure. The world is a browser or a desktop that can be reset. The grader should read the file or the account that the clicks were supposed to change, not only the screenshot at the last frame. Screenshot-only checks are easy to satisfy with a picture that looks finished.
These environments are the right type when the software has no usable API, or when the skill you want is the screen itself. They are the wrong type when a stable tool would test the decision more cleanly. GUI tasks are still brittle. Specify the reset, the allowed applications, and the artifact the grader opens. That type is written out in computer-use RL environments: training agents on screens.
Enterprise software replicas
A replica, sometimes called a digital twin of an app, is a sandbox that behaves like a CRM, an ERP, or a ticketing system without touching production. Fidelity is the question: data, permissions, webhooks, and the ugly edge cases. A clone that only has the happy path will train an agent that fails on the real exceptions. Seeded records from a real company can make the state less generic, and they bring rights and privacy work the replica vendor does not own.
Ask what resets, whether parallel attempts collide, and how a version of the replica is pinned. A buggy clone wastes the training run. A faithful clone that includes personal data you were not allowed to hold is a different failure. The page for that type is enterprise software replicas as RL environments.
Simulated-user conversation environments
Multi-turn support and sales tasks need someone on the other side of the conversation. A simulated user has a hidden goal. The agent has a policy. The grader looks at the end state and, separately, at whether the things said were allowed. Grading only the transcript misses a refund that violated the rule. Grading only the database misses a promise the agent was not allowed to make.
Simulated users are a type, not a source of truth. Scenarios drawn from real support history can be more specific than a generic angry customer, and they can also smuggle personal details into the prompt. The type does not decide that tradeoff. Your brief does. The longer page is simulated-user environments for multi-turn agents.
Single-task vs many-task environments
One task is a unit test for the loop: reset, act, grade. A training set is many tasks that share the action space and vary the situation. If every task is a slight rewording of the first, you have not bought coverage. If the tasks do not share tools, you have bought several environments and called them one.
Say which you need. A benchmark of twenty hand-built tasks can be the right purchase. A thousand tasks with a loose grader can be the wrong one. Count is not quality.
Choosing a type for your training goal
Match the type to the skill. Tool use for decisions that are already API calls. Computer use when the interface is the job. A replica when you need the application's rules without production risk. A simulated user when the conversation is part of the outcome. Then name the grader in the same sentence as the type. A type without a check is a demo.
We do not stock these types. Describe the goal on the buyer request if you want a comparison with profiles holders have allowed to be discussed. Nothing on this page is a listing.
- Name the action space before you name a vendor.
- Say what the agent is forbidden to observe.
- Write the grader as a check on the ending, not as a vibe.
- Separate one illustrative task from the set you would actually train on.
Questions buyers ask
Is an RL gym the same as one of these types?
Gym is a loose label for a resettable world. The type is the action space and the observation. Ask which of the four you are being shown.
Can one environment mix tools and a screen?
Yes. Say so in the spec. A mix still needs one story of what the grader reads at the end.
Do you sell replicas of named products?
No. This page compares types. It does not claim a sandbox of any commercial application is available.
Where are worked examples?
The examples page shows specimens of named tasks. They are not customers and not a menu of types you can order.
Should every agent project start with computer use?
No. If the job is a tool call, a screen is extra surface area for the grader to get wrong.
Does picking a type guarantee a match?
No. A type is a way to write the need. A written need can still have no source.
Describe the data you need
Tell us the use, the records, and the limits. A request is not a promise that a matching dataset is available.
Tell us what data you need