Reinforcement learning environments
Tool-use RL environments for agents that call APIs
Last updated /Reinforcement learning environments
A tool-use RL environment lets an agent call functions instead of clicking a screen. The agent sees a schema. It sends arguments. The environment changes hidden state and later a grader reads that state. The transcript of the call is not the result. The ticket, the stage, or the ledger after the call is the result. This page is that type. The comparison of types is types of RL environments, and what each one trains. The map of the whole silo is reinforcement learning environments: what they are and what real work adds.
The neighboring types, now written, are computer-use RL environments: training agents on screens, enterprise software replicas as RL environments, and simulated-user environments for multi-turn agents. Records of real tool use, which are not yet an environment, are why records of software tool use matter to agents.

What a tool-use environment exposes
The agent is shown the tools it may call and the arguments each tool accepts. It is not shown the database those tools write, unless a tool is explicitly a read. A good package lists every function, says which ones change state, and resets that state between attempts. A package that dumps an admin client into the prompt and calls it an environment has skipped the spec.
Examples of the job, not of a product we sell: assign a ticket, post a note, move a CRM stage, or apply a support macro that updates a record. The example is the shape. It is not inventory.

Tool schemas, MCP servers and hidden state
A schema is the contract for one call: name, arguments, and what the return is allowed to reveal. Hidden state is everything the grader can see that the agent should not, such as the correct assignee or the balance before the attempt. If the correct ending is inside the tool description, the agent can copy it.
MCP is one published way to expose tools to a model. The project introduction is https://modelcontextprotocol.io/ . A server that speaks that protocol is not, by itself, an RL environment. You still need a reset, a hidden state, and a grader. Treat the protocol as plumbing. Treat the task as the thing you are buying.
Read actions vs write actions
A read can be safe and still leak the answer. A write can look successful in the log and leave the record unchanged. Separate them in the spec. Say which writes are allowed, which are forbidden, and which side effects must be undone on reset. An environment that permits every write is not a picture of a job with permissions.
Resets have to undo the writes. If two attempts share one database, the second attempt is graded on the first attempt's mess. That is not a harder task. It is a broken one.
How the result is graded
Grade the state the tools were supposed to change. A ticketing task can pass only when the ticket fields match the outcome you named, not when the function names sounded right. A support update can pass only when the refund flag, or the refusal, matches the policy. Partial credit is a choice you should see written down. A score with no readable check is a demo.
Where business workflows fit
IT queues, CRM stages, and support desks already look like tool calls: read the case, change a field, leave a note. That resemblance is why this type gets used for those jobs. It does not mean your export is an environment. Someone still has to name the allowed calls and the check, and the owner of the records still has to agree.
A screen is the wrong type when a stable API would test the decision more cleanly. A replica is a different purchase: a whole app, not a list of functions. Pick the type before you pick a vendor.
What to ask a vendor
- The full tool list, with reads and writes marked.
- What the agent cannot see, including the goal state.
- How reset restores the database, and whether parallel runs collide.
- The grader, in code or in a spec a second person can rerun.
- Which of IT, CRM, or support the sample actually implements. A label is not a sample.
We do not stock these environments. A request does not mean a sandbox of your ticketing API is waiting. Compare any sample with the list above yourself.
Questions buyers ask
Is an MCP server an RL environment?
No. MCP is a way to expose tools. An environment also needs a task, hidden state, a reset, and a grader.
Should the agent see the database?
Only through the reads you named. The goal state used for grading stays hidden.
Do you sell a ticketing sandbox?
No. The ticketing example on this page is a shape. It is not a product.
Can one environment mix tools and a screen?
Yes, if the spec says so. You still need one account of what the grader reads at the end.
Is a function-call log the grade?
Not by itself. The grade is the state after the calls. A log can show the path. It is not the outcome.
Will you certify a vendor's tools?
No. Use the questions above. We do not stamp schemas.
Describe the data you need
Tell us the use, the records, and the limits. A request is not a promise that a matching dataset is available.
Tell us what data you need