Reinforcement learning environments

An RL environment requirements template

Last updated /Reinforcement learning environments

An RL environment requirements template is the page you fill before you ask for a quote. It names the training use, the domain, the tools, the hidden state, the tasks, the grader, the difficulty, the delivery, and the rights. A paragraph of adjectives produces a paragraph of adjectives. A filled template can be compared. This is not a form we score, and filling it does not mean a matching environment exists.

The buying sequence is how to buy RL environments and task sets. The pillar is reinforcement learning environments: what they are and what real work adds. The pages that use this spec later are acceptance testing an RL environment, licensing RL environments and task sets, and seed data privacy in RL environments. A brief for records, which is a cousin and not this template, is what belongs in an AI dataset requirements brief.

Filled-in requirements template with sections for tools, tasks and grader, labeled Buyer guide
A blank template filled with tools, tasks, and a grader. Buyer guide is the chip, not a completed order.

Goal and training use

Say what the training run is for: reinforcement learning on the tasks, evaluation only, or supervised updates on solved trajectories. Those are different permissions and different slices. A goal of 'make the agent better at work' is not a use. Name the job the agent should become able to finish, and say what done looks like before you name a vendor.

Filled-in requirements template with sections for tools, tasks and grader, labeled Buyer guide
A blank template filled with tools, tasks, and a grader. Buyer guide is the chip, not a completed order.

Domain and workflow

Name the domain in the language of the work: IT tickets of one class, a CRM stage change, a three-way match, a support refund. Name the workflow as a person would recognize it, not as a software brand. If two workflows do not share tools or a check, they are two specs. Do not staple them to look thorough.

Action space and tools

List the actions. Tool calls, clicks, or messages to a simulated user. For tools, mark reads and writes. Say what the agent must not see, including the goal state. A schema with every admin function is not a job. It is an unbounded client.

State and seed data

Describe the starting state and where the seed comes from: synthetic, real records with personal details removed, or a mix. Say which fields are in the observation and which are only for the grader. If the seed is real, the privacy page is the next constraint, not a footnote.

Tasks and difficulty

Say how many tasks, of what kind, and how difficulty will be measured: which model, which grader, how many attempts. Do not paste a percent with no denominator. A task the model never passes needs a blind human solve before you treat it as hard rather than impossible. The numbers people quote in interviews are not your acceptance band unless you write them into this spec yourself.

Grader and evidence

Name the grader type: a state comparison, tests, a rubric, or a combination. Say what evidence you will receive: source or a readable spec, passes, fails, and a cheat that must fail. A grader described as 'AI scored' is not filled in.

Delivery format and runtime

Say how you will run it. Reset and step, a container, a version you can pin, and a second copy that does not see the first. A notebook that only the author can reset is not a delivery format. If you cannot name the runtime, you cannot accept the package later.

Rights and exclusivity

Say whether the package may be sold to others, which uses are in, and who owns the seed. Exclusive is a choice with a cost, not a default. Leave the clause writing to counsel. This template only forces the question onto the page before the quote.

Leave a field blank only when you have decided it does not apply, and write 'not in scope' so a vendor cannot fill it with a surprise. A blank that means 'we forgot' will be priced as if you had agreed. Compare two quotes only after both vendors answered the same blanks. If they answered different blanks, you do not have two prices for one environment.

  • Use, domain, and the definition of done.
  • Tools, hidden state, and seed source.
  • Tasks, difficulty method, grader, and runtime.
  • Rights, and whether the deal is exclusive.

Send the filled page with the buyer request if you want a need compared with profiles. A completed template is not a match.

Questions buyers ask

Is this template a contract?

No. It is the spec you write before a quote. Counsel still reviews any license.

Do you fill it in for us?

No. The blanks are yours. We do not invent your domain or your grader.

Can one template cover training and evaluation?

Only if you say both uses and accept that the slices and the rights may differ. Otherwise split them.

What if we do not know the task count yet?

Say the range you will accept and what would make you stop. A blank count is not a spec.

Does sending this guarantee a source?

No. A written need can still have no package behind it.

Where does acceptance fit?

After a sample exists. The acceptance page is the test. This page is what the test is measured against.

Describe the data you need

Tell us the use, the records, and the limits. A request is not a promise that a matching dataset is available.

Tell us what data you need