Reinforcement learning environments

How to buy RL environments and task sets

Last updated /Reinforcement learning environments

How to buy RL environments is a procurement sequence, not a catalog. You decide whether to build, buy, or partner. You name the world, the tasks, and the grader before you ask for quotes. You test a sample before payment. You license the uses you will actually run, including whether the deal is exclusive. This page does not publish a Big Data/Little Guy price. Where a public range exists, it is quoted from Epoch AI and linked, and it is not an offer from us.

The definition of the object is reinforcement learning environments: what they are and what real work adds. Roles, without company names, are who builds RL environments and who pays for them. What makes the work heavy, still without a rate card of ours, is what drives the cost of an RL environment.

Buyer path from scope to vendor, sample test, license and delivery, labeled Buyer guide
Scope, vendor, test, license. Buyer guide is the chip. This site does not quote an environment price.

Build, buy or partner

Build when the skill is core and you can keep the grader honest in house. Buy when a vendor already has the world and you can test it. Partner when the missing piece is someone else's operating history, which has an owner who can refuse. Mixing the three in one sentence is how teams skip the test. Pick one primary path for the first purchase, and write down what would make you switch.

Sourcing the underlying records, when the path is a license from a company rather than a finished environment, is how a buyer sources proprietary business data. That guide does not sell environments either.

Buyer path from scope to vendor, sample test, license and delivery, labeled Buyer guide
Scope, vendor, test, license. Buyer guide is the chip. This site does not quote an environment price.

The three kinds of vendors

One kind builds tasks and graders in a domain you name. Another sells a platform: resets, tooling, and a place to run attempts. A third sits on real workflows and needs the data owner's permission before anything is packaged. The same firm can wear more than one hat. Ask which hat is on for this quote. A platform fee is not a task set. A task set is not a right to someone else's tickets.

We introduce a stated need to a profile when one exists. We are not the vendor of a finished environment, and we are not the lab that trains on it. The role page is the longer version of that split.

What to specify before you ask for quotes

Write the domain, the tools or the screen, the hidden state, the number and kind of tasks, the grader, the difficulty band and how it will be measured, the delivery format, and the rights. A quote against a paragraph of adjectives will come back as a paragraph of adjectives. A quote against a spec can be compared. The field list is an RL environment requirements template.

  • The action space and what the agent must not see.
  • What a passing attempt changes, in the system of record or the file.
  • Which uses you need: training, evaluation, or solved trajectories for supervised fine-tuning.
  • Whether the package may be sold to others.

How pricing is quoted (cited ranges only)

We do not quote environment prices. Epoch AI's FAQ, based on interviews with people at environment startups and labs, reports ranges those interviewees described. The page is https://epoch.ai/gradient-updates/state-of-rl-envs . Multiple interviewees put a common task, with its grader, around $200 to $2,000. One founder said $20,000 for a single task would be rare but possible, and the FAQ ties that rare figure to especially complex software-engineering tasks.

On environments rather than single tasks, the same FAQ says SemiAnalysis reported website replicas around $20,000 each, and one interviewee mentioned figures around $300,000 for a higher-fidelity replica of a complex product. Contract comments in the FAQ include six to seven figures per quarter, and one researcher mentioned seeing contracts in the $300,000 to $500,000 range, varying with the number of tasks. Two founders told Epoch that exclusive deals run roughly four to five times non-exclusive ones. Epoch also cites a Mechanize estimate of about $2,400 in compute spent per task during training, as context for why a cheap, weak task can waste the run.

Those are their reported ranges, not invoices, not a median we measured, and not a Big Data/Little Guy price. Read the FAQ before you repeat a number. Our own page on cost drivers still has no rate card, on purpose.

Acceptance testing before payment

Pay for a sample before you pay for a library. Reset the world and run it twice. Confirm the grader fails the attempts that should fail, including a cheat. Ask for the pass-rate method rather than a single percent. Read the documentation far enough to rerun the check without the vendor on the call. A container that arrives is not an accepted environment. The checklist is acceptance testing an RL environment.

The questions to put in that test, and the contract questions beside them, belong in your buying checklist. This page names them. It does not perform them for you.

Licensing and exclusivity

The license should say what you may do with the environment and with any seed data inside it: reinforcement learning, evaluation, supervised fine-tuning on solved traces, or some narrower set. Exclusive means others do not get the same package. The Epoch interviews above are the source for the idea that exclusivity is priced at a multiple. Whether you should pay that multiple is your decision, not a rule. The longer page is licensing RL environments and task sets.

Questions to take to counsel, without clause text from us, are questions buyers should ask counsel before a training license. That list is not an environment contract.

Privacy of seed data

If the environment was seeded from a real company, ask how personal details were removed, what was synthesized, and who still has access at run time. A realistic sequence that keeps a customer's identity is not a better product. It is a problem you inherited. Credentials and secrets do not belong in the seed. What to check is seed data privacy in RL environments.

Describe what you need on the buyer request. A request is not a promise that a package exists, and it is not a quote. We will not turn the Epoch ranges into a price list with our name on it.

Questions buyers ask

Will you tell us what to pay?

No. The only figures on this page are the ones Epoch AI reported at the URL in the pricing section. They are not our prices.

Is exclusive always worth the multiple?

Epoch's interviewees described a multiple. They did not say you should pay it. Ask what you are keeping from competitors and for how long.

Do you sell RL environments?

No. We do not operate a shelf of finished environments. A profile, when one exists, is a description a holder allowed.

Can we skip the sample if the demo looked good?

You can, and you will be buying the demo. The sample is how you see the grader fail.

Does a data-sourcing brief replace this spec?

No. A brief for records and a spec for an environment answer different questions. You may need both. Neither guarantees a source.

Who accepts the delivery?

Your team. This page does not mean we ran the reset or signed the license.

Describe the data you need

Tell us the use, the records, and the limits. A request is not a promise that a matching dataset is available.

Tell us what data you need