Reinforcement learning environments

Acceptance testing an RL environment

Last updated /Reinforcement learning environments

Acceptance testing an RL environment is the decision to pay, or not, after a sample. Reset it twice. Break the grader. Read the pass-rate method. Ask for a blind solve. Read the docs far enough to rerun the check without the vendor. Arrival of a container is not acceptance. The buying path is how to buy RL environments and task sets. The pillar is reinforcement learning environments: what they are and what real work adds.

The spec you test against is an RL environment requirements template. The license is still licensing RL environments and task sets. The seed is still seed data privacy in RL environments. A file delivery has a similar habit of checking format and version before you call it accepted: what a receiving team checks before accepting a dataset.

Acceptance test report with pass and fail rows for one environment, labeled Buyer guide
A report with pass and fail rows. The rows are an illustration, not a test we ran.

Reset and repeatability

Run reset, take a few actions, reset again, and confirm the start matches. Run a second copy at the same time and confirm it does not see the first copy's writes. If the second episode begins where the first ended, you are not looking at a training environment. You are looking at a shared scratchpad. The report drawing on this page is an illustration of pass and fail rows. It is not a report from a package we tested.

Acceptance test report with pass and fail rows for one environment, labeled Buyer guide
A report with pass and fail rows. The rows are an illustration, not a test we ran.

Grader soundness and hack tests

Try a cheat the spec says must fail: the answer left in a readable file, a status flag set with no other work, a string the test echoes. A sound grader fails those. Epoch AI's FAQ treats that property, a high reward only when the task is solved, as a top quality bar from their interviews: https://epoch.ai/gradient-updates/state-of-rl-envs . That is their report. Your test is whether this sample fails the cheat you can think of.

Pass-rate spread on your model

Ask for the model name, the grader version, and the number of attempts behind any rate. The same FAQ says interviewees did not want tasks at 0 percent or 100 percent, and described a minimum around 2 to 3 percent, about one success in 64 or 128 rollouts. Those figures are not your cutoff unless your spec says so. A single percent with no denominator is not a spread. A histogram without the method is a picture.

Blind human solves

A task the model never passes might be hard or impossible. The FAQ notes that a zero is not proof of difficulty unless someone else solved it without having written the grader. Ask for that blind solve on the tasks you will keep. An author solving their own task while looking at the answer does not count.

Throughput and runtime

You need to know whether the sample can run the number of attempts your training plan assumes, on the runtime in the spec. We do not publish a throughput number. If the vendor quotes one, ask how it was measured and on what hardware. A demo that only runs while they share a screen has no throughput you can accept.

Documentation and versioning

The docs should let a person who was not on the call reset, step, and rerun the grader. The version should be nameable, so a later drop is not a silent replacement. Silence after download is not acceptance. Name the person who marks the rows passed.

Write the misses down before anyone starts a training job. A chat reaction is not a record. If the sample fails reset, stop there. Do not 'temporarily' train on a world that drifts, and do not treat a vendor's summary of the test as your notes. The point of the sample is that your team can see the failure without the author's narration. If you cannot name who will mark the rows, the test does not have an owner and it will not happen.

  • Reset twice. Parallel copies do not collide.
  • A cheat fails for the reason the spec names.
  • Any pass rate names the model, the grader, and the attempt count.
  • A blind solve exists for tasks the model never passes.
  • The version is written down.

We do not run this test for you. A sample that looks busy is not a pass.

Questions buyers ask

Is download the same as acceptance?

No. Acceptance is a recorded pass on reset, the grader, the measurement, and the version.

Do we have to use the 2 to 3 percent band?

No. That band is from Epoch's interviewees, at the URL above. Your spec sets the band.

Who marks the report?

Your receiving team. This checklist does not mean our staff executed it.

Can acceptance replace the license review?

No. A sample can pass every row and still be outside the grant. Keep the license questions open.

What if the cheat passes?

Do not accept the sample. Send the miss back in writing before anyone trains.

Are the rows on the image a real report?

No. They show the shape of a report. They are not results.

Describe the data you need

Tell us the use, the records, and the limits. A request is not a promise that a matching dataset is available.

Tell us what data you need