lunar-lander-standard

Lunar lander (Box2D) · solved 10 of 11 agent attempts pass

The task. The June 2026 harness needed 8 chained runs and 138 proposals (~40 h) to reach its accepted controller (reward 268.6); the oracle is that controller.

First Box2D task in the suite; the image installs gymnasium[box2d] from source.

Gate. the task is solved when the success rate over the grader's held-out seeds is `>= 0.9`

Axes: plant:box2d-lunar-lander, scenario:standard_v0, actuation:two-engine-continuous

Each video re-runs the attempt's final controller through the task's own evaluator on a public seed (never the grader's held-out ones). The table under each video is the official grade on the held-out seeds.

Best attemptSol (cursor-cli (pre-clock), 3 min) · standard_v0

Graded on the held-out seeds: passes every gate on the grader's held-out seeds.

criterion	graded	needed
base:default	1.000	≥ 0.900

How the agent got there: 3 min · 20 tool calls · 3 edit-tool edits · 8 runs of its own Python (tests, experiments, rewrites) · no tuner runs · official verifier 3× (success rates in order: 0.00, 1.00, 1.00).

The agent's final message

Implemented `/app/work/controller.py` with a readable feedback controller. Validation: - Visible seeds 0–4: 100% achievement and terminal success - 5,000 nominal seeds: 100% success, zero crashes - Minimum reward: 233.64 - Syntax check passedTask complete. The final controller passes all visible seeds and 5,000 nominal stress-test seeds with zero crashes. No follow-up is needed.

Controller family: PID/PD

A hand-tuned PD cascade computes a desired lean angle from lateral position and velocity to drive the side jet, while the main engine tracks a glide-slope height and descent rate, switching to pure vertical-speed damping once a leg touches down.

Shown: public seed 0 (standard_v0); this episode passes the default rung. This episode: total_reward 305, crashed no, landed yes, final_landing_error 0.0511.

controller.py · trial lunar-lander-standard__Sd2cQUr

FailureCodex (GPT) (cursor-cli (pre-clock), 2 min) · standard_v0

Graded on the held-out seeds: misses the main gate narrowly (success rate between 0.5 and the gate).

criterion	graded	needed
base:default	0.875	≥ 0.900

How the agent got there: 2 min · 12 tool calls · 3 edit-tool edits · 0 runs of its own Python (tests, experiments, rewrites) · repository tuner 2× · official verifier 5× (success rates in order: 0.00, 0.00, 0.20, 1.00, 0.94).

Pre-clock session: the agent had no clock and may have stopped early believing its hour was over; treat a failure here as a lower bound.

The agent's final message

Implemented a hand-written continuous controller in `/app/work/controller.py` and tuned its gains using the provided non-gradient tuner. What changed: - Replaced zero-action policy with a heuristic PD controller: - lateral position/velocity -> target tilt - lateral tilt/rate -> side-engine command - altitude/vertical speed -> main-engine command - touchdown mode when legs contact to damp vertical speed and settle - Kept gains in `PARAMS` and bounds in `PARAM_BOUNDS` for readability/tunability. - Added a short docstring describing the control law and gain roles. Validation runs: - `seeds 0-4`, `standard_v0`: **success_rate = 1.000** (5/5), ladder `achievement=1.00`, `terminal=1.00`. - Extra check `seeds 5-20`, `standard_v0`: **success_rate = 0.938** (15/16), matching the tuner’s held-out estimate and above the 0.9 target. Your final deliverable is in place at: - `/app/work/controller.py`

Controller family: lookup table / heuristic rules

The controller computes a target tilt and hover height from lateral position and velocity, applies PD corrections to angle and altitude, and maps them to main and side engine commands, with a gentler touchdown mode and engine cut-off once landed.

Shown: public seed 16 (standard_v0); this episode fails the default rung. This episode: total_reward 196, crashed no, landed yes, final_landing_error 0.0193. It was the first failing one of 17 public seeds tried.

controller.py · trial lunar-lander-standard__xaQfHbP