hopper-parity-notune
Hopper (MuJoCo Hopper-v5) · solved 3 of 9 agent attempts pass
more
The fast rung (18 m in 8 s, alive, on 90 % of seeds) plus a mean return within 10 % of SB3-zoo TD3 measured on the grader's seeds (3532 -> 3179). Solved in the 2026-10-23 sweep by two of eight sessions: the oracle is Sol 5.6's design (19.0 m, return 3377, 4.4 % below TD3), Opus 5.5 reached 18.3 m / 3289; a third Opus session built a sampling-based MPC on MuJoCo itself that exceeded the verifier's time budget. The fast designs fall on every seed under -30 % friction or -20 % motor gain, like TD3.
Gate. per-seed success is the `fast` rung of this benchmark's ladder ['alive', 'forward', 'brisk', 'fast'] (`--success-mode fast`), not the evaluator's default flag; the task is solved when the success rate over the grader's held-out seeds is `>= 0.9`; the aggregate `total_reward_mean` over the grader's held-out seeds (base scenario) must be `>= 3179` (rung `metric:total_reward_mean`)
Axes: plant:hopper, scenario:standard_v0, gate:fast_18m, rl-parity:td3-3532-minus-10pct, ablation:no-tuner
Each video re-runs the attempt's final controller through the task's own evaluator on a public seed (never the grader's held-out ones). The table under each video is the official grade on the held-out seeds.
standard_v0| criterion | graded | needed |
|---|---|---|
| base:fast | 0.975 | ≥ 0.900 |
| metric:total_reward_mean | 3515 | ≥ 3179 |
The agent's final message
The controller in `/app/work/controller.py` passes the gate on every seed set I tried. Total time used was about 46 of the 60 minutes. | Seeds | `fast` rate (gate ≥ 0.9) | mean return (gate ≥ 3179) | |---|---|---| | 0–4 (the evaluator's command) | 1.00 | 3584 (20.7 m mean) | | 0–9 | 1.00 | 3579 | | 10–49 | 0.95 | 3428 | | 50–149 | 0.99 | 3575 | That is about 97% of 150 seeds reaching 18 m, with a return on par with the TD3 reference (3532). The failures are torso pitch going past 0.2 rad. **Control law:** `u = clip(W · φ, −1, 1)` - `φ` is the 11 state values measured from the standing pose and scaled to roughly unit size. They are split into a stance copy and a flight copy using `foot_contact`, plus a stance-only constant and a general constant. That gives one 3×11 gain matrix for stance, one for flight, and two offset columns. - The docstring explains what each block of gains does: the attitude servo, the knee thrust and stiffness, the toe push-off, foot placement in flight, and folding the leg. - The commands often hit their limits, so in practice the gait behaves like switched on/off control. **How W was found:** black-box CMA-ES over its 72 entries, with no neural network and no gradients. Moving with a reduced survival bonus in the objective was what got the search past "stand still". The final round optimised the true return on 6 and then 12 training seeds to make it robust. Things to be aware of: - **Not a hand-designed controller.** I started with a Raibert-style…
The controller scales the Hopper observation, splits it into stance and flight copies by foot contact, and applies a CMA-ES-tuned linear matrix with clipping to output the three joint torques.
Shown: public seed 0 (standard_v0); this episode passes the fast rung. This episode: episode_steps 1000, x_final 20.6, min_height 1.04, max_abs_pitch 0.17, n_flights 13, total_reward 3570.
controller.py · agent transcript · trial hopper-parity-notune__D6idt5p
standard_v0| criterion | graded | needed |
|---|---|---|
| base:fast | 0.000 | ≥ 0.900 |
| metric:total_reward_mean | 3043 | ≥ 3179 |
The agent's final message
The final controller is in place at `/app/work/controller.py`. It does not reach the fast gate, and I'm reporting that plainly. **Result on the visible seeds 0 to 4 (official evaluator)** | Metric | Value | Gate | |---|---|---| | alive / forward / brisk | 1.00 / 1.00 / 1.00 | | | fast (x ≥ 18 m) | 0.00 | ≥ 0.9 | | total_reward_mean | 3126 | ≥ 3179 | | x_final_mean | 17.0 m | | On 12 held-out seeds of my own, it averaged 16.9 m with a return of 3107 and never fell. The grader's fast rung and return gate are both missed by a small margin, so under the stated scoring this submission does not pass. **What the controller is.** A two-mode linear state feedback law: the foot-contact flag selects a stance or flight gain set, and each motor command is a bias plus gains on the scaled measurements (pitch and rate, the three joint angles and rates, forward and vertical velocity, height, and time since the last contact change). The docstring in the file explains each gain's role. The tuned gait locks the knee straight and pogo-hops on the ankle at about 2.0 m/s, with the hip holding torso pitch inside roughly ±0.1 rad. **How it was built.** Hand-structured Raibert-style controllers (fold-then-thrust stance, foot placement in flight) hopped in place but could not move forward without pitching over, because the leg is far heavier than the torso and cannot be swung in flight. The breakthrough was two-fold: switching to the linear two-mode structure, and tuning it with CMA-ES on distance…
On each step the foot-contact flag selects stance or flight gain matrices, and joint commands are the clipped affine combination of scaled body state, velocities, height error and phase time, with all gains found by CMA-ES.
Shown: public seed 0 (standard_v0); this episode fails the fast rung. This episode: episode_steps 1000, x_final 16.8, min_height 1.04, max_abs_pitch 0.0959, n_flights 15, total_reward 3096. It was the first failing one of 1 public seeds tried.
controller.py · agent transcript · trial hopper-parity-notune__J284DsR