hopper-forward
Hopper (MuJoCo Hopper-v5) · solved 16 of 22 agent attempts pass
more
Solved by stock agents in one session (2026-10-23 sweep: 10/15 sessions clear forward; best 11.0 m, return 2379, PPO level); the oracle is that design (Opus 5.5: ankle centre-of-pressure balance, knee thrust, tuner-fitted gains). SB3-zoo TD3 scores 3532 (20 m) on the grader's seeds; the hand-written Raibert-style structure hopped in place.
Gate. per-seed success is the `forward` rung of this benchmark's ladder ['alive', 'forward', 'brisk', 'fast'] (`--success-mode forward`), not the evaluator's default flag; the task is solved when the success rate over the grader's held-out seeds is `>= 0.9`
Axes: plant:hopper, scenario:standard_v0, gate:forward_3m, rl-reference:td3-3532
Each video re-runs the attempt's final controller through the task's own evaluator on a public seed (never the grader's held-out ones). The table under each video is the official grade on the held-out seeds.
standard_v0| criterion | graded | needed |
|---|---|---|
| base:forward | 1.000 | ≥ 0.900 |
The agent's final message
The controller in `/app/work/controller.py` now passes the gate. It hops forward at about 1.6 m/s, roughly 13 m in 8 s, where the `forward` rung needs only 3 m. I finished at about 38 minutes of the 60. **Results with the `forward` success mode, nominal plant (`standard_v0`):** - **Visible seeds 0–4 (official evaluator):** 5/5, average distance 12.9 m. - **Seeds 3000–3099 (official evaluator, never used for tuning):** 100/100. - **Seeds 100–159 and 200–999 (my own copy of the simulation loop):** 860/860. These were used only to choose between candidate parameter sets, not to tune them. The earlier parameter sets scored 96–98% on those same seeds. The failures were always the same: speed crept up from hop to hop until the leg trailed too far behind and the torso dropped below 0.7 m. The final set hasn't hit that on any seed I've run, but it has no explicit speed limiter, so that remains the most likely way for it to fail. **How it works:** - **Centre of mass:** a planar model of the robot's link lengths and masses gives the centre of mass relative to the foot, and its forward speed, at every step. - **Stance:** the hip holds the torso pitch near −0.09 rad. The ankle shifts the foot's pressure point so the centre of mass stays at a speed-dependent spot over the foot. The knee crouches like a soft spring, then pushes off. The push stops just before the knee straightens fully, because hitting the joint limit jolts the torso. The ankle adds a push at take-off. - **Flight:** the…
A hand-designed hop-cycle state machine estimates COM position and speed from a planar kinematic model, regulates torso pitch with hip PD in stance, thrusts the knee and ankle for push-off, and places the foot in flight using Raibert's speed-based foot-placement law, with parameters refined by random local search.
Shown: public seed 0 (standard_v0); this episode passes the forward rung. This episode: episode_steps 1000, x_final 12.9, min_height 1.04, max_abs_pitch 0.139, n_flights 14, total_reward 2607.
controller.py · agent transcript · trial hopper-forward__EWa6sDK
standard_v0| criterion | graded | needed |
|---|---|---|
| base:forward | 0.000 | ≥ 0.900 |
The agent's final message
The controller in `/app/work/controller.py` does not pass the gate. It stays alive for all 1000 steps on seeds 0–4, but it only travels about 1 m, and the task needs at least 3 m. On the five visible seeds (`standard_v0`, `--success-mode forward`) the ladder reads alive 1.00, forward 0.00, brisk 0.00, fast 0.00. The final x is about 0.93–1.02 m per seed, and the return is about 1120. Because I'm submitting under the 3 m bar, the expected score is zero. I did not run any of the perturbation scenarios. **What I tried:** - **Hand-written controllers.** Two Raibert-style state-machine controllers, tuned by random search, collapsed to standing still or fell. - **Periodic-gait search.** An evolution-strategy search over a controller with a periodic knee, hip and ankle pattern found fast hops at about 2–4 m/s. These exploited an anti-damped ankle "windmill" and fell between about 2 and 3 s every time, so none could be submitted. - **Sign finding.** A negative hip command pitches the torso back, in stance and in flight. My first feedback sign was wrong, and fixing it removed the ankle exploit. - **Final search.** With the corrected hip sign and the ankle gains bounded, the search found a stable low-amplitude shuffle. It creeps forward about 1 m in 8 s and then stalls. Larger perturbation steps (σ) did not improve it; the one variant I ran fell at step 976. The file's docstring explains the control law and each gain. The parameters are plain constants in the file. I stopped at abo…
A touchdown-resynchronized sinusoidal clock drives periodic crouch/extend patterns on all three joints, with added PD pitch stabilization on the hip, PD foot leveling on the ankle, velocity-error feedback, and a knee thrust during upward contact, with gains found by evolution strategy.
Shown: public seed 0 (standard_v0); this episode fails the forward rung. This episode: episode_steps 1000, x_final 0.975, min_height 0.801, max_abs_pitch 0.124, n_flights 1, total_reward 1122. It was the first failing one of 1 public seeds tried.
controller.py · agent transcript · trial hopper-forward__KjzAsw2