hopper-robust
Hopper (MuJoCo Hopper-v5) · solved 8 of 15 agent attempts pass
more
SB3-zoo TD3 (3532 nominal) falls on every seed under -30 % friction and -20 % motor gain; the two fastest agent designs from the flat sweep (10-11 m) do too; the oracle is the rails-first Opus design (6.9 m) that survives all four on 40/40 seeds.
Gate. per-seed success is the `forward` rung of this benchmark's ladder ['alive', 'forward', 'brisk', 'fast'] (`--success-mode forward`), not the evaluator's default flag; the task is solved when the success rate over the grader's held-out seeds is `>= 0.9`; the same gate must also hold on the sibling scenario(s) `heavy_torso_20`, `low_friction_30`, `weak_motors_20`, `gravity_high_10` (`--scenarios`)
Axes: plant:hopper, scenario:standard_v0+physics-suite, gate:forward_3m, robustness:torso-friction-motors-gravity
Each video re-runs the attempt's final controller through the task's own evaluator on a public seed (never the grader's held-out ones). The table under each video is the official grade on the held-out seeds.
standard_v0| criterion | graded | needed |
|---|---|---|
| base:forward | 1.000 | ≥ 0.900 |
| scenario:heavy_torso_20 | 1.000 | ≥ 0.900 |
| scenario:low_friction_30 | 1.000 | ≥ 0.900 |
| scenario:weak_motors_20 | 1.000 | ≥ 0.900 |
| scenario:gravity_high_10 | 1.000 | ≥ 0.900 |
The agent's final message
The controller is finished and verified. Time used is about 38 of the 60 minutes. **Result.** `/app/work/controller.py` passes the official evaluator on every visible seed and scenario at the `forward` rung. On my own 100-seed sweep it passes 497 of 500 rollouts across the five scenarios, with the worst scenario at 98 of 100, so the 0.9 gate has a comfortable margin on held-out seeds. | Scenario | Seeds 0-99 pass | Typical distance | |---|---|---| | standard_v0 | 99/100 | 9 m | | heavy_torso_20 | 100/100 | 9 m | | low_friction_30 | 100/100 | 8 m | | weak_motors_20 | 98/100 | 11 m | | gravity_high_10 | 100/100 | 9 m | **How the controller works.** It is a stance/flight state machine. In stance a posture servo re-establishes a forward-leaning crouch: the ankle sets the shank lean, the knee holds the crouch angle, the hip holds torso pitch. The shank lean is shifted by the speed error, which is what regulates forward speed. After a short settle time the knee and ankle fire a brief thrust pulse. In flight the knee holds a fixed fold, the hip damps pitch, and the ankle pre-sets a flat foot. The gait is a bounding hop near 1 m/s with about 15 flights per episode and a mean return around 2100. **What mattered along the way.** - Probing the plant showed that with the foot planted a positive hip command pitches the torso nose-down, the opposite of the flight case. My first attitude loop had the sign inverted. - The leg is far heavier than the torso, so foot placement through the h…
On each touchdown the controller settles into a crouch using PD servos on pitch, shank lean (biased by speed error) and knee, then fires a brief knee and ankle thrust, and in flight holds knee and flat foot, regulates pitch with clipped hip torque, and pushes the thigh away from its limit.
Shown: public seed 0 (standard_v0); this episode passes the forward rung. This episode: episode_steps 1000, x_final 8.88, min_height 1.04, max_abs_pitch 0.105, n_flights 15, total_reward 2109.
controller.py · agent transcript · trial hopper-robust__mrcDmWC
heavy_torso_20| criterion | graded | needed |
|---|---|---|
| base:forward | 1.000 | ≥ 0.900 |
| scenario:heavy_torso_20 | 1.000 | ≥ 0.900 |
| scenario:low_friction_30 | 1.000 | ≥ 0.900 |
| scenario:weak_motors_20 | 1.000 | ≥ 0.900 |
| scenario:gravity_high_10 | 1.000 | ≥ 0.900 |
The agent's final message
The controller is finished and verified. Time used is about 38 of the 60 minutes. **Result.** `/app/work/controller.py` passes the official evaluator on every visible seed and scenario at the `forward` rung. On my own 100-seed sweep it passes 497 of 500 rollouts across the five scenarios, with the worst scenario at 98 of 100, so the 0.9 gate has a comfortable margin on held-out seeds. | Scenario | Seeds 0-99 pass | Typical distance | |---|---|---| | standard_v0 | 99/100 | 9 m | | heavy_torso_20 | 100/100 | 9 m | | low_friction_30 | 100/100 | 8 m | | weak_motors_20 | 98/100 | 11 m | | gravity_high_10 | 100/100 | 9 m | **How the controller works.** It is a stance/flight state machine. In stance a posture servo re-establishes a forward-leaning crouch: the ankle sets the shank lean, the knee holds the crouch angle, the hip holds torso pitch. The shank lean is shifted by the speed error, which is what regulates forward speed. After a short settle time the knee and ankle fire a brief thrust pulse. In flight the knee holds a fixed fold, the hip damps pitch, and the ankle pre-sets a flat foot. The gait is a bounding hop near 1 m/s with about 15 flights per episode and a mean return around 2100. **What mattered along the way.** - Probing the plant showed that with the foot planted a positive hip command pitches the torso nose-down, the opposite of the flight case. My first attitude loop had the sign inverted. - The leg is far heavier than the torso, so foot placement through the h…
On each touchdown the controller settles into a crouch using PD servos on pitch, shank lean (biased by speed error) and knee, then fires a brief knee and ankle thrust, and in flight holds knee and flat foot, regulates pitch with clipped hip torque, and pushes the thigh away from its limit.
Shown: public seed 0 (heavy_torso_20); this episode passes the forward rung. This episode: episode_steps 1000, x_final 10.2, min_height 1.05, max_abs_pitch 0.0978, n_flights 15, total_reward 2271.
controller.py · agent transcript · trial hopper-robust__mrcDmWC
weak_motors_20| criterion | graded | needed |
|---|---|---|
| base:forward | 0.925 | ≥ 0.900 |
| scenario:heavy_torso_20 | 1.000 | ≥ 0.900 |
| scenario:low_friction_30 | 1.000 | ≥ 0.900 |
| scenario:weak_motors_20 | 0.825 | ≥ 0.900 |
| scenario:gravity_high_10 | 1.000 | ≥ 0.900 |
The agent's final message
`/app/work/controller.py` is in place, but it sits right at the 0.9 gate, so it may not pass. It is a Raibert-style hopping state machine. The gains came from a CMA-ES search over its 26 parameters. That is gradient-free black-box tuning, not network training. The docstring in the file explains each gain. On 20 seeds I hadn't tuned on (20–39), per-scenario success was: | Scenario | Success | |---|---| | `standard_v0` | 18/20 | | `heavy_torso_20` | 20/20 | | `low_friction_30` | 20/20 | | `weak_motors_20` | 18/20 | | `gravity_high_10` | 20/20 | - **Speed and falls:** Successful runs finish alive at roughly 4–7 m, enough for `forward` but not `brisk`. The failures are falls, which can happen on any seed. - **Variance:** Seeds 0–3 gave 19/20, with `standard_v0` seed 2 falling after 214 steps. The grader's seeds could land either side of 0.9. - **Not required for the gate:** It fails `noisy_delay_v1` (falls within a few hundred steps), which isn't one of the scenarios the gate checks. - **Official verifier:** I ran it on seeds 20–39 against an earlier candidate, not the final file. That candidate scored 1.00 on `standard_v0`, 0.70 on `heavy_torso_20`, 1.00 on `low_friction_30`, 0.95 on `weak_motors_20` and 1.00 on `gravity_high_10`. The final controller beat it on heavy torso, but I only checked that with my own runner. I stopped at about 52 of the 60 minutes. A further tuning pass over more seeds made `weak_motors_20` collapse to 0/20, so I discarded it and kept the earlier …
A contact/flight state machine compresses then thrusts the knee in stance while hip and ankle PD loops regulate pitch, and in flight it folds and extends the leg and sets thigh angle from forward velocity error for foot placement.
Shown: public seed 5 (weak_motors_20); this episode fails the forward rung. This episode: episode_steps 878, x_final 4.73, min_height 1.14, max_abs_pitch 0.202, n_flights 11, total_reward 1467. It was the first failing one of 6 public seeds tried.
controller.py · agent transcript · trial hopper-robust__b5JkS8T
standard_v0| criterion | graded | needed |
|---|---|---|
| base:forward | 0.825 | ≥ 0.900 |
| scenario:heavy_torso_20 | 0.975 | ≥ 0.900 |
| scenario:low_friction_30 | 1.000 | ≥ 0.900 |
| scenario:weak_motors_20 | 1.000 | ≥ 0.900 |
| scenario:gravity_high_10 | 1.000 | ≥ 0.900 |
Pre-clock session: the agent had no clock and may have stopped early believing its hour was over; treat a failure here as a lower bound.
The agent's final message
Final state confirmed. The controller at `/app/work/controller.py` passes the `forward` gate comfortably on all five scenarios over seeds 0–39: | scenario | success rate (40 seeds) | |---|---| | standard_v0 | 0.95 | | heavy_torso_20 | 1.00 | | low_friction_30 | 1.00 | | weak_motors_20 | 1.00 | | gravity_high_10 | 0.975 | All are above the 0.9 gate, with mean distance ~6.8 m per episode (the requirement is 3 m) and typical returns around 1850. **How the controller works.** It's a Raibert-style phase machine keyed on foot contact and vertical velocity, with ~12 tuned gains: - **CROUCH/ABSORB** — knee PD to a slightly bent posture (−0.29 rad); a panic reflex extends the knee fully if the torso sinks below 0.85 m. - **THRUST** — fires at the bottom of compression: near-full knee extension (reduced when above the target speed, ~1.57 m/s) plus a toe-up ankle bias that injects nose-down angular momentum so the flight phase starts recoverable. - **FLIGHT** — knee refolds; the hip runs a pitch servo whose reaction torque doubles as foot placement (correcting nose-down pitch swings the leg forward, which brakes on the next landing) with a small thigh-posture term. - **Stance balance** — the hip servos torso pitch toward a small lean (backed off as speed grows), and the ankle shifts the centre of pressure along the long foot based on torso-over-foot position, forward speed, and pitch rate. This ankle law alone keeps the robot standing indefinitely. **Key findings along the way:** t…
A four-phase hop cycle state machine drives the knee for crouch, thrust, flight refold, and absorb while the hip servos pitch (with speed-dependent lean and flight foot placement) and the ankle shifts the center of pressure, with gains tuned via PARAMS.
Shown: public seed 17 (standard_v0); this episode fails the forward rung. This episode: episode_steps 635, x_final 3.04, min_height 0.688, max_abs_pitch 0.154, n_flights 14, total_reward 1013. It was the first failing one of 18 public seeds tried.
controller.py · trial hopper-robust__kLeQ53R
low_friction_30| criterion | graded | needed |
|---|---|---|
| base:forward | 0.975 | ≥ 0.900 |
| scenario:heavy_torso_20 | 0.975 | ≥ 0.900 |
| scenario:low_friction_30 | 0.225 | ≥ 0.900 |
| scenario:weak_motors_20 | 0.925 | ≥ 0.900 |
| scenario:gravity_high_10 | 0.825 | ≥ 0.900 |
The agent's final message
That notification is from the earlier wait on the es5 search, which I had already stopped and superseded. Nothing changes: the installed controller remains the es5 best, and the summary above stands.
A hand-designed stance/flight state machine uses PD loops for torso, knee and ankle, with a speed-modulated heel-roll then toe-push ankle sequence for thrust and braking, and its gains were tuned by a (1+lambda) evolution strategy across the robustness scenarios.
Shown: public seed 0 (low_friction_30); this episode fails the forward rung. This episode: episode_steps 631, x_final 4.33, min_height 1.09, max_abs_pitch 0.207, n_flights 10, total_reward 1171. It was the first failing one of 1 public seeds tried.
controller.py · agent transcript · trial hopper-robust__yDaHHBA