humanoid-robust-parity
Humanoid (MuJoCo Humanoid-v5) · solved 2 of 9 agent attempts pass
more
Open by construction when queued; solved in session on 2026-10-25 under the Claude Code protocol (one hour, zero-torque starter) by Sonnet 5.5 (this oracle) and Fable 5.1 (0.66 m/s, return 5624, all siblings 1.00), 2 of 9 attempts, with three Opus 5.5 near misses. RL trained for robustness does not clear it: SB3 SAC with the physics knobs and pushes re-sampled every episode (rl_references/humanoid/train_sb3_sac_dr.py, 2 M steps, the 1.9 M checkpoint) reaches brisk rates 0.93 / 0.93 / 0.62 / 0.82 / 0.57 on the siblings at return 5222-5424 (tests/reference/, sb3-sac-sac_dr_physicspush_*). Noise + delay is left out on purpose: nominal SAC falls there (0.05 alive) and only SAC trained with noise and delay survives it (0.875-1.00 alive at 0.3-0.4 m/s, sb3-sac-sac_dr_all_*). Provenance under tests/reference/.
Gate. per-seed success is the `brisk` rung of this benchmark's ladder ['alive', 'forward', 'brisk', 'fast'] (`--success-mode brisk`), not the evaluator's default flag; the task is solved when the success rate over the grader's held-out seeds is `>= 0.9`; the same gate must also hold on the sibling scenario(s) `heavy_torso_20`, `low_friction_30`, `weak_motors_15`, `gravity_high_10`, `push_75` (`--scenarios`); the aggregate `total_reward_mean` over the grader's held-out seeds (base scenario) must be `>= 5020` (rung `metric:total_reward_mean`)
Axes: plant:humanoid, scenario:standard_v0+physics-at-rl-envelope+push_75, gate:brisk_8m, rl-parity:sac-v5-1.8M-minus-10pct, robustness:rl-envelope
Each video re-runs the attempt's final controller through the task's own evaluator on a public seed (never the grader's held-out ones). The table under each video is the official grade on the held-out seeds.
standard_v0| criterion | graded | needed |
|---|---|---|
| base:brisk | 1.000 | ≥ 0.900 |
| scenario:heavy_torso_20 | 1.000 | ≥ 0.900 |
| scenario:low_friction_30 | 1.000 | ≥ 0.900 |
| scenario:weak_motors_15 | 1.000 | ≥ 0.900 |
| scenario:gravity_high_10 | 1.000 | ≥ 0.900 |
| scenario:push_75 | 1.000 | ≥ 0.900 |
| metric:total_reward_mean | 5624 | ≥ 5020 |
The agent's final message
Done. The final controller is at `/app/work/controller.py` and passes the official verifier on the visible seeds. 44 minutes of the 60-minute budget were used. **Result (official `hl_control.harbor.verify`, seeds 0-4, brisk mode)** | Scenario | Success rate | |---|---| | standard_v0 | 1.00 (return 5629, gate 5020) | | heavy_torso_20 | 1.00 | | low_friction_30 | 1.00 | | weak_motors_15 | 1.00 | | gravity_high_10 | 1.00 | | push_75 | 1.00 | On fresh seeds, the controller is 20/20 brisk on every scenario for seeds 40-59, and 59/60 on push seeds 60-119 (the previous parameter set was 53/60 there, which is why I swapped). Mean progress is about 10 m per episode, against the 8 m brisk threshold. **What the controller is.** A SIMBICON-style stepping controller: joint PD servos on gait-phase targets, with a two-state stance machine, world-frame swing-thigh placement from mass-centre velocity, lateral placement from lateral velocity and roll, torso pitch and roll servos at the stance hip, and a near-straight stance knee. The gait that works is a fast jog, roughly 100 touchdowns per episode. The docstring explains the law and every gain is commented. **What mattered most while building it** - The feet are point contacts, so no yaw authority and no static standing. A stance-hip yaw servo just drove hip_z to its limits. - The knees are weak (80 N m). A stiff knee servo cancels hip flexion through its reaction torque, and a bent knee at touchdown catapults the body. Separate soft swi…
A phase-clocked right/left stance state machine places the swing foot using COM velocity and offset from the stance foot (SIMBICON), while the stance hips servo torso pitch and roll and all joints are PD-driven and clipped.
Shown: public seed 0 (standard_v0); this episode passes the brisk rung. This episode: episode_steps 1000, x_final 10.1, min_height 1.26, max_abs_pitch 0.243, n_steps 99, total_reward 5640.
controller.py · agent transcript · trial humanoid-robust-parity__UmDYVEW
push_75| criterion | graded | needed |
|---|---|---|
| base:brisk | 1.000 | ≥ 0.900 |
| scenario:heavy_torso_20 | 1.000 | ≥ 0.900 |
| scenario:low_friction_30 | 1.000 | ≥ 0.900 |
| scenario:weak_motors_15 | 1.000 | ≥ 0.900 |
| scenario:gravity_high_10 | 1.000 | ≥ 0.900 |
| scenario:push_75 | 1.000 | ≥ 0.900 |
| metric:total_reward_mean | 5624 | ≥ 5020 |
The agent's final message
Done. The final controller is at `/app/work/controller.py` and passes the official verifier on the visible seeds. 44 minutes of the 60-minute budget were used. **Result (official `hl_control.harbor.verify`, seeds 0-4, brisk mode)** | Scenario | Success rate | |---|---| | standard_v0 | 1.00 (return 5629, gate 5020) | | heavy_torso_20 | 1.00 | | low_friction_30 | 1.00 | | weak_motors_15 | 1.00 | | gravity_high_10 | 1.00 | | push_75 | 1.00 | On fresh seeds, the controller is 20/20 brisk on every scenario for seeds 40-59, and 59/60 on push seeds 60-119 (the previous parameter set was 53/60 there, which is why I swapped). Mean progress is about 10 m per episode, against the 8 m brisk threshold. **What the controller is.** A SIMBICON-style stepping controller: joint PD servos on gait-phase targets, with a two-state stance machine, world-frame swing-thigh placement from mass-centre velocity, lateral placement from lateral velocity and roll, torso pitch and roll servos at the stance hip, and a near-straight stance knee. The gait that works is a fast jog, roughly 100 touchdowns per episode. The docstring explains the law and every gain is commented. **What mattered most while building it** - The feet are point contacts, so no yaw authority and no static standing. A stance-hip yaw servo just drove hip_z to its limits. - The knees are weak (80 N m). A stiff knee servo cancels hip flexion through its reaction torque, and a bent knee at touchdown catapults the body. Separate soft swi…
A phase-clocked right/left stance state machine places the swing foot using COM velocity and offset from the stance foot (SIMBICON), while the stance hips servo torso pitch and roll and all joints are PD-driven and clipped.
Shown: public seed 0 (push_75); this episode passes the brisk rung. This episode: episode_steps 1000, x_final 10.5, min_height 1.22, max_abs_pitch 0.428, n_steps 100, total_reward 5673.
controller.py · agent transcript · trial humanoid-robust-parity__UmDYVEW
push_75| criterion | graded | needed |
|---|---|---|
| base:brisk | 0.975 | ≥ 0.900 |
| scenario:heavy_torso_20 | 1.000 | ≥ 0.900 |
| scenario:low_friction_30 | 0.975 | ≥ 0.900 |
| scenario:weak_motors_15 | 0.925 | ≥ 0.900 |
| scenario:gravity_high_10 | 0.925 | ≥ 0.900 |
| scenario:push_75 | 0.775 | ≥ 0.900 |
| metric:total_reward_mean | 5534 | ≥ 5020 |
The agent's final message
`/app/work/controller.py` passes the full gate on the visible seeds, but `push_75` is probably below 0.9 on the grader's held-out seeds. **Official verifier, seeds 0–4:** `solved=True`. Brisk success is 5/5 on the nominal plant and on each of the five sibling scenarios. `total_reward_mean` is 5661 against the 5020 gate, and the mean distance is about 10.5 m (brisk needs 8 m). **My own wider check, seeds 0–39:** | Scenario | Brisk successes | |---|---| | `push_75` | 33/40 (0.83) | | `standard_v0` | 38/40 (0.95) | | `weak_motors_15` | 40/40, but some runs only reach about 8.15 m | Seeds 0–9 passed on every other scenario. So `push_75` probably falls short of the 0.9 gate on held-out seeds, and `weak_motors_15` has little distance to spare. **Controller:** a walking controller modelled on SIMBICON, an established hand-designed biped gait. The docstring explains each gain. - **Stepping:** the legs swap on a fixed clock of about 4.8 steps per second. The swing foot's landing point is set from the body's forward and sideways velocity, then turned into hip and knee angles. - **Balance:** the standing leg's knee is held locked straight, and its hip keeps the torso upright, correcting for the swinging leg's pull. - **Turning:** the feet are spheres, so they can't twist against the floor. Yaw is controlled by lengthening one side's step and shortening the other's. The arms swing against the legs, and the waist joint helps hold the heading. **How I got there:** tuning gains by han…
A clock-driven (effectively fixed cadence) two-state walker swaps stance and swing legs, placing the swing foot by a Raibert/capture-point law via IK and PD while the stance hip applies PD torso pitch/roll/yaw torques with swing-hip reaction compensation, with CMA-ES-tuned gains.
Shown: public seed 10 (push_75); this episode fails the brisk rung. This episode: episode_steps 239, x_final 2.17, min_height 0.99, max_abs_pitch 1.15, n_steps 19, total_reward 1283. It was the first failing one of 11 public seeds tried.
controller.py · agent transcript · trial humanoid-robust-parity__WmKhPco
low_friction_30| criterion | graded | needed |
|---|---|---|
| base:brisk | 1.000 | ≥ 0.900 |
| scenario:heavy_torso_20 | 1.000 | ≥ 0.900 |
| scenario:low_friction_30 | 0.875 | ≥ 0.900 |
| scenario:weak_motors_15 | 1.000 | ≥ 0.900 |
| scenario:gravity_high_10 | 0.950 | ≥ 0.900 |
| scenario:push_75 | 0.000 | ≥ 0.900 |
| metric:total_reward_mean | 5506 | ≥ 5020 |
The agent's final message
The controller doesn't pass the gate. It walks on the nominal plant and four of the five perturbed ones, but it falls in every `push_75` episode, so the official result is `solved=False`. `/app/work/controller.py` is the best version I found. This is the official evaluator run on visible seeds 0–4 with the exact grading flags: | Scenario | brisk success | notes | |---|---|---| | standard_v0 | **5/5** | `total_reward_mean` 5512 (gate ≥ 5020: pass), about 8.4–8.5 m in 15 s | | heavy_torso_20 | **5/5** | | | weak_motors_15 | **5/5** | | | gravity_high_10 | **5/5** | | | low_friction_30 | 4/5 | one seed falls at step 849 | | push_75 | **0/5** | survives 200–700 steps, then falls after a push | **The controller** (described in the file's docstring): - **Steps:** the legs take turns standing and swinging, and they swap when the swinging foot touches down. - **Upright posture:** the hip of the standing leg holds the torso's forward lean, sideways tilt and turning. - **Foot placement:** each step lands where the current velocity says it should to catch the body. Small left/right differences in step length correct the heading, because the round feet can't grip the floor to resist turning. - **Knees:** the swinging knee folds and then straightens before landing; the standing knee stays nearly straight. The gains came from a derivative-free cross-entropy search across the scenarios, run with my own scripts. There was no gradient-based learning. **Margins and risks:** - **Thin speed…
A SIMBICON-style gait state machine swaps stance and swing legs on foot contact, placing the swing foot by center-of-mass velocity feedback and holding the torso upright through the stance hip, with all joints driven by PD torques and clipped.
Shown: public seed 4 (low_friction_30); this episode fails the brisk rung. This episode: episode_steps 849, x_final 7.98, min_height 0.999, max_abs_pitch 0.82, n_steps 155, total_reward 4700. It was the first failing one of 5 public seeds tried.
controller.py · agent transcript · trial humanoid-robust-parity__ZpnRVxT
weak_motors_15| criterion | graded | needed |
|---|---|---|
| base:brisk | 1.000 | ≥ 0.900 |
| scenario:heavy_torso_20 | 0.950 | ≥ 0.900 |
| scenario:low_friction_30 | 1.000 | ≥ 0.900 |
| scenario:weak_motors_15 | 0.800 | ≥ 0.900 |
| scenario:gravity_high_10 | 0.725 | ≥ 0.900 |
| scenario:push_75 | 0.000 | ≥ 0.900 |
| metric:total_reward_mean | 5608 | ≥ 5020 |
The agent's final message
The task is **not solved**. The controller walks briskly on the nominal plant and on most physics changes, but the `push_75` scenario fails on every seed. Because the gate is all-or-nothing, this submission would score zero. I stopped at 43 of 60 minutes: the last 15 minutes of search made no progress on pushes, and the verified version is in place. **Official verifier, visible seeds 0–4, brisk rung:** | Scenario | Brisk success | Gate (≥ 0.9) | |---|---|---| | `standard_v0` | 5/5, mean return 5608 (needs ≥ 5020) | pass | | `low_friction_30` | 5/5 | pass | | `gravity_high_10` | 5/5 | pass | | `heavy_torso_20` | 4/5 | borderline | | `weak_motors_15` | 4/5 | borderline | | `push_75` | **0/5** | **fail** | **What `/app/work/controller.py` does** (the docstring explains each gain): - **Gait:** a two-state stepping cycle with short steps of about 0.2 s. The feet are spheres with no ankle, so the robot can only balance by stepping. - **Stance leg:** its hip holds the torso upright and on heading, minus the torque of the swinging hip. - **Swing foot:** aimed by predicting where the mass centre will be at touchdown, then reached with closed-form leg IK on a kinematic copy of the model. The swing `hip_z` keeps the leg pointing along world x. - **Lateral support:** the abdomen leans the torso over the stance hip, because the 40 N·m `hip_x` motors can't hold the pelvis level alone. - **Motor scaling:** torques are converted to commands with the observed `obs["gear"]`, which compensat…
A clocked two-state walker where the stance leg's hip torques regulate torso pitch, roll and heading, while the swing foot is steered by LIP-predicted capture-point targets via closed-form IK and PD, with a push-triggered recovery parameter set.
Shown: public seed 4 (weak_motors_15); this episode fails the brisk rung. This episode: episode_steps 193, x_final 1.89, min_height 0.984, max_abs_pitch 0.855, n_steps 17, total_reward 1075. It was the first failing one of 5 public seeds tried.
controller.py · agent transcript · trial humanoid-robust-parity__2yNVviR