humanoid-parity-notune
Humanoid (MuJoCo Humanoid-v5) · solved 3 of 9 agent attempts pass
more
The fast rung (15 m in 15 s, alive, on 90 % of seeds) plus a mean return within 10 % of a committed RL reference measured on the grader's own seeds: SB3 SAC trained here on Gymnasium Humanoid-v5 for 2 M steps with the RL-zoo hyperparameters (rl_references/humanoid/train_sb3_sac.py; the plant reproduces Gymnasium step for step, verified seed by seed); the reference is the 1.8 M-step checkpoint, the best of the last four on 20 non-task seeds (the 2 M final falls on 1 of 20): return 5579 +- 666 (min 1425), alive 0.97, 11.8 m, 0.80 m/s, fast 0.00; inside Gymnasium 5667 +- 28 -> bound 5020. The SAC policy itself walks at about 0.8 m/s and does not clear the fast rung. The oracle is the classical fast_v1 design (return 6083), so the rung is reachable without learning; the agents' question is whether a session finds it. Provenance under tests/reference/.
Gate. per-seed success is the `fast` rung of this benchmark's ladder ['alive', 'forward', 'brisk', 'fast'] (`--success-mode fast`), not the evaluator's default flag; the task is solved when the success rate over the grader's held-out seeds is `>= 0.9`; the aggregate `total_reward_mean` over the grader's held-out seeds (base scenario) must be `>= 5020` (rung `metric:total_reward_mean`)
Axes: plant:humanoid, scenario:standard_v0, gate:fast_15m, rl-parity:sac-v5-1.8M-minus-10pct, ablation:no-tuner
Each video re-runs the attempt's final controller through the task's own evaluator on a public seed (never the grader's held-out ones). The table under each video is the official grade on the held-out seeds.
standard_v0| criterion | graded | needed |
|---|---|---|
| base:fast | 1.000 | ≥ 0.900 |
| metric:total_reward_mean | 6351 | ≥ 5020 |
The agent's final message
The controller in `/app/work/controller.py` passes the gate. With the official evaluator, seeds 0–99 on `standard_v0` with `--success-mode fast` gave a 100/100 success rate and a mean return of 6352 (the gate is 5020). The robot walked about 22.6 m on average, where 15 m is needed. Seeds 0–4 gave a return of 6370. Total time used was about 25 minutes (16:46 to 17:11). **How it works:** it's a hand-written walking controller in the style of SIMBICON, a classic rule-based method for biped walking. - **Step timing:** the stance and swing legs swap every 0.073 s, or earlier if the swing foot touches down. The result is a fast shuffle of about 13 steps per second at roughly 1.45 m/s. - **Foot placement:** the swing hip places the foot ahead or to the side by an amount set by how far and how fast the mass centre is moving relative to the stance foot. This is what catches the body forwards and sideways. - **Knees:** the swing knee folds early in the step to clear the floor, then straightens. - **Torso:** pitch, roll and yaw are held by a feedback loop applied through the stance hip. - **Abdomen and arms:** held at the zero posture. The docstring explains each gain. **How I got the numbers:** the structure is mine, but the gain values came from a black-box random-perturbation search run on simulated episodes. That is derivative-free tuning, with no gradients and no neural network, and the controller itself doesn't learn. Before that, I tried sampling-based predictive control using…
A hand-structured SIMBICON walker alternates stance and swing legs on a timer or early on swing-foot contact, placing the swing foot by COM position and velocity, balancing the torso via the stance hip with PD, and holding the knees, abdomen and arms with PD servos, with gains tuned by black-box random search.
Shown: public seed 0 (standard_v0); this episode passes the fast rung. This episode: episode_steps 1000, x_final 23, min_height 1.25, max_abs_pitch 0.2, n_steps 199, total_reward 6387.
controller.py · agent transcript · trial humanoid-parity-notune__kZcgNqj
standard_v0| criterion | graded | needed |
|---|---|---|
| base:fast | 0.000 | ≥ 0.900 |
| metric:total_reward_mean | 5991 | ≥ 5020 |
The agent's final message
That notification is only the background `sleep` finishing, so nothing has changed. The final state is the one I reported. `/app/work/controller.py` is alive on all 10 visible seeds. It reaches about 13.5 m, which is the `brisk` rung but not the 15 m `fast` rung. The success rate is 0 against the 0.9 required, so it is below the gate. I'm not continuing. About 54 of the 60 minutes were used when I stopped, and the last search run didn't find a faster stable version.
A SIMBICON-style walker alternates right and left stance states on foot contact or timeout, using PD servos toward target joint angles with COM-based swing foot placement and torso pitch, roll and yaw feedback, with parameters found by derivative-free search.
Shown: public seed 0 (standard_v0); this episode fails the fast rung. This episode: episode_steps 1000, x_final 13.6, min_height 1.27, max_abs_pitch 0.199, n_steps 63, total_reward 6004. It was the first failing one of 1 public seeds tried.
controller.py · agent transcript · trial humanoid-parity-notune__PVZRFUB
standard_v0The agent's final message
The submission does not pass the gate. With 50 of the 60 minutes used and each full evaluation taking about 4 minutes, I stopped here. `/app/work/controller.py` holds the best version I measured. **Final official evaluation (seeds 0–4, standard_v0, `--success-mode fast`):** | Seed | Steps | Distance | Return | |---|---|---|---| | 0 | 1000 | 12.0 m | 5699 | | 1 | 1000 | 12.9 m | 5754 | | 2 | 317 (fell) | 3.7 m | 1787 | | 3 | 1000 | 13.5 m | 5819 | | 4 | 1000 | 13.1 m | 5762 | - **Ladder:** alive, forward and brisk at 0.8; fast at 0.0 (no seed reached 15 m). - **Mean return:** 4964, just under the 5020 gate. - **Speed:** the seeds that survive average about 0.85 m/s, so walking speed needs to rise by about 15%. One seed in five still falls. **What the controller does:** it is a sampling-based model-predictive controller, with no learning. - Each step it rebuilds the full state from `gym_obs` into its own copy of the MuJoCo model. For speed, that copy plans with simpler physics (Euler at 5 ms instead of the real RK4 at 3 ms). - It tries 48 noisy versions of the current plan over the next 0.75 s (50 control steps), running them in parallel with `mujoco.rollout`. - It keeps the cheapest one and applies its first command. - The cost penalises low torso height, leaning, forward speed off a 2 m/s target, sideways drift, turning away from +x, the capture point drifting away from the feet, control effort, and predicted falls. - The docstring explains each weight. **What I learned …
Each step it rebuilds the simulator state from observations, rolls out 48 noisy piecewise-linear control sequences over a 50-step horizon in a copy of the MuJoCo model, scores them on torso height, uprightness, forward speed, and capture-point cost, applies the first command of the cheapest sequence, and warm-starts the next plan from it.
Shown: public seed 0 (standard_v0); this episode fails the fast rung. This episode: episode_steps 1000, x_final 12.2, min_height 1.07, max_abs_pitch 1.14, n_steps 100, total_reward 5712. It was the first failing one of 1 public seeds tried.
controller.py · agent transcript · trial humanoid-parity-notune__WA9EyMY