humanoid-fast
Humanoid (MuJoCo Humanoid-v5) · solved 2 of 10 agent attempts pass
more
Reachable with the classical structure: the oracle is the humanoid-forward design with gait and gains re-tuned by the repository tuner (three rounds, 1800 evaluations), fast on 40/40 seeds at return 6083. The agents' question is whether that design is found from scratch in one session; the handoff twin starts from the brisk design.
Gate. per-seed success is the `fast` rung of this benchmark's ladder ['alive', 'forward', 'brisk', 'fast'] (`--success-mode fast`), not the evaluator's default flag; the task is solved when the success rate over the grader's held-out seeds is `>= 0.9`
Axes: plant:humanoid, scenario:standard_v0, gate:fast_15m, rl-reference:sac-v5-local
Each video re-runs the attempt's final controller through the task's own evaluator on a public seed (never the grader's held-out ones). The table under each video is the official grade on the held-out seeds.
standard_v0| criterion | graded | needed |
|---|---|---|
| base:fast | 0.950 | ≥ 0.900 |
The agent's final message
The task is solved on the nominal plant. The official evaluator passed **50/50 seeds** on the `fast` rung (seeds 0–49, `standard_v0`, `--success-mode fast`). Each run walked about 16.7 m in 15 s, roughly 1.1 m/s, against the 15 m needed. My own harness, which gives the same results as the evaluator, scored 189/190 on seeds 0–189. I used about 29 of the 60 minutes. **How the controller works** (`/app/work/controller.py`; the docstring explains each gain): - **Gait timing:** a two-state machine alternates the legs. The swing ends when the swing foot touches down, or after a fixed time at the latest. - **Torso balance:** the stance hip holds the torso's pitch and roll with a PD law, after subtracting the torque the swing hip already puts on the pelvis. The pitch target is a slight forward lean, which drives the walking speed. - **Foot placement:** the swing foot is aimed at a point ahead of and beside the mass centre, set from its velocity relative to the target speed. The foot is lifted, then pushed below floor level late in the swing to force touchdown. Hip and knee angles come from inverse kinematics of the leg. - **Heading:** the abdomen's yaw joint gets extra feedback on body yaw. This change took the success rate from about 90% to 99.5%. Oddly, it only helps with a negative gain (−50 with rate gain −3.5), the opposite sign to the one I expected. **What made it work:** - **Feet:** they are spheres with no twisting friction, so the stance hip can't steer the heading. - **G…
A two-state gait machine alternates stance and swing legs, where the stance hip produces torso pitch/roll balance torque while the swing foot is steered by a velocity-dependent Raibert-style target through IK and PD tracking, with gains found by random search.
Shown: public seed 0 (standard_v0); this episode passes the fast rung. This episode: episode_steps 1000, x_final 17, min_height 1.24, max_abs_pitch 0.385, n_steps 47, total_reward 6265.
controller.py · agent transcript · trial humanoid-fast__HfttBZv
standard_v0| criterion | graded | needed |
|---|---|---|
| base:fast | 0.150 | ≥ 0.900 |
The agent's final message
The optimiser is no longer running, and `/app/work/controller.py` loads fine with the `best_e` parameters (gait frequency 3.12 Hz). The notification was only the earlier background wait finishing, so nothing changed. The controller still doesn't pass the gate. It reached the `fast` rung on 1 of 5 visible seeds and on 6 of 24 fresh seeds, against the 90% needed. About 54 of the 60 minutes are used, which isn't enough for another tuning round.
Each leg tracks a sinusoidal joint-angle reference with the legs half a period apart, driven by joint PD and clipped to ±0.4, while torso pitch/roll, COM velocity and yaw feedback adjust hip and abdomen setpoints to keep the humanoid upright and moving at about 1.16 m/s.
Shown: public seed 0 (standard_v0); this episode fails the fast rung. This episode: episode_steps 237, x_final 5.18, min_height 0.983, max_abs_pitch 0.424, n_steps 24, total_reward 1552. It was the first failing one of 1 public seeds tried.
controller.py · agent transcript · trial humanoid-fast__egzxBCJ
standard_v0The agent's final message
The humanoid now walks at about 1.1 m/s and passes the `fast` rung on 9 of the 10 seeds I tested (0–9). That is right at the 0.9 gate, so the held-out result could fall either side of it. The final controller passed seed 0 under the official evaluator; the other nine seeds were run with a scratch runner on the same environment. **How it works:** it is model-predictive control, with no learning involved. - Each step, the controller copies the true simulator state from `obs["gym_obs"]` into its own copy of the same MuJoCo model, planning with a 5 ms physics step instead of 3 ms. - It tries 64 candidate control sequences 0.55 s ahead and applies the first step of the cheapest. - The cost rewards forward speed (target 1.4 m/s) and penalises sideways drift, dropping height, leaning, turning away from +x, the capture point drifting away from the feet, and falling. - The key change was sampling joint-angle targets for a PD law rather than raw torques. With torques it fell within 2–7 s, even when only asked to stand still. - The damping gain has to stay small (kv = 0.02). The plant holds each command for 15 ms, and larger damping makes the real robot drift away from what the planner predicted. **Results:** | Settings | Seeds 0–4 | Seeds 5–9 | Failure | |---|---|---|---| | 0.45 s horizon, target 1.3 m/s | 5/5 | 4/5 | seed 7 fell at 11.1 m | | 0.55 s horizon, target 1.4 m/s (submitted) | 4/5 | 5/5 | seed 4 fell at 9.2 m | I kept the second because every passing seed clears 15 m wit…
Each step it copies the true state into an internal MuJoCo model, rolls out 64 perturbed spline joint-target sequences with PD servos, picks the lowest-cost one as the new warm-started nominal, and applies its first target through a clipped PD law.
Shown: public seed 2 (standard_v0); this episode fails the fast rung. This episode: episode_steps 993, x_final 16.1, min_height 0.98, max_abs_pitch 0.773, n_steps 124, total_reward 5911. It was the first failing one of 3 public seeds tried.
controller.py · agent transcript · trial humanoid-fast__LsskYwN