walker2d-parity-notune
Walker2d (MuJoCo Walker2d-v5) · solved 2 of 9 agent attempts pass
more
The fast rung (20 m in 8 s, alive, on 90 % of seeds) plus a mean return within 10 % of SB3-zoo TD3 measured on the grader's seeds (4701 -> 4231; rl_references/hopper/run_sb3_reference.py --plant walker2d). Solved 2026-10-26 under the Claude Code protocol (Opus 2/3, Fable 1/3, Sonnet 0/3); the solution is Opus 5.5's clocked linear feedback law with gains found by in-session CMA-ES (return 5265, 112 % of TD3). It is fragile: on 16 public seeds it fails all four walker2d-robust perturbations (torso +20 %, friction -30 %, motors -20 %, gravity +10 %: forward rate 0.00 each). The textbook alternative, Opus 5.5's SIMBICON walker re-tuned in three tuner rounds, reaches 18.1 m at 2.26 m/s and return 3254 (69 % of TD3) and survives all four perturbations; no SIMBICON-family design has cleared this gate. In this ablation's own runs (no tuner) Opus and Sonnet passed with clocked CPG walkers (returns 4679 and 4371) that keep partial robustness (torso +20 % 1.00, gravity +10 % about 0.9, motors -20 % 0.00 on public seeds).
Gate. per-seed success is the `fast` rung of this benchmark's ladder ['alive', 'forward', 'brisk', 'fast'] (`--success-mode fast`), not the evaluator's default flag; the task is solved when the success rate over the grader's held-out seeds is `>= 0.9`; the aggregate `total_reward_mean` over the grader's held-out seeds (base scenario) must be `>= 4231` (rung `metric:total_reward_mean`)
Axes: plant:walker2d, scenario:standard_v0, gate:fast_20m, rl-parity:td3-4701-minus-10pct, ablation:no-tuner
Each video re-runs the attempt's final controller through the task's own evaluator on a public seed (never the grader's held-out ones). The table under each video is the official grade on the held-out seeds.
standard_v0| criterion | graded | needed |
|---|---|---|
| base:fast | 1.000 | ≥ 0.900 |
| metric:total_reward_mean | 4371 | ≥ 4231 |
The agent's final message
`/app/work/controller.py` now passes the gate on seeds it was never tuned on. On seeds 200–229 (standard plant), all 30 reach `fast`, with a mean return of 4373 (gate 4231) and a mean final x of 27.0 m. I also ran seeds 100–139 on the saved controller's parameters. All 40 survived the full 1000 steps, the shortest run reached 26.3 m, and the mean return was 4377. I only checked the standard scenario, not the five perturbation scenarios. The controller is a fixed-rhythm walking pattern with feedback on top. A clock drives the thigh, knee and ankle targets of each leg, and the left leg runs half a cycle behind the right. Each motor gets a PD command toward its target, clipped to [-1, 1]. Both thigh targets are also shifted by torso pitch, pitch rate and forward speed. The amplitudes ramp up over the first second so the robot starts from standing. The docstring in the file explains each part. I tuned the 22 parameters with CMA-ES, a gradient-free search, running the simulator on seeds 10–15. No neural network or gradient learning was involved. The search target was half the mean return plus half the worst seed's return, which is what removed the falls. An earlier version scored higher on the seeds it was tuned on but fell on 11 of 40 fresh seeds (mean return 3781). The margin over the return gate is about 140 on average (4373 and 4377 against 4231), so the grader's seeds could land close to it. The session used about 50 of the 60 minutes.
A fixed-frequency phase oscillator generates sinusoidal thigh, knee and ankle targets (left leg offset by half a cycle) tracked by a clipped PD servo, while torso pitch and forward-speed error modify the hip targets to stabilize the gait.
Shown: public seed 0 (standard_v0); this episode passes the fast rung. This episode: episode_steps 1000, x_final 27.5, min_height 0.937, max_abs_pitch 0.634, n_steps 33, total_reward 4440.
controller.py · agent transcript · trial walker2d-parity-notune__MaAeTpB
standard_v0| criterion | graded | needed |
|---|---|---|
| base:fast | 1.000 | ≥ 0.900 |
| metric:total_reward_mean | 3576 | ≥ 4231 |
The agent's final message
The controller does **not** pass the gate. It clears the `fast` rung on every visible seed, but its mean return is about 3576, short of the required 4231. Since a submission below the gate scores zero, this one will likely score zero as it stands. I stopped at 51 of the 60 minutes because the search had flattened out and no run was going to close the gap in the time left. **Results on visible seeds 0–29:** | | Result | Required | |---|---|---| | Alive and ≥ 20 m (`fast`) | 30 / 30 seeds | ≥ 90 % | | Mean distance | 20.6 m | — | | Mean return | about 3578 | ≥ 4231 (about 26 m) | The official command on seeds 0–4 gives the same: `fast` = 1.00, return 3576, about 43 % flight time. So the gait is a reliable run at roughly SAC speed (about 2.6 m/s). Reaching TD3 return would need about 3.3 m/s. **How the controller works** (`/app/work/controller.py`, self-contained, with a docstring per term): 1. **Clock:** a fixed-rate phase at about 1.8 Hz. The left leg runs half a cycle behind the right, and the gait fades in over the first fraction of a second. 2. **Joint targets:** each leg's hip, knee and ankle angles follow a simple periodic curve set by a handful of coefficients. The left leg gets a small posture offset, so the gait can be asymmetric. 3. **PD tracking:** each joint is driven toward its target with separate stiffness and damping gains for hip, knee and ankle. 4. **Torso balance:** the hips correct torso lean and its rate, leaning more on the leg that's on the ground. Th…
A fixed-frequency clock drives half-cycle-offset two-harmonic hip/knee/ankle targets tracked by PD, with added pitch and velocity feedback on hips, contact-weighted torso balance and a linear state-feedback correction, all gains tuned by CMA-ES.
Shown: public seed 0 (standard_v0); this episode passes the fast rung. This episode: episode_steps 1000, x_final 20.7, min_height 0.955, max_abs_pitch 0.682, n_steps 55, total_reward 3582. It was the first failing one of 1 public seeds tried.
controller.py · agent transcript · trial walker2d-parity-notune__9YXcNLQ
standard_v0| criterion | graded | needed |
|---|---|---|
| base:fast | 0.000 | ≥ 0.900 |
| metric:total_reward_mean | 3317 | ≥ 4231 |
The agent's final message
API Error: Can't reach the API server — check your internet or DNS (EAI_AGAIN)
A global phase clock generates cosine joint targets for each leg (left offset by half a cycle) tracked by PD control, with the clock nudged toward a nominal phase at foot touchdown, stance-hip torso-pitch feedback, velocity-dependent swing-thigh shift, and constant stance feedforward torques, with parameters tuned by CMA-ES.
Shown: public seed 0 (standard_v0); this episode fails the fast rung. This episode: episode_steps 1000, x_final 18.7, min_height 1.02, max_abs_pitch 0.686, n_steps 37, total_reward 3336. It was the first failing one of 1 public seeds tried.
controller.py · agent transcript · trial walker2d-parity-notune__JeBJDdd