hopper-bangbang
Hopper (MuJoCo Hopper-v5) · solvedexploration 8 of 9 agent attempts pass
Exploration task. Built after the paper's sweep to test whether a task can force a mechanism outside the catalogue (docs/exploration_new_mechanisms.md, docs/soft_arm_spec.md). Not part of the benchmark tiers or of the paper's main counts; the soft arm is reported in the paper as a plant outside the sweep, the other tasks are not; all sessions ran under the paper's protocol.
more
Each motor command is quantised to -1, 0 or +1 before it is applied, so proportional-derivative servoing has no small corrections and every joint is driven at full gear or not at all; a design must modulate (pulse-width, sliding-mode, dithering) or build the gait around the three levels. Physics and sensing nominal; gate forward (3 m) on 0.9 of the seeds. Open: the hopper-forward oracle scores 0 with bang-bang motors.
Gate. per-seed success is the `forward` rung of this benchmark's ladder ['alive', 'forward', 'brisk', 'fast'] (`--success-mode forward`), not the evaluator's default flag; the task is solved when the success rate over the grader's held-out seeds is `>= 0.9`
Axes: plant:hopper, scenario:bangbang_v1, gate:forward_3m, exploration:new-mechanism, actuation:bang-bang
Each video re-runs the attempt's final controller through the task's own evaluator on a public seed (never the grader's held-out ones). The table under each video is the official grade on the held-out seeds.
bangbang_v1| criterion | graded | needed |
|---|---|---|
| base:forward | 1.000 | ≥ 0.900 |
The agent's final message
`/app/work/controller.py` hops forward on the bang-bang plant and clears the gate on every seed I could check. Visible seeds 0–4 pass 5/5 and seeds 0–9 pass 10/10, with a mean final x of about 8.9 m. On 80 other seeds (100–179) it passes 78/80, so about 97.5%, against the 0.9 needed. Total time used was about 22 minutes of the 60. **How it works.** Each motor command is turned into −1/0/+1 by error-feedback (sigma-delta) dithering, so the average torque matches a smooth command. At 125 Hz with the heavy joint armature, the motors then act roughly like continuous ones. The control law is split by whether the foot is on the ground. | Phase | Knee | Ankle | Hip | |---|---|---|---| | Stance | virtual spring plus a thrust while it extends | balances leg lean, damped by body speed, with a toe push on extension | torso-pitch PD, plus a pull toward a thigh angle that keeps it off 0 rad | | Flight | tucks to a set angle | returns to neutral | moves to a set thigh angle with a pitch PD | The docstring in the file explains the gains. **How the gains were found.** I wrote the stance/flight structure by hand and found the gains with a small evolution-strategy search of my own. I wrote my own search because the supplied `tune` tool scores a standing robot at 0.4, so it settled on standing still. My search scores forward distance directly. No neural networks or gradient learning were involved. Two later search rounds found nothing better, so I kept this version. The gains are only chec…
A contact-based stance/flight state machine computes continuous joint torques (knee virtual spring plus thrust, ankle lean balance, hip pitch PD in stance; tuck and posture PD in flight) and quantizes them to {-1,0,1} via sigma-delta dithering.
Shown: public seed 0 (bangbang_v1); this episode passes the forward rung. This episode: episode_steps 1000, x_final 9.32, min_height 1.09, max_abs_pitch 0.138, n_flights 15, total_reward 2163.
controller.py · agent transcript · trial hopper-bangbang__fEr2kuR
bangbang_v1| criterion | graded | needed |
|---|---|---|
| base:forward | 0.425 | ≥ 0.900 |
The agent's final message
The final controller is in place at `/app/work/controller.py`. It does not reach the 0.9 gate on my estimates, so I want to be clear about where it stands. **Result on the official evaluator (visible seeds 0 to 4, `bangbang_v1`, forward rung):** | metric | value | |---|---| | forward success | 4 / 5 | | alive | 4 / 5 | | mean final x | 3.5 m | | mean return | 1265 | On 28 seeds (0 to 27) with my own harness, 14 pass the forward rung and 20 survive the whole episode. The expected held-out rate is therefore around 0.5, below the 0.9 gate. The failing seeds either fall in the first second after an oversized early hop (pitch leaves the band on a straight-leg landing), or stay alive but stop just short of 3 m. **What the controller does.** Every joint runs a ternary sliding-mode rule: a full 200 N m pulse fires only when a surface `error + T * error_rate` leaves a deadband sized to the measured per-pulse kick (about 1 rad/s of pitch rate per pulse). In stance the hip holds torso pitch, the knee regulates the leg lean (pitch minus thigh) and fires a timed, speed-capped extension burst for the hop, and the ankle balances the torso ahead of the foot. In flight the knee folds then re-extends for landing, but any knee pulse that would push pitch the wrong way is vetoed, and the knee takes over pitch when the thigh sits at its 0 rad limit where the hip has no authority. Knee extension pulses are cancelled before the knee can slam its stop, which was the dominant cause of pitch falls…
A touchdown-triggered state machine applies ternary bang-bang sliding-surface pulses at hip, knee and ankle to hold torso pitch, regulate leg lean and balance over the foot in stance, fires a burst of knee-extension pulses for the hop, and in flight controls pitch, knee fold and foot level.
Shown: public seed 4 (bangbang_v1); this episode fails the forward rung. This episode: episode_steps 138, x_final 0.449, min_height 1.13, max_abs_pitch 0.205, n_flights 3, total_reward 192. It was the first failing one of 5 public seeds tried.
controller.py · agent transcript · trial hopper-bangbang__5zN5eD3