humanoid-brisk-proprio
Humanoid (MuJoCo Humanoid-v5) · solvedexploration 3 of 9 agent attempts pass
Exploration task. Built after the paper's sweep to test whether a task can force a mechanism outside the catalogue (docs/exploration_new_mechanisms.md, docs/soft_arm_spec.md). The -proprio tasks replace the full-state observation by on-board sensing: encoders, IMU and foot contacts on the humanoid, pressures and bend angles on the soft arm. Not part of the benchmark tiers or of the paper's main counts; the soft arm is reported in the paper as a plant outside the sweep, the other tasks are not; all sessions ran under the paper's protocol.
more
joint encoders, a torso IMU (gyro, gravity direction, specific force with an unknown constant bias per seed) and foot contact switches. Root height, horizontal velocity, heading, the mass centre and the foot positions have to be reconstructed, and at this speed the heading is the binding estimate: the full-observation brisk design falls behind any estimator because it servos the heading stiffly to a value no proprioceptive walker can know; the oracle slows the gait and steers through the foot placement instead. Flat counterpart of humanoid-forward-proprio one rung up.
Gate. per-seed success is the `brisk` rung of this benchmark's ladder ['alive', 'forward', 'brisk', 'fast'] (`--success-mode brisk`), not the evaluator's default flag; the task is solved when the success rate over the grader's held-out seeds is `>= 0.9`
Axes: plant:humanoid, scenario:proprio_v1, gate:brisk_8m, exploration:partial-observability, sensing:proprioceptive
Each video re-runs the attempt's final controller through the task's own evaluator on a public seed (never the grader's held-out ones). The table under each video is the official grade on the held-out seeds.
proprio_v1| criterion | graded | needed |
|---|---|---|
| base:brisk | 1.000 | ≥ 0.900 |
The agent's final message
`/app/work/controller.py` now reaches the `brisk` rung on every seed I tried. It passed 10 of 10 on the official verifier with `--seeds 0-9`. On my own batch run it passed 40 of 40 on seeds 0–39 and 160 of 160 on seeds 100–259 (`proprio_v1`, `brisk` success: alive for 1000 steps and at least 8 m). Mean progress was about 17 m, so most runs also clear the `fast` rung. On the verifier's 10 seeds, `fast` was 0.90. I did not run the perturbation scenarios, since the task grades only `proprio_v1`. **How it works** - **State estimate:** it keeps a private copy of the MuJoCo model. It builds the torso attitude from `roll`/`pitch` and integrates the gyro for heading. That gives the mass-centre and foot positions. It gets the mass-centre velocity from whichever foot is in contact, since that foot is stationary. - **Stance leg:** hip torque holds the torso upright, and the knee holds a fixed leg length. - **Swing leg:** the foot goes to a target placed relative to the mass centre by a Raibert-style rule (forward offset from velocity, lateral offset from width plus sideways velocity). Inverse kinematics turns that into joint targets, tracked with joint PD. - **Stepping:** the legs swap when the swing foot touches down or a timeout expires. The first 0.3 s is spent standing on both legs. - **Tuning:** I tuned the 25 gains with CMA-ES on seeds 20–31, using fitness that rewards staying alive and is capped at 11 m of progress. Later fitness runs were not needed once the batch validation pa…
A two-state stance/swing walker that estimates COM velocity from the stationary stance foot, places the swing foot via a Raibert velocity-based rule tracked by IK and joint PD, while the stance leg holds torso pitch/roll and a knee-set leg length.
Shown: public seed 0 (proprio_v1); this episode passes the brisk rung. This episode: episode_steps 1000, x_final 17.4, min_height 1.07, max_abs_pitch 0.532, n_steps 123, total_reward 6258.
controller.py · agent transcript · trial humanoid-brisk-proprio__nwWHxYX
proprio_v1| criterion | graded | needed |
|---|---|---|
| base:brisk | 0.875 | ≥ 0.900 |
The agent's final message
The controller doesn't pass the gate. It's brisk on about 80–85% of seeds, and the task needs 90%. Under the grading rule, a submission below the gate scores zero. **Results.** On the official evaluator (seeds 0–4) it reaches 0.80 at alive, forward and brisk. Four seeds walk the full 15 s and cover 8.5–9 m; seed 0 falls at step 170. On my own runs over seeds 0–71 it's brisk on about 83%, with 12-seed blocks ranging from 75% to 100%. When it fails, it falls; it never survives but comes up short on distance. **What's in `/app/work/controller.py`:** - **Sensing:** heading comes from integrating the gyro. Mass-centre position and velocity come from the MuJoCo model, assuming the foot in contact rolls without slipping. Checked against the true state, both were accurate whenever a foot was down. - **Gait:** the legs alternate stance on a timer. After each touchdown there is a short phase on both feet that damps sideways speed and corrects heading. In the swing, the hip aims the thigh at an angle set by where the mass centre is and how fast it moves (SIMBICON style), and the knee follows a fixed bend-then-straighten profile. The stance hip holds the torso upright and cancels the swing hip's push-back. - **Gains:** found by random search on seeds 0–3. The docstring explains each gain and records this status. **It's fragile.** Changing a single parameter, for example the target speed from 0.81 to 0.7 or 0.55 m/s, drops it to 0% brisk. A second search run on 8 seeds overfit and did …
A foot-contact-driven state machine alternates stance legs, using SIMBICON swing-hip angles set by CoM offset and velocity with a two-stage knee schedule, torso PD on the stance hip, and CoM velocity estimated from the kinematic model assuming a non-slipping pivot foot.
Shown: public seed 0 (proprio_v1); this episode fails the brisk rung. This episode: episode_steps 170, x_final 1.53, min_height 0.986, max_abs_pitch 0.845, n_steps 17, total_reward 934. It was the first failing one of 1 public seeds tried.
controller.py · agent transcript · trial humanoid-brisk-proprio__Xwm3MxN
proprio_v1| criterion | graded | needed |
|---|---|---|
| base:brisk | 0.350 | ≥ 0.900 |
The agent's final message
The controller does not meet the gate: it reaches the `brisk` rung (8 m in 15 s without falling) on only about 19% of held-out seeds, against the 90% required. It still falls on most seeds, so I'm submitting it knowing it fails. I stopped at roughly 52 of the 60 minutes. `/app/work/controller.py` holds the best parameter set I found. **What the controller does.** A clock-driven gait generator moves the two legs in antiphase through hip, knee and lateral weight-shift references. Joint PD control tracks them, and IMU pitch, roll and gyro feedback adjusts the hips and abdomen. I tuned its 24 parameters with CMA-ES, a derivative-free search, on the visible seeds. The docstring in the file describes the control law and the role of each gain. **Results.** - **Baseline:** zero control falls in about 0.6 s. - **Gait:** the tuned gait walks at up to about 0.9 m/s on some seeds, but it falls on most. - **Held-out seeds:** the submitted parameters reached `brisk` on 3 of 16 fresh seeds (20–35), which is the 19%. - **Overfitting:** a later parameter set that scored higher on the tuning seeds did worse on the fresh ones (12.5%, 2 of 16), so I kept the earlier one. - **Scenario:** I only evaluated the `proprio_v1` scenario. With more time, the next steps would be tuning on many more seeds at once and driving the gait phase from foot-contact events instead of a fixed clock.
A fixed-frequency clock generates sinusoidal hip, knee, and lateral references for the two legs in antiphase, tracked by joint PD servos, with torso pitch/roll and gyro feedback shifting the references for balance and gains tuned by CMA-ES.
Shown: public seed 0 (proprio_v1); this episode fails the brisk rung. This episode: episode_steps 164, x_final 2.11, min_height 0.997, max_abs_pitch 0.416, n_steps 20, total_reward 963. It was the first failing one of 1 public seeds tried.
controller.py · agent transcript · trial humanoid-brisk-proprio__M9dsvvB
proprio_v1The agent's final message
The controller is not confirmed to pass: on the official evaluator, seeds 0–4 score **0.8 brisk**, under the 0.9 gate. Every seed stays on its feet for the full 15 s; the one miss, seed 3, stops at 7.68 m against the 8 m threshold. I used about 50 of the 60 minutes. A full 5-seed evaluation takes 6–7 minutes and I had no untested improvement worth the rest, so I stopped. **How it works** (`/app/work/controller.py`, no learning): 1. **State estimation from onboard sensors only.** - Joint angles and rates come straight from the encoders. - Attitude is roll and pitch from the IMU, with yaw integrated from the gyro starting at 0. - Root velocity and height come from a filter. The accelerometer is integrated every step, which is essentially exact here apart from its small fixed bias. While a foot is touching the floor, the estimate is gently pulled toward what the leg geometry implies, since a planted foot doesn't move. - Relying on contact geometry alone gave about 0.35 m/s RMS velocity error, because the gait spends about 20% of its time with both feet off the ground. Adding the accelerometer brought that down to about 0.04 m/s. 2. **Planning on its own copy of the robot model.** Each 15 ms step it simulates 128 perturbed torque plans 0.72 s ahead, all in parallel. It scores them on forward speed, sideways drift, head height, staying upright, balance, facing +x, foot spacing and a fall penalty, and applies the first command of the best plan. **Results:** | Version…
Each step it estimates the full humanoid state from onboard sensors, samples 128 noisy piecewise-linear control plans, rolls them out in a internal MuJoCo model, picks the lowest-cost plan under a speed/upright/capture-point/feet-gap cost, and applies its first command.
Shown: public seed 0 (proprio_v1); this episode passes the brisk rung. This episode: episode_steps 1000, x_final 8.39, min_height 1.1, max_abs_pitch 0.874, n_steps 113, total_reward 5399. It was the first failing one of 1 public seeds tried.
controller.py · agent transcript · trial humanoid-brisk-proprio__dMRzu9J