humanoid-forward-proprio
Humanoid (MuJoCo Humanoid-v5) · solvedexploration 2 of 9 agent attempts pass
Exploration task. Built after the paper's sweep to test whether a task can force a mechanism outside the catalogue (docs/exploration_new_mechanisms.md, docs/soft_arm_spec.md). The -proprio tasks replace the full-state observation by on-board sensing: encoders, IMU and foot contacts on the humanoid, pressures and bend angles on the soft arm. Not part of the benchmark tiers or of the paper's main counts; the soft arm is reported in the paper as a plant outside the sweep, the other tasks are not; all sessions ran under the paper's protocol.
more
joint encoders, a torso IMU (gyro, gravity direction, specific force with an unknown constant bias per seed) and foot contact switches. Root height, horizontal velocity, heading, the mass centre and the foot positions, which every passing design of the full-observation tasks read directly, are not measured and have to be reconstructed; leg odometry alone is biased by the landing foot sliding, the accelerometer alone drifts.
Gate. per-seed success is the `forward` rung of this benchmark's ladder ['alive', 'forward', 'brisk', 'fast'] (`--success-mode forward`), not the evaluator's default flag; the task is solved when the success rate over the grader's held-out seeds is `>= 0.9`
Axes: plant:humanoid, scenario:proprio_v1, gate:forward_3m, exploration:partial-observability, sensing:proprioceptive
Each video re-runs the attempt's final controller through the task's own evaluator on a public seed (never the grader's held-out ones). The table under each video is the official grade on the held-out seeds.
proprio_v1| criterion | graded | needed |
|---|---|---|
| base:forward | 1.000 | ≥ 0.900 |
The agent's final message
The controller in `/app/work/controller.py` passes. In my own runs it completed all 380 seeds I tried (0–179 and 200–399) at the `forward` rung: it stays up for the full 15 s and the mass centre covers about 8 m, roughly 0.53 m/s. The official verifier on seeds 0–4 reports a success rate of 1.0, return about 5500, and alive/forward/brisk/fast at 1.0/1.0/0.6/0.0. It took about 18 of the 60 minutes. **How it works** - **Estimation:** - Roll and pitch come from the IMU, and yaw is integrated from the gyro. - The controller loads its own copy of the MuJoCo model and poses it with the measured joints and attitude. That gives the mass centre, the foot contact points and their Jacobians. - Velocity comes from integrating the accelerometer, corrected each step by the velocity implied by a foot that has been planted for at least 2 steps. - **Control law:** - Legs alternate stance and swing on a 0.22 s timer, or switch early when the swing foot lands. - The swing foot is placed ahead of the mass centre by an amount that grows with walking speed, plus a correction toward a target of 0.2 m/s along world x. The placement allows for the knee bend. - The stance hip holds the torso's pitch and roll level and subtracts the swing hip's torque. Knee, hip-twist, abdomen and arm joints are held by simple spring-damper loops. - **Tuning:** I picked the structure and starting gains by hand. A derivative-free random search over the `PARAMS` dict, run on simulated episodes and scored mai…
A two-state alternating-stance walker places the swing foot using a capture-point rule on an estimated COM velocity, while the stance hip servoes torso pitch and roll with SIMBICON swing-torque coupling and PD servoes handle the knees, abdomen and arms.
Shown: public seed 0 (proprio_v1); this episode passes the forward rung. This episode: episode_steps 1000, x_final 8.41, min_height 1.26, max_abs_pitch 0.127, n_steps 102, total_reward 5531.
controller.py · agent transcript · trial humanoid-forward-proprio__RBFjp2M
proprio_v1| criterion | graded | needed |
|---|---|---|
| base:forward | 0.300 | ≥ 0.900 |
The agent's final message
Stopping here with the final controller in place. The clock reads 16:05, about 18 minutes before the budget ends, and any further parameter change would need another multi-seed validation to be trustworthy. **Outcome: the gate is not met.** The final controller at `/app/work/controller.py` walks, but not reliably enough. The official evaluator on seeds 0-4 reports 40% alive and 20% forward. On forty seeds it was not tuned on, my harness measured about 50% alive and 33% forward, with a mean progress of 2.5 m. The grader's threshold is 90% forward. **What the controller does.** The feet are spheres with no ankles, so static standing is impossible and the robot has to balance by stepping. The controller is a Raibert-style walker: - The MuJoCo model is loaded inside the controller and filled with the joint angles and IMU attitude. Assuming the stance foot's contact point is fixed, it reconstructs the mass-centre position and velocity that the sensing does not provide. - The stance hip holds the upper body with PD terms on both the chest attitude from the IMU and the pelvis attitude from the model. The stance knee is a soft spring with a feed-forward extension. - The swing foot is placed by Raibert's rule relative to the mass centre, with a lateral capture-point gain, through an analytic leg inverse kinematics tracked by soft joint PD. - Steps alternate every 0.22 s. The 31 gains were tuned by random search on seeds 0-15 and checked on held-out seeds. **Key findings along the …
A Raibert-style stepping controller that reconstructs COM position and velocity from proprioception via a MuJoCo kinematic model, places the swing foot relative to the COM using speed and lateral-velocity feedback with analytic leg IK, and holds the torso upright with stance-hip PD while alternating legs on a timer or at touchdown.
Shown: public seed 0 (proprio_v1); this episode fails the forward rung. This episode: episode_steps 1000, x_final 2.23, min_height 1.09, max_abs_pitch 0.471, n_steps 68, total_reward 5038. It was the first failing one of 1 public seeds tried.
controller.py · agent transcript · trial humanoid-forward-proprio__z8HSqKv
proprio_v1The agent's final message
The controller doesn't pass: no visible seed reached the `forward` rung, so this submission scores zero. The robot falls on every run, typically after 100–600 of the 1000 steps. The best episode lasted 597 steps and covered 2.6 m; the gate needs all 1000 steps and 3 m. I stopped at 16:45, about 53 of the 60 minutes. **What's in `/app/work/controller.py`** - **State estimator** (rebuilds the full state from the on-board sensors alone): - Joints come from the encoders and roll/pitch from the IMU. - Yaw comes from integrating the gyro. - Root velocity comes from integrating the accelerometer (which turned out to be exact apart from its small bias), pulled towards the velocity implied by a foot on the floor not moving. - Root height comes from placing the lowest touching foot on the floor. - **Sampling-based predictive control on the sandbox's own MuJoCo model**: 64 perturbed plans of joint target angles, looking 0.75 s ahead, are simulated in parallel each step. The cheapest plan wins, and a PD law turns its first targets into the clipped motor commands. The cost penalises: - drifting from a 0.4 m/s target speed; - torso height loss and tilt; - the balance point (where the mass centre is heading) leaving the space between the feet; - hopping; - fast joint motion. - The docstring explains each gain and says plainly that the controller doesn't pass. **What I learned** - Sampling raw motor torques can't even keep it standing, with or without true state. Sampling…
Each step the controller estimates root pose and velocity from IMU, encoders and foot contacts, then runs predictive sampling (64 Gaussian perturbations of a 5-knot joint-target plan rolled out in parallel in MuJoCo) with a velocity/height/uprightness/capture-point cost, applying the best plan's first target through a clipped PD law.
Shown: public seed 0 (proprio_v1); this episode fails the forward rung. This episode: episode_steps 284, x_final 1.24, min_height 0.996, max_abs_pitch 0.545, n_steps 49, total_reward 1420. It was the first failing one of 1 public seeds tried.
controller.py · agent transcript · trial humanoid-forward-proprio__V22dpXd