hopper-parity

`/app/work/controller.py` passes both gates on the standard scenario, and I stopped after about 38 of the 60 minutes. | Seeds | `fast` success | Mean reward | Shortest run (18 m needed) | Largest \|pitch\| (limit 0.2) | |---|---|---|---|---| | 0–4 (the given command) | 5/5 | 3451 | — | — | | 0–99 | 100/100 | 3450 | 19.48 m | 0.144 rad | | 200–499 (not used in tuning) | 300/300 | 3451 | 19.52 m | 0.150 rad | That is above the TD3 reference of 3532 ± 12 only in distance terms near its regime; in reward it sits slightly below TD3 but well above the 3179 gate. **What the controller is:** a single linear feedback law on the 11-value Gym observation: `a = clip(K·((s − MU)/SD) + B, −1, 1)`. It has 36 numbers (a 3×11 gain matrix and 3 biases) and keeps no internal state. The gait cruises at about 2.45 m/s, roughly 19.6 m in the 8 s episode. The docstring explains the law and what each row and bias does. Individual gains don't have a stand-alone physical meaning, because they were found by search. **How I got the gains:** a CMA-ES search (`opt.py`), with no gradients and no neural network. Each candidate was scored by full rollouts on scratch seeds 100–107. - **First structure abandoned:** a hand-built Raibert-style controller that switches on foot contact either learned to stand still or lunged forward and fell. The linear law started hopping within a few minutes of search. - **Lesson from one candidate:** one fast version (19.43 m mean) overfit the 8 tuning seeds and failed on 1…

The controller does not reliably pass the gate yet. It clears the return target, but reaches 18 m on about 83% of seeds against the required 90%, so on the grader's seeds it will most likely score zero. About 52 of the 60 minutes are used, and I've stopped the searches. **What's in `/app/work/controller.py`:** a single linear feedback law, `u = clip(K @ (gym_obs - s0) + b, -1, 1)`. It has no clock or phase logic: hopping emerges as a stable cycle of about 14 hops in 8 s at roughly 2.3 m/s. I tuned the gains with a hand-written CMA-ES (an evolution strategy, no gradients or neural networks). The docstring explains what each group of gains does. **Results on 65 seeds the search never saw (0–4 and 20–79):** | | Final controller | Gate | |---|---|---| | Mean return | about 3240 (3251 on seeds 0–4) | ≥ 3179 — passes | | Reached 18 m (`fast`) | 54/65 (83%) | ≥ 90% — fails | | No fall | 60/65 | — | On the official seeds 0–4 it reaches 18 m on 3 of 5. Seed 1 stops at 17.93 m and seed 2 falls at step 871. **Choice you may want to revisit:** I submitted the faster version over a slightly slower one that never fell on those 60 seeds (seeds 20–79). The slower one, at `/app/work/cand_rp6.py`, reaches 18 m on about 77% with a return of about 3268. I picked the faster one because only the 18 m rate and mean return are graded. If the ladder's "alive on every seed" rung earns partial credit, `cand_rp6.py` would be the better pick. **What I learned:** - The sandbox has only 2 CPUs (12 ar…

I didn't get the controller to pass the gate. The version now in `/app/work/controller.py` is the best I found, and it will score zero. **Official evaluator, seeds 0–4:** `fast` rate 0.20 (gate is 0.9), mean reward 2935 (gate is 3179). Four of the five seeds stay up for the full 8 s, but three of those end at 17.3–17.5 m, just short of 18 m. Seed 3 falls at step 491. **On 48 other seeds (my own faster simulation, same physics):** 33% reach `fast`, 73% stay up the whole episode, mean reward about 3050. **What I tried:** 1. **Hand-built state machine.** Hip holds the torso angle in stance, knee crouches then pushes, foot is placed by speed in flight. With its settings searched by CMA-ES (no gradients), it settled on standing still or slow ankle hops of about 1 m. A clock-driven rhythmic version did the same. 2. **What worked better: one fixed linear feedback law.** Each motor command is a weighted sum of the 11 measured states, plus a foot-contact term and an offset (39 numbers in all). I searched the weights with CMA-ES on 12 cores. During the search, any run that fell behind a target pace of 2.25–2.4 m/s was cut short, which stopped the search settling on standing. Later rounds capped the reward for distance and added a bonus for surviving to 18 m. That raised it from nothing to about 3000 reward and a 20–40% `fast` rate, then it stopped improving. **Why it falls short:** the gait is very sensitive to small differences. Changing only the rounding in the arithmetic changes…

The controller doesn't pass the grading gate yet. The best version reaches the "brisk" rung, not "fast": it survived all 1000 steps on seeds 0 and 1 but covered only 14.1 m and 13.4 m (return 2764 and 2679). The gate needs 18 m on 90% of seeds and a mean return of at least 3179. Under the stated rules this scores zero. **What's in `/app/work/controller.py`:** a model-predictive controller, with no learning involved. - **Plan:** Each step the observation gives the full robot state except `x`, which doesn't affect the dynamics. Every 2 control steps the controller copies that state into its own MuJoCo model of the same hopper. It then plans 1 s ahead, by sampling plans and refining around the best one, using 48 samples and 3 rounds. - **Joint targets:** The plan is a set of joint-angle targets, which a PD servo turns into motor commands (proportional gain 6, damping 0.25, clipped to [−1, 1]). The planning model has the same servo built into its motors. - **Cost:** It mostly penalises falling. It also penalises missing a 2.4 m/s speed target, plus soft limits on pitch and torso height. The docstring explains each gain. **What I found along the way:** - I checked that the planning model reproduces the environment step for step, so model error isn't the problem. - Planning raw torques failed: the robot fell within about 200 steps whatever the cost settings. Switching to joint-angle targets made it run at the right speed but still fall. - A 0.5 s horizon was too short to see past…