pendulum-hold-delay
Inverted pendulum swing-up · solvedexploration 9 of 9 agent attempts pass
Exploration task. Built after the paper's sweep to test whether a task can force a mechanism outside the catalogue (docs/exploration_new_mechanisms.md, docs/soft_arm_spec.md). Not part of the benchmark tiers or of the paper's main counts; the soft arm is reported in the paper as a plant outside the sweep, the other tasks are not; all sessions ran under the paper's protocol.
more
For the upright linearisation (unstable pole p = sqrt(g/l) = 3.13 rad/s) no memoryless state feedback u = -k1 theta - k2 theta_dot stabilises the upright at this delay: with the plant's own 20 ms discretisation the closed-loop spectral radius over a grid of gains exceeds one from 18 samples (0.36 s) on, while the delay stays below the 2/p = 0.64 s bound beyond which no linear time-invariant controller of any kind could stabilise it (Middleton and Miller, 2007). A controller with memory, a predictor advancing the stale measurement through the known model and its own command history, is required by construction; the torque limit (2 N m against a gravity torque of 9.81 sin theta) leaves it a basin of about 0.2 rad, and prediction errors grow by exp(p tau) = 4 over the delay. Open: the memoryless oracle holds the last 3 s on every seed without the delay and on none with it.
Gate. per-seed success is the `terminal` rung of this benchmark's ladder ['achievement', 'terminal'] (`--success-mode terminal`), not the evaluator's default flag; the task is solved when the success rate over the grader's held-out seeds is `>= 0.9`
Axes: dynamics:pendulum, actuation:torque-limited-swingup, sensing:delay-22-samples, gate:terminal-hold-150, exploration:new-mechanism, provable-limit:memoryless-feedback
Each video re-runs the attempt's final controller through the task's own evaluator on a public seed (never the grader's held-out ones). The table under each video is the official grade on the held-out seeds.
base| criterion | graded | needed |
|---|---|---|
| base:terminal | 1.000 | ≥ 0.900 |
The agent's final message
The controller passes the terminal gate on every seed tested, so the submission is complete well inside the time budget. **Result** | Seeds | Terminal success | Final upright error | |---|---|---| | 0-4 (visible) | 5/5 | ~2e-9 rad | | 0-199 | 200/200 | ~2e-9 rad | **What the controller does** (`/app/work/controller.py`) - **State prediction.** The plant model is known exactly and the controller remembers every torque it has applied. Each step it replays its torque history through the pendulum dynamics with RK4 from the delayed measurement, which recovers the current state. This removes the 0.44 s delay from the loop entirely, so the hold is as tight as an undelayed controller. - **Swing-up.** Energy pumping on the predicted state: torque pushes in the direction of the velocity while energy is below the upright rest level, saturating at the 2 N·m limit. Swing-up takes about 6 s on every seed, bounded by the torque limit rather than the gains. - **Hold.** Linear state feedback on the predicted angle and rate once the pendulum is inside the catch region, with hysteresis so it only drops back to swing-up after a genuine fall. The hold begins near step 305 and the pendulum settles to numerically zero error for the final 3 s. The docstring explains the law and each gain, and a `PARAMS` dict with bounds is exposed for the tuner. The main residual risk is the swing-up margin: the hold starts about 0.8 s before the latest allowed start, but all 200 seeds landed within a 0.2 s win…
The controller predicts the current pendulum state by integrating its stored torque history forward from the delayed measurement, then uses energy-pumping swing-up until the predicted state is near upright and switches to PD state feedback to hold, with hysteresis and torque clipping.
Shown: public seed 0 (base); this episode passes the terminal rung. This episode: final_upright_error 4.01e-09, final_abs_velocity 1.48e-07, success_hold_steps 150, total_reward 14.9.
controller.py · agent transcript · trial pendulum-hold-delay__8U6m2yH