← all tasks

platoon-combo-20-strict

Vehicle platoon · solved 5 of 6 agent attempts pass

The task. Open. Every controller that passed platoon-combo-20 at the 1.35 rung scores 0/40 at the 1.0 rung; the one controller that reached the 1.0 rung (Opus, 2026-10-22 pilot, 39/40) did so with a 17 m first-follower excursion and fails the 5 m per-vehicle bound.

Gate. no collision; spacing-error RMS <= 3.0 m; per-vehicle peak spacing error <= 5.0 m; string peak gain <= 1.0; success rate >= 0.9 on 40 verifier-only seeds

Axes: gate:peak_5m, n_vehicles:20, observation:local-delayed-noisy, heterogeneity:tau-and-limits, weak-brake:vehicle-10, leader:hard_brake_train, spacing:constant-time-headway, gate:string_1_0

Each video re-runs the attempt's final controller through the task's own evaluator on a public seed (never the grader's held-out ones). The table under each video is the official grade on the held-out seeds.

Best attemptOpus 5.5 (claude-code, 37 min) · hard_combo_v1

Graded on the held-out seeds: passes every gate on the grader's held-out seeds.

criteriongradedneeded
base:string_1_00.950≥ 0.900

How the agent got there: 37 min · 65 tool calls · 2 edit-tool edits · 46 runs of its own Python (tests, experiments, rewrites) · no tuner runs · official verifier 2× (success rates in order: 1.00, 1.00).

The agent's final message

The controller in `/app/work/controller.py` clears the strict string-stability gate (gain ≤ 1.0) at about 94% on the seeds I tested, above the 0.9 target, but not by much. **Results** - **Official evaluator, seeds 0–4:** 5/5 succeed. Worst string gain is 0.85. - **Official evaluator, seeds 5–44:** 40/40 succeed. No collisions, worst string gain 0.985, worst per-vehicle error 4.31 m against the 5 m limit. - **My own harness, seeds 0–143:** 135/144 succeed (93.75%). The grader's seeds are different, so expect somewhere around 90–95% there. **How the controller works** (the docstring in the file explains each gain): 1. **It works out its own state now.** Its own speed and acceleration readings are exact, just late, so it replays its remembered commands through the actuator model to the current step. It also measures its own actuator lag from consecutive readings. 2. **It estimates the vehicle ahead with a small Kalman filter.** This was the biggest single improvement (31/48 → 41/48 seeds). The filter combines the noisy relative-velocity reading every step with the exact acceleration reading when it isn't dropped. Runs of dropouts had been leaving followers braking hard after the car ahead had already started accelerating. 3. **It removes the speed-sensor bias.** It compares the summed relative-velocity readings with the change in measured gap. The gap-sensor bias (up to about 0.5 m) can't be observed, and it is what uses up most of the 1 m allowance. 4. **Spacing law:** `a_ref…

Controller family: string-stable CACC / platoon

Each follower predicts the present gap and predecessor motion from delayed noisy observations using a Kalman filter and model rollforward, computes a CTH reference acceleration that gives first-order error decay, clips it to index-dependent caps, and inverts the actuator lag to produce the command.

Shown: public seed 0 (hard_combo_v1); this episode passes the string_1_0 rung. This episode: collision no, min_gap 14.7, spacing_error_rms 0.275, spacing_error_abs_max 3.27, string_peak_gain 0.761, episode_steps 600, total_reward -304.

controller.py · agent transcript · trial platoon-combo-20-strict__w3qnr2a

FailureFable 5.1 (claude-code, 16 min) · hard_combo_v1

Graded on the held-out seeds: misses the main gate narrowly (success rate between 0.5 and the gate).

criteriongradedneeded
base:string_1_00.525≥ 0.900

How the agent got there: 16 min · 17 tool calls · 0 edit-tool edits · 15 runs of its own Python (tests, experiments, rewrites) · no tuner runs · official verifier 1× (success rates in order: 1.00).

The agent's final message

The final controller is in place at `/app/work/controller.py` and the official verifier passes all 5 visible seeds (0-4) at the `string_1_0` rung. On broader seed blocks it reaches 75 %, which is below the 0.9 gate, so I am reporting this as a strong but not fully solved result. **What the controller does** - Reconstructs its own state exactly from the delayed own-state reading plus the stored command history. The actuator lag is identified from consecutive own-acceleration samples, so the lag inversion makes all vehicles respond alike. - Tracks the predecessor with a 4-state Kalman filter (gap, velocity, acceleration, relative-velocity bias), skipping dropped acceleration readings, and propagates it over the known observation delay. - Applies the ideal constant-time-headway law `a_des = (dv + KP·e)/h`, which has no overshoot and no string amplification when estimates are accurate. - Caps commanded braking: the first follower absorbs the leader's pulses within the 5 m bound, and a uniform downstream cap equalizes heterogeneous brake limits so peaks shrink along the string. The cap relaxes when the predicted error drops below -2 m. - Pre-brakes vehicles whose brake limit is weak relative to the predecessor's deceleration (vehicle 10). **Results** | Seed block | Success (strict 1.0 rung) | Collisions | |---|---|---| | 0-4 (official verifier) | 5/5 | 0 | | 0-19 | 15/20 | 0 | | 20-39 | 15/20 | 0 | Spacing-error RMS is about 0.3 m and the reward is around -290 per seed. No co…

Controller family: string-stable CACC / platoon

Each follower predicts the delayed own and predecessor states to the current step, computes a CTH desired acceleration from the predicted spacing and velocity errors, inverts the identified actuator lag to get the command, and clips it with brake caps and a collision-safety override.

Shown: public seed 8 (hard_combo_v1); this episode fails the string_1_0 rung. This episode: collision no, min_gap 15.9, spacing_error_rms 0.412, spacing_error_abs_max 4.93, string_peak_gain 1.16, episode_steps 600, total_reward -320. It was the first failing one of 9 public seeds tried.

controller.py · agent transcript · trial platoon-combo-20-strict__JyFkYA5