From LunarLander to String-Stable Platooning

In the last two posts, I used LunarLander as a small testbed for an LLM-guided heuristic-learning control harness. That was useful because the feedback loop was fast: the agent could read metrics, edit a handwritten controller, tune exposed parameters, and carry a compact whiteboard between attempts.

But LunarLander also has a weakness as a research benchmark. A good language model may already know many landing-controller patterns. So the next question is sharper:

Can the same harness synthesize a decentralized multi-agent controller where the target is not just reward, but string stability under delay and uncertainty?

This post records the platooning experiment end to end. It started with a clean stage-one result on delay and noise, then moved through hard heterogeneity, and finally hit a useful limit on a combined stress test. The final result is not a proof of worst-case string stability, but it is more interesting than a single nice rollout: the harness produced local, non-neural controllers that are clearly better than simple classical baselines under the hard combined setting, while a later Optuna-tuned robust CACC baseline showed that the strongest result is better read as competitive controller synthesis, not as a claim that LLM search dominates all tuned classical designs.

Why platooning?

Vehicle platooning is a good next target because the important property is not only tracking error. It is whether disturbances amplify as they travel down the string of vehicles.

In a simple reward-only setup, a controller can look good while quietly passing too much disturbance to downstream vehicles. For platooning, this is exactly the wrong failure mode. A follower near the tail should not experience a larger spacing transient than the vehicle in front of it.

That makes string stability a natural target for this harness. It also moves the experiment closer to networked control: local observations, delayed information, heterogeneous vehicles, and robustness to leader disturbances.

Environment

The experiment used a small environment ladder rather than a single fixed task.

The simulator is lightweight: a scripted leader and 19 followers, where the same decentralized controller is applied to each follower. Each follower only sees local predecessor information.

The first hard target was hard_delay_noise_v1. After that passed, I moved to hard_heterogeneous_v1, and then to hard_combo_v1.

The key metric is string_peak_gain. It is computed from peak spacing errors along the platoon, using the maximum of:

• neighbor peak amplification: downstream peak error divided by upstream peak error
• tail peak amplification: tail peak error divided by the first follower peak error

For these rounds, I added a dedicated string_performance success mode. The main gate was:

string_peak_gain_max <= 1.35
collision_rate = 0
spacing_error_rms_mean <= 3.0
success_rate high enough on train and holdout

This matters because earlier performance-style runs could pass too easily by finding a safe-enough controller with a loose negative reward target. The new mode makes the platooning claim explicit: suppress amplification.

One detail is worth spelling out because it changes how I read the later failures. The random seeds are not mainly making the cars start unrealistically close together. With the default d0 = 8, time_headway = 1, and nominal_speed = 20, the desired initial gap is about 28 meters, with only +/- 0.5 meters of initial gap jitter. In hard_combo_v1, the hard part is the interaction of delay, dropout, noise, bias, heterogeneous actuation, and repeated braking.

First accepted controller

The first accepted hard-scenario controller was candidate_0010, resumed from the hard_delay_noise_v1 platooning run and accepted under string_performance.

On the target scenario it passed both train and holdout gates:

Here is a rollout from the accepted candidate:

What did the controller learn?

The controller is still a small handwritten Python function. It is not a neural network. Its structure is close to a robust CACC-like law:

• proportional spacing feedback
• relative-velocity damping
• small predecessor acceleration feedforward
• strong own-acceleration damping
• dropout mitigation by remembering the last nonzero predecessor acceleration
• command smoothing
• jerk limiting
• emergency braking based on gap and time-to-collision
• capped catch-up acceleration for large spacing errors

The tuned parameters were:

CANDIDATE_0010_PARAMS = {
    "catchup_accel_fraction": 0.7615343988101403,
    "catchup_error_limit": 21.999018501858096,
    "command_smoothing": 0.3439886173049022,
    "emergency_brake_fraction": 0.9257460030328349,
    "gap_margin_emergency": 7.047212781184004,
    "jerk_limit": 5.550824552116185,
    "ka_ff": 0.29310224838384874,
    "ka_self": 1.46641879653996,
    "kp": 0.6079893788853924,
    "kv": 1.6797829618437539,
    "ttc_margin_emergency": 4.975383082915987,
}

The interesting part is the design pressure. The accepted controller is not just trying to minimize reward penalties. It is shaped by a string-gain gate, so it becomes more conservative about amplification: stronger damping, smoother commands, safety overrides, and reduced reliance on a noisy feedforward channel.

Frozen robustness comparison

After acceptance, I froze candidate_0010 and compared it against:

• zero
• acc_pd
• cacc
• damped_cacc

The full suite used 10 fixed seeds and 8 scenarios:

• nominal_v0
• lead_brake_v0
• lead_sine_v0
• stop_go_v0
• delay_noise_v0
• heterogeneous_v0
• weak_brake_mid_v0
• delayed_heterogeneous_brake_v0

The aggregate result:

The important reading is not “candidate has the best reward.” It does not. cacc still has better average reward. The important reading is:

• candidate_0010 has the best average string gain.
• candidate_0010 has zero collisions across this full comparison.
• candidate_0010 survives delay/noise where cacc and damped_cacc degrade.
• The remaining weakness is the combined delayed heterogeneous case, where success is 0.700.

The scenario-level candidate result:

Error diagnostics

The spacing-error heatmap gives a more direct look at the transient behavior over vehicle index and time:

The peak-error propagation plot is closer to the string-stability question:

These plots are useful because the raw reward can hide the mechanism. A controller can have a worse reward because it keeps larger safety margins, but still be better for the networked-control objective if it prevents amplification and collisions.

Harder ladder

After the first acceptance, I pushed the same harness up the hard-scenario ladder. This is where the experiment became more informative.

One small naming caveat: candidate names are local to a run. The candidate_0010 in the hard-heterogeneous run is a new candidate in that run, not literally the same file as the hard_delay_noise_v1 candidate above.

The ladder result was:

The hard-heterogeneous result was cleaner than I expected. With broad actuator lag/limit spread and one weak-braking vehicle, the accepted candidate passed both train and holdout with low spacing error:

hard_combo_v1 changed the story. It combines the previous stressors: repeated leader braking, delay jitter, observation noise, acceleration dropout, sensor bias, heterogeneous dynamics, and a weak-braking follower. The best useful candidate there was candidate_0002.

On the original train seeds, candidate_0002 passed. On the fixed holdout seeds, it failed the string-performance gate:

The holdout failure was seed 893411: no collision, a healthy minimum gap, but string_peak_gain = 1.544, above the 1.35 gate. That is a qualitatively different failure from the simple baselines crashing.

I tried one more small repair loop. It was instructive but not a win. A later candidate_0009 passed the original train seeds but made the holdout worse, with success 0.600, collision 0.200, and max string gain 3.321. A later candidate_0012 fixed seed 893411 on a five-seed middle gate, but failed seeds 0 and 667812, ending at success 0.600 and max string gain 1.372. In other words, the repair was trading one tail failure for another.

At that point the right move was not another 20-round LLM run. It was a fresh-seed sweep.

Fresh-seed sweep

I evaluated candidate_0002, the later candidate_0012, three simple baselines, and then a tuned robust CACC baseline on the same 50 fresh hard_combo_v1 seeds.

This table changed my interpretation twice. First, candidate_0002 is not just a seed-specific patch: on 50 unseen seeds it stayed collision-free and passed 45 out of 50. But it is also not a worst-case string-stable controller. The five failed seeds were all string-gain tail failures, not collisions:

Second, the tuned baseline matters, but it should be interpreted carefully. This was not an independent robust CACC design dropped in from nowhere. It was a second-stage baseline built from the structure exposed by the LLM-discovered controller: damping, predecessor-acceleration filtering/feedforward, command smoothing, jerk limiting, emergency braking, and capped catch-up. Once I gave that robust CACC-shaped family a fair Optuna tuning loop, it reached 46 out of 50 fresh seeds, stayed collision-free, and slightly improved the worst observed string gain compared with the raw LLM candidate.

That makes the endpoint clearer and less inflated. The harness found a much stronger controller than the simple baselines for this hard combined setting. But the final result is not “LLM beats classical control.” It is closer to: the LLM-guided harness discovered a useful robust-CACC-like structure, and Optuna then refined that structure slightly better than the raw LLM candidate.

There is also a prior-knowledge caveat. The structure is not novel in the control-theoretic sense. Most of its ingredients are familiar CACC or robust-control motifs: damping, feedforward, filtering, smoothing, jerk limits, safety guards, and string-stability-oriented objectives. A language model may well have seen these patterns in papers, code, or technical explanations. So the result should not be read as “the LLM invented CACC.” It is better read as an engineering result: the harness could assemble known motifs, adapt them to this custom simulator and hard metric, and turn them into a runnable controller through failure-driven iteration.

Tuned robust CACC baseline

The Optuna baseline was deliberately not a neural controller. It was also not meant to be a fully independent baseline. I fixed a robust CACC-like controller family by distilling the mechanisms that kept appearing in the successful LLM candidates, especially candidate_0002, and then tuned only its gains and thresholds:

• spacing feedback
• relative-velocity damping
• predecessor-acceleration feedforward
• own-acceleration damping
• predecessor-acceleration filtering
• delay-aware feedforward decay
• command smoothing
• jerk limiting
• gap/time-to-collision emergency braking
• capped catch-up acceleration

There is one important fairness caveat. During inspection, I found that the strong LLM candidate effectively uses rollout-local shared smoothing/filter state. A strictly per-follower-memory Optuna controller could not reproduce that behavior. So I used optuna_robust_cacc_shared, which allows the same kind of rollout-local shared state but resets it at the start of each rollout and seed. This is a fair comparison to the discovered candidate, but it is not the same as proving a strictly decentralized per-follower memory design.

This makes the comparison three-layered:

The tuned baseline’s training and holdout metrics were:

That holdout max gain looks worse than candidate_0002, but the broader Fresh-50 result was slightly better. This is exactly why the fresh sweep was useful: a five-seed gate can overstate or understate tail behavior depending on which rare cases it samples.

Interpretation

This result is the point where the harness feels like it is aiming at a real networked-control property rather than just solving another benchmark.

The claim I would make is modest:

1. The harness can synthesize an interpretable controller that is much stronger than simple hand-designed baselines on the hard combined platooning setting.
2. Adding string_performance changes the search pressure in the right direction.
3. The discovered controller is not magic; it looks like a plausible robust local CACC variant.
4. A strong tuned robust CACC baseline can match or slightly exceed the best LLM-discovered hard-combo candidate, but that baseline reuses the controller family suggested by the LLM search.
5. The current result is statistical and empirical, not a formal guarantee.
6. The hard-combo failures identify a real next problem: rare string-gain amplification without collision.

The guarantee point matters. This is not a proof of formal string stability. It is also not a full industrial platooning stack. It is a controlled experiment showing that an LLM-guided code-search loop can move toward a network-level control objective when the objective is made explicit. The tuned-baseline result makes the story healthier: the harness is useful less as a final numerical optimizer and more as a structure-discovery loop. Once the right controller family is exposed, ordinary black-box tuning can do what it is good at: calibrate gains and thresholds.

What this does not yet prove

This stage does not yet prove:

• formal string stability
• worst-case robustness over all random seeds or all uncertainty realizations
• scalability to 50 or 100 vehicles
• robustness to burst communication loss
• robustness to cut-in or cut-out events
• superiority over a tuned MPC or formally designed robust CACC controller
• superiority over carefully tuned robust baselines; the Optuna-tuned robust CACC baseline slightly outperformed the best hard-combo LLM candidate on this Fresh-50 suite
• strict decentralization if rollout-local shared smoothing/filter state is disallowed
• that the whiteboard mechanism is causally necessary

The last point is especially important. The right ablation is not another nice GIF. It is a code-search curve: full whiteboard versus no whiteboard, no reflection, no previous-candidate summaries, and random/no-whiteboard baselines.

Next

For this stage, I would stop here.

The experiment has done enough to be useful: it produced accepted controllers for delay/noise and heterogeneity, found a strong but imperfect hard-combo controller, and then showed on 50 fresh seeds that the remaining issue is tail string-gain amplification rather than collision avoidance. Another round of single-seed patching would be less informative than changing the evaluation protocol.

The Optuna baseline adds one more useful closure: the best hard-combo LLM candidate is genuinely strong, but not uniquely strong. A tuned robust CACC family can reach the same regime. That is a better endpoint than an easy win over weak baselines.

If I continue this line, the next version should make the task harder in dimensions that matter for networked control and report robustness more directly:

1. Scale from 20 vehicles to 50 vehicles.
2. Use bursty packet dropout, not only independent acceleration dropout.
3. Add stochastic communication delay with a longer tail, for example 0 to 8 steps.
4. Add heterogeneous mass, actuator lag, acceleration limit, and braking limit together.
5. Add one or two weak-braking vehicles at unknown positions.
6. Add sensor bias that drifts slowly during the rollout.
7. Add cut-in, cut-out, or platoon split/merge events.
8. Report a Pareto frontier: nominal reward versus worst-case string gain.
9. Separate strictly per-follower memory controllers from rollout-local shared-state controllers.

For the next acceptance target, I would keep:

collision_rate = 0
success_rate >= 0.9 on a fresh-seed suite
string_peak_gain_max <= 1.35 on fixed holdout and fresh seeds

but add a full-suite robustness metric so a controller cannot overfit only the training scenario. I would also keep the tuned robust CACC baseline in the loop from the start, and add a stronger model-based baseline such as MPC if the goal is to make a broader control claim.

Reproduction

The accepted run was:

harness_runs/platooning/hard_delay_noise_v1/zero/run_20260611-202910-614914_round001_seeds0-1-2
candidate_0010

The frozen full comparison command was:

PYTHONPATH=src .venv311/bin/python -m hl_control.eval.run_platooning_robustness \
  --heuristic-workspace harness_runs/platooning/hard_delay_noise_v1/zero/run_20260611-202910-614914_round001_seeds0-1-2/candidates/candidate_0010/candidate_workspace \
  --heuristic-controller candidate_0010 \
  --include-baseline zero acc_pd cacc damped_cacc \
  --seeds 0 1 2 3 4 5 6 7 8 9 \
  --suite standard lead delay heterogeneity combo \
  --output-dir experiments/platooning_robustness

The output folder was:

experiments/platooning_robustness/20260611-215844-179716

The hard-heterogeneous accepted run was:

harness_runs/platooning/hard_heterogeneous_v1/zero/run_20260611-234904-746176_round001_seeds0-1-2
candidate_0010

The best hard-combo parent was:

harness_runs/platooning/hard_combo_v1/zero/run_20260612-080317-420061_round001_seeds0-1-2
candidate_0002

The 50-seed hard-combo sweep outputs were:

experiments/platooning_seed_sweep/candidate_0002/20260612-212849-088659
experiments/platooning_seed_sweep/candidate_0012/20260612-213007-156787
experiments/platooning_seed_sweep/acc_pd/20260613-115748-824820
experiments/platooning_seed_sweep/cacc/20260613-115749-139933
experiments/platooning_seed_sweep/damped_cacc/20260613-115748-823188
experiments/platooning_seed_sweep/optuna_robust_cacc_shared/20260613-115449-551506

For example, the fresh-seed sweep for candidate_0002 was:

SEEDS=(58421 92734 140569 203847 286105 319774 401226 452991 506318 537902 \
  590144 641337 682019 734508 781245 826730 874192 918604 963771 995318 \
  12763 68244 155902 236481 309115 370226 429884 493021 558730 604118 \
  665902 719443 775016 831557 889201 944602 982331 34791 101884 178223 \
  249650 333908 391770 468502 522614 612709 706381 804226 871009 953440)

PYTHONPATH=harness_runs/platooning/hard_combo_v1/zero/run_20260612-080317-420061_round001_seeds0-1-2/candidates/candidate_0002/candidate_workspace/src \
  .venv311/bin/python -m hl_control.eval.run_platooning \
  --controller candidate_0002 \
  --scenario hard_combo_v1 \
  --seeds "${SEEDS[@]}" \
  --output-root experiments/platooning_seed_sweep/candidate_0002

The tuned robust CACC run was:

PYTHONPATH=src .venv311/bin/python -m hl_control.eval.tune_platooning_optuna \
  --controller-family shared \
  --scenario hard_combo_v1 \
  --n-trials 400 \
  --n-jobs 4 \
  --sampler-seed 2 \
  --output-dir experiments/platooning_optuna_robust_cacc_shared

It wrote:

experiments/platooning_optuna_robust_cacc_shared/20260613-115427-591661

The fresh-seed sweep for the tuned baseline used the frozen parameters from that run:

PYTHONPATH=src .venv311/bin/python -m hl_control.eval.run_platooning \
  --controller optuna_robust_cacc_shared \
  --scenario hard_combo_v1 \
  --controller-params-json experiments/platooning_optuna_robust_cacc_shared/20260613-115427-591661/best_params.json \
  --seeds "${SEEDS[@]}" \
  --output-root experiments/platooning_seed_sweep/optuna_robust_cacc_shared