Six posts in, we have a working answer to one of agentic AI's hardest problems. A user describes a task in English. A multi-agent pipeline turns it into PDDL, validates it, solves it, and returns a verified plan. On standard domains it gets close to 100%. The combination of LLMs and formal planning works.
It also has a billing problem. Every refinement step in Post 6's pipeline calls a frontier model β GPT-5-mini, GPT-5, Gemini-3-Flash β to decide which repair agent to invoke next. A few steps per problem, twenty-odd cents per task, thousands of problems: orchestration cost dominates everything. Worse, it pins deployment to a continuous, paid API connection. Lab-scale, edge, and air-gapped use cases β robotics, manufacturing, classrooms β are priced out.
This post asks a question that should have been asked sooner: does the orchestrator need to be a frontier LLM at all? The repair agents do the heavy lifting. The orchestrator's job is one decision at a time: which agent, given this state. That's a 21-way classification problem β and we already have, by construction, the perfect supervisor sitting in the loop: the validator that decides whether a plan is correct.
The headline result of the author's recent paper β a system called HALO, for Hybrid Agent-Learned Orchestrator β is that a small QLoRA-tuned Llama-3-8B model, trained on a few thousand verifier-accepted refinement trajectories and paired with a thin hardcoded-rule layer, matches or beats both a prompted GPT-5-mini orchestrator and a Gemini-3-Flash orchestrator across 12 PDDL domains. Against GPT-5-mini, HALO actually exceeds the teacher on every benchmark family (by 3.3, 37.3, and 2.8 percentage points on PlanBench, Natural Plan, and classical planning). Orchestration cost drops from about $0.18 to $0.004 per task β roughly 45Γ cheaper than GPT-5-mini and 15β20Γ cheaper than the already-cheap Gemini-3-Flash baseline. LLM calls per episode drop by 40β50%; per-step self-correction climbs from 50% to 83%. The rest of this post unpacks how, and what it means for agentic AI more broadly.
The Bottleneck We Inherited
Post 6's pipeline β the agentic PDDL framework of La Malfa et al. (2025) β decomposes the refinement loop into thirteen specialised repair agents. AgentSyntaxPDDL repairs syntax errors, AgentHallucinations strips predicates the LLM invented, AgentFastDownwardsAdapter rewrites the PDDL for Fast Downward, and so on. The cycle: orchestrator picks an agent, agent edits the PDDL, planner re-runs, validator re-runs, orchestrator picks again. The loop terminates when the validator accepts the plan or the iteration budget runs out.
The structure is excellent. Specialised agents with narrow prompts beat one big prompt asking an LLM to "fix the PDDL." Verifier-checked refinement provides hard correctness guarantees. The combination lifts smaller models to near-frontier accuracy.
The economics are not. At every refinement step the orchestrator β itself a frontier LLM β is fed the entire state and emits one agent name. We pay frontier-LLM rates on a multi-thousand-token prompt to receive a five-word answer, ten times per problem.
Where the Cost Hides
Per-step LLM call to a frontier model
- ~2,000-token state in every prompt
- Frontier LLM API call per refinement step
- Returns one agent name (~5 tokens out)
- Several steps per problem Γ thousands of problems
- Cannot run offline / on-device
One forward pass on a local 8B model
- State encoded once, truncated to 2,048 tokens
- Single-token decision (~10 ms on a 24 GB GPU)
- Three trivial cases handled by hardcoded rules
- 40β50% fewer LLM calls per episode end-to-end
- Runs offline, on-device, air-gapped
The economics of orchestration. A prompted frontier-LLM orchestrator pays full price for a tiny output, every step, every problem. HALO pays roughly 45Γ less than GPT-5-mini and 15β20Γ less than the already-cheap Gemini-3-Flash β for matching or better decision quality. Gemini-3-Flash is the right comparator: it's the strongest competitive cost baseline, and HALO is still more than an order of magnitude cheaper than it.
The orchestrator's output space is tiny β 21 agent IDs β but it's being asked from a generalist model that doesn't know that. We're paying for an oracle that can write essays and translate between languages, then using exactly one of its 32,000 vocabulary tokens. The mismatch is the cost.
The Credit Assignment Problem
So why hasn't this been done already? The decision "which agent, given this state" has no ground-truth label in any dataset β the right choice at step 3 depends on what happens at steps 4 through 8, and the only verifiable signal (whether the final plan succeeds) is several causal steps removed from each individual decision. This is the classic credit assignment problem.
The textbook tool is reinforcement learning: emit a sparse reward at the end and propagate gradients back through the trajectory. RL works, but it's slow. Sparse rewards need thousands of episodes, and a thousand episodes of a PDDL refinement loop is a thousand Γ ten frontier-LLM calls of data collection before training even begins. The cost we're trying to remove gets paid up front.
Credit Assignment: Three Ways to Decide Who Did the Right Thing
Sparse RL: only the terminal reward is observed; the policy must figure out which step earned it.
?
?
?
?
+1
Dense RL (shaped rewards): hand-craft a per-step signal. Powerful but brittle β reward hacking is real.
+0.2
+0.1
β0.3
+0.4
+1
Verifier-filtered imitation (this paper): keep only trajectories the verifier accepts. Every kept step is by definition a step on a winning path.
β
β
β
β
VAL β
Three ways to assign credit in a refinement loop. The verifier-filtered approach side-steps both RL's sparsity and shaped-reward brittleness: the data-acceptance filter is the credit signal.
The paper takes a third path. Treat each accepted trajectory as a sequence of demonstrably correct decisions and learn from those directly. A verifier-accepted trajectory is, by construction, a path that ended in a valid plan; whatever decisions it contains worked in the operational sense that the planner and validator certified them. This reframes orchestrator training from reinforcement learning to verifier-filtered imitation β the credit signal isn't a reward propagated through time, it's a binary acceptance gate applied to whole trajectories before any gradient is computed. The verifier we already trust to certify plans doubles as the data-acceptance filter for training.
Agentic frameworks built around formal verifiers already have a credit signal β they just haven't been using it. Every refinement trajectory the verifier accepts is a sequence of (state, agent) pairs that worked. That's per-step supervision, available for free, sitting in the framework's logs.
Why "Train the Orchestrator" Belongs on Every Agentic Roadmap
Multi-agent frameworks β CrewAI, AutoGen, LangGraph, MetaGPT, Post 6's agentic PDDL pipeline β almost all share the same architecture: a pool of specialised agents coordinated by a prompted generalist LLM. The generalist is treated as load-bearing. It isn't: the orchestration decision is a small discrete classification problem in a structured state space, and every one of these frameworks ships with some downstream verifier β a test suite, a type checker, a linter, a deployment health-check, a policy engine, a compiler. PDDL is just the most natural setting because its verifier is the most rigorous. The same template β collect trajectories, filter by verifier acceptance, fine-tune a small model on the surviving (state, action) pairs β applies wherever a closed-loop multi-agent system has a way to check its own work.
When You Can Train the Orchestrator
A multi-agent system admits a trainable orchestrator when:
- A discrete agent pool. The decision is "pick one of N," not "write arbitrary code." N in the dozens is fine; N in the thousands is harder.
- A verifiable terminal. Some external check determines whether the final output is correct. Compiler, test suite, type checker, validator, policy engine, manual review β anything binary.
- A trajectory log. The framework records, per step, the state the orchestrator saw and the agent it picked. Most frameworks already do this for debugging.
- A modestly-sized state. The encoded state fits in a small model's context window (a few thousand tokens). PDDL fits comfortably. So does most code review, refund processing, or CI/CD state.
If you tick all four boxes, the orchestrator can be small, local, and learned.
A short checklist for whether a multi-agent system is a candidate for orchestrator training. The PDDL setting hits every box hard; most agentic pipelines hit every box at least softly.
The Method, End to End
Figure 1 shows the framework with the three contributions highlighted: a trained orchestrator at the centre of the refinement loop, a hybrid hardcoded-plus-learned policy, and an expanded 21-agent action space.
Source: Training the Orchestrator: A Supervised Approach to End-to-End PDDL Planning with LLM Agents
Description: The end-to-end framework. A natural-language specification is converted to a draft PDDL pair, iteratively refined inside the loop until the verifier accepts a plan; the plan is then rendered back into natural language. Three contributions sit at the centre β (1) a locally-served orchestrator trained on verifier-accepted trajectories, (2) a hybrid of hardcoded rules and a learned policy, (3) an agent pool expanded from 13 to 21.
Figure 1 β The agentic PDDL framework with the trained orchestrator at its centre.
The high-level picture is unchanged from Post 6: NL spec in, draft PDDL refined inside a verifier-checked loop, plan rendered back to English. What changes is the orchestrator at the centre. The next four subsections describe the moving parts: the expanded action space, the verifier-filtered trajectory collection, the hybrid policy, and the SFT recipe.
1 β The 21-Agent Action Space
The orchestrator chooses one of 21 agents. Thirteen are the LLM-prompted repair routines from La Malfa et al.; five are pure Python plan-repair routines from Armony et al. (2025), each running in milliseconds with zero LLM cost; three are PDDL-aware LLM agents from NL2Plan (Gestrin et al. 2024) addressing modelling errors the baseline 13 don't cover. Agent IDs are integers 0β20, chosen so that every ID is a single token under the Llama-3, Qwen-2.5, and Gemma-2 tokenisers.
The 21-Agent Action Space
Each agent reads the relevant slice of state and returns a candidate update; the framework re-runs the planner and validator on the result.
The expanded 21-agent action space. The orange outline marks the eight agents added in this work. The five deterministic repair routines contribute zero LLM cost β and, as we'll see, end up being some of the most-used agents on classical-planning benchmarks.
The five deterministic agents earn their place: when the planner returns a plan that's almost right β wrong variable bindings, out-of-order actions, a redundant step β calling AgentVariableSwapper for a millisecond is dramatically better than calling another LLM. The three PDDL-aware agents plug a different gap: AgentTypeHierarchyFixer reorganises :types, AgentPredicateGeneralizer widens an over-specific predicate to its parent type, and AgentInitialStateSuggester infers missing :init facts from the NL spec. Each one fixes a class of modelling error the baseline 13 silently struggle with.
2 β Verifier-Filtered Trajectory Collection
This is the part of the method doing the conceptual work. The training pipeline follows the GABAR template (Mangannavar et al. 2025) applied to orchestration rather than action ranking: pass training problems through a strong prompted teacher, log every step of every rollout, and filter aggressively for trajectories the verifier accepts.
Source: Training the Orchestrator: A Supervised Approach to End-to-End PDDL Planning with LLM Agents
Description: (a) Training problems from 12 PDDL domains pass through a strong prompted teacher (GPT-5-mini, with Gemini-3-Flash for diversity). Rollouts go through a three-pass filter β hard verifier filter that discards trajectories not ending in a valid plan, spec-level augmentation, LLM-as-judge soft filter β producing β12β15k (state, agent) pairs. The orchestrator Ο_ΞΈ is fine-tuned on these pairs with single-token cross-entropy under QLoRA. (b) The same Ο_ΞΈ is then dropped into the Refiner block of the framework, wrapped by a hardcoded-rule Layer 1 to form Ο_hybrid β together: HALO.
Figure 2 β Training pipeline (a) and inference-time deployment (b).
Three-Pass Trajectory Filter
Teacher rollouts ~thousands
GPT-5-mini (with Gemini-3-Flash for diversity) runs as the prompted orchestrator over training problems from 12 PDDL domains spanning PlanBench, Google Natural Plan, and classical planning. Every step is logged: the state, the agent chosen, whether a rule would have caught it, the agent's response, the post-execution state, and validator metrics before and after.
Verifier acceptance ~200 seeds survive
A trajectory is kept only if the validator (VAL / uVAL) certifies the terminal plan as valid. Partial trajectories that exhaust the iteration budget without producing a valid plan are discarded. The verifier never enters the loss; it's purely a data-acceptance gate.
Spec-level paraphrasing ~5k trajectories
Survivors are augmented by paraphrasing the natural-language spec, renaming objects, and varying goal surface forms. These perturbations preserve the validator's acceptance assignment while expanding the dataset roughly 25-fold from the seed trajectories.
LLM-as-judge rating ~2k trajectories
A frontier LLM rates each augmented trajectory on per-step rationale coherence and final-plan quality. Trajectories scoring below the threshold are dropped. The soft filter removes cases where the final plan is valid but intermediate steps are noisy β keeps the strong examples, drops the lucky ones.
Supervised (state, agent) pairs ~12β15k
Every (s_t, a_t) pair from surviving trajectories becomes one training example β except steps where a Layer-1 rule would have fired, which are excluded so the model never wastes capacity learning patterns a rule already resolves correctly.
Three-pass filter. The hard verifier filter is the conceptual core β it converts whole-trajectory acceptance into per-step labels. The augmentation and soft-judge passes scale and clean the data; nothing in either of them touches the verifier's binary signal.
Two details matter. First, the verifier never appears in the loss. It's an external Boolean: this trajectory ended in a valid plan. Per-step decisions are tagged as supervision purely because the trajectory they belong to was accepted as a whole β the verifier hardens which data the gradient sees, not which direction the gradient points.
Second, the training set excludes steps where a Layer-1 rule would have fired. The model should never be asked to predict patterns a rule already resolves correctly; excluding them sharpens the supervised signal.
The verifier is doubly useful: at training time as a data-acceptance gate, and at inference time as the termination oracle. The same component plays both roles. It's a structural property of agentic frameworks built around formal verifiers β and it's why this approach generalises beyond PDDL.
3 β A Hybrid Hardcoded-plus-Learned Policy
At inference the orchestrator is two layers stacked. Layer 1 is a tiny set of hardcoded rules that resolve trivially-decidable decisions instantly. Layer 2 is the trained model, called only when no rule fires. Together: Οhybrid.
Οhybrid = Rules β Trained Model
Three trivially-decidable cases Β· zero model cost
- If πt is empty (cold start) β pick
AgentEmergency. - If et contains a syntax error β pick
AgentSyntaxPDDL. - If V(πt, π«t, Οt) = valid β pick
NoOpAgent(terminate).
Each rule returns its agent name immediately. No model call. No prompt assembly. Captures roughly one-third of all decisions across the benchmarks.
QLoRA-tuned 8B model Β· single-token decision Β· <10 ms
If no Layer-1 rule fires, the trained policy ΟΞΈ sees the encoded state and emits a single token from {0, β¦, 20}. Sampling is greedy (temperature 0), consistent with the per-decision cross-entropy training objective.
The hybrid policy. Layer 1 handles the three trivially decidable cases by rule; Layer 2 handles everything else by learned single-token classification. The split is propagated into training: the supervised set excludes any step that Layer 1 would have caught, so the model spends its capacity on the genuinely ambiguous decisions.
The three rules are picked because they are unambiguous β any reasonable orchestrator would make the same choice from these states. Ablations in the paper confirm both layers are necessary: removing the rules raises trained-policy queries by ~35% per episode; removing the trained model collapses success rate.
4 β Supervised Fine-Tuning with a Single-Token Target
The fine-tuning recipe is intentionally boring: standard QLoRA, AdamW, single-token cross-entropy. The state st is rendered into a structured prompt with eight capped sections (NL spec, current PDDL domain and problem, current plan, validator errors, planner logs, recent agent history, available-agent list), left-truncated within each section to fit a 2,048-token context. The target is a single agent-ID token aβ
β {0, β¦, 20}; all prompt tokens carry label = β100:
One token per example. The gradient does not propagate through the encoded state. Architecture is QLoRA at rank 16, Ξ± = 32, on the attention and MLP projections. AdamW at LR 2 Γ 10β5, effective batch 16, three epochs. A full Llama-3-8B fine-tune on the assembled ~2k trajectories takes about six wall-clock hours on a single 8ΓA100 node; inference is <10 ms per decision on a 24 GB consumer GPU.
Single-token output isn't cosmetic. It guarantees that one forward pass produces one validly-formatted decision. No parsing, no JSON repair, no "the model said SyntaxFixer but the registry has it as AgentSyntaxPDDL" fragility. A startup pass verifies that every agent ID tokenises to exactly one token under the active tokeniser β the load-bearing detail that makes deployment robust.
What It Buys You
HALO is evaluated on 12 PDDL domains across three benchmark families β PlanBench (Blocksworld, Depots, Logistics, Mystery Blocksworld, Obfuscated Deceptive Logistics), Google Natural Plan (Calendar, Meeting, Trip), and classical planning (Blocksworld, Hanoi, Childsnack, Floortile). Baselines are the unmodified La Malfa pipeline with GPT-5-mini and Gemini-3-Flash as the prompted orchestrator; the agent pool is held at 21 for all configurations.
Success Rate by Benchmark Family
Fraction of problems whose final PDDL + plan satisfies the validator
HALO exceeds GPT-5-mini by 3.3, 37.3, and 2.8 percentage points on PlanBench, Natural Plan, and classical planning respectively β the largest gain on Natural Plan, where GPT-5-mini drops to 51.5%. HALO sits within 3 pp of the stronger Gemini-3-Flash baseline on PlanBench and Natural Plan, and beats it by 2.8 pp on classical planning. *Gemini-3-Flash classical: Borealis subset only.
Orchestration Cost per Task
USD per problem (PlanBench / classical figures shown; Natural Plan is slightly higher)
prompted
prompted
QLoRA, local
Per-task orchestration cost. HALO is ~45Γ cheaper than GPT-5-mini and 15β20Γ cheaper than Gemini-3-Flash β depending on benchmark family and whichever orchestrator-task cost falls in the published $0.06β$0.08 (Gemini) and $0.18β$0.22 (GPT-5-mini) range. The reduction has two complementary sources: (a) a small local forward pass costs orders of magnitude less than a frontier-LLM API call even before counting tokens, and (b) the hybrid policy resolves ~35% of decisions with no model call at all.
Per-Step Self-Correction Rate
Fraction of refinement steps that strictly improve LEA / StV validator metrics over the previous step
frontier LLM
Οhybrid
Per-step self-correction. HALO's per-step decision improves the validator's view of the PDDL 83% of the time, against 50% for the prompted baseline. Fewer bad decisions per episode means fewer wasted iterations β and is what lets HALO match frontier orchestrators with far fewer calls.
Total LLM Calls per Episode
Sum of orchestrator + agent LLM calls per problem (deterministic agents contribute zero)
LLM calls per episode. HALO cuts calls by 46β54% vs GPT-5-mini and by ~40% vs Gemini-3-Flash across the three families. Layer-1 rules eliminate the orchestrator call at ~35% of steps; the higher per-step self-correction rate also shortens trajectories by reducing wasted refinement iterations.
Across all three benchmark families HALO exceeds the GPT-5-mini prompted baseline on success rate β the same teacher whose trajectories were the supervision source β while costing ~45Γ less. Against the stronger Gemini-3-Flash baseline, HALO sits within 3 pp on PlanBench and Natural Plan, and beats it by 2.8 pp on classical planning, for 15β20Γ less cost. Per-step self-correction climbs from 50% to 83%; total LLM calls per episode drop by 40β50%.
The supervised student beats its own teacher on terminal success rate, on every family. Two things make this work. First, the verifier filter keeps only trajectories that succeeded β the student inherits the teacher's best behaviour rather than its average. Second, the Layer-1 hardcoded rules catch trivially decidable cases (cold start, syntax errors, valid plan) that the prompted teacher sometimes mishandles. The student's per-decision strategy is still bounded by what the teacher exhibits in accepted trajectories, but its terminal accuracy isn't.
Where the Wins Come From
Ablation: What Each Piece Contributes
| Configuration | What changes vs. headline | Effect |
|---|---|---|
| ΟΞΈ only (no rules) | Drop Layer 1; learned model on every step | β success rate Β· +35% queries/ep. |
| 13 agents (no expansion) | Restrict action space to baseline pool | β on PlanBench Β· β on Natural Plan + classical (vs GPT-5-mini) |
| No verifier filter | Keep all teacher trajectories, including failures | β success rate (consistent across domains) |
| Held-out domains | Train on 8 of 12 domains, test on the other 4 | Within β€8 pp of in-distribution rates |
| Qwen-2.5 / Gemma-2 | Same recipe, different base model family | Within β€2 pp of Llama-3-8B on PlanBench |
Each ablation tests one design choice. Removing the verifier filter is the most damaging β the model picks up biased (state, agent) pairs from failed trajectories. Removing the rules makes the model do more work for marginal gain. Restricting to 13 agents matters most on Natural Plan and classical planning, where the deterministic plan-repair agents earn their keep. Cross-family parity (Llama, Qwen, Gemma agreeing within 2 pp) suggests the signal is in the data, not the base model.
Three takeaways. The verifier filter is load-bearing β unfiltered teacher trajectories include rollouts that looked reasonable but ended in failure, and the unfiltered model picks up locally-plausible but globally-wrong selections. The expanded action space matters most off PlanBench, where deterministic plan-repair and PDDL-modelling agents earn their keep. Generalisation holds across model families and held-out domains: Llama, Qwen, and Gemma agree within 2 pp; held-out test rates stay within 8 pp of in-distribution.
The matching success rate at 1% of the cost says something specific: the orchestration decision did not actually require a frontier model. We were paying frontier rates not because the task was hard, but because we were using a generalist for a specialist's job. With the right supervision, an 8B model is enough.
RoboSort: HALO at Work
For the finale, the warehouse description is brand-new β a different facility with rules the model has not seen verbatim. The orchestrator must pick repair agents step by step until the validator accepts a plan. The visual below runs the same trajectory through both a prompted GPT-5-mini orchestrator (left) and HALO (right).
RoboSort, Post 7: Two Orchestrators, One Refinement Loop
Specification: "RoboSort, same warehouse as Posts 1β6. Three shelves (A, B, C), one build zone. Five pieces: L1 and L2 (legs) on Shelf A, Beam on Shelf B, Roof and Flag on Shelf C. Support chain: legs support beam, beam supports roof, roof supports flag. Gripper carries one piece at a time. Goal: 5-piece tower assembled in the build zone, every support predicate satisfied."
Same warehouse as Posts 1β6 Β· support chain: L1, L2 β Beam β Roof β Flag
Both orchestrators arrive at the same valid plan Β· here is the build zone after RoboSort executes it
Same 21-agent action space, same validator. The prompted GPT-5-mini pays for a full-state prompt at every step β including the cold start, syntax error, and terminal accept that HALO's hybrid policy resolves by rule. When the decision is genuinely ambiguous (step 3), HALO picks the right agent on the first try where the prompted baseline takes two attempts. Combined effect: fewer iterations, far fewer model calls, ~45Γ lower per-task cost, the same final plan.
Beyond the Warehouse: Who's Conducting Your Multi-Agent Framework?
Every multi-agent framework in production has an orchestrator β router (LangGraph), manager (AutoGen), crew lead (CrewAI), product manager (MetaGPT), or just "the GPT-4 we prompt with the system prompt." Same decision: which agent handles this subtask. PDDL is unusual because the verifier is watertight; most agentic systems still have some verifier β and that's enough.
Where This Leaves Us
HALO has three honest limitations β the teacher bounds the per-decision strategies the learned policy can express (terminal success can still exceed the teacher, as the surprising result above showed, but qualitatively new agent-selection patterns require a signal beyond imitation), the verifier is itself a ceiling (Fast Downward timeouts get discarded indiscriminately), and the 21-agent space doesn't cover PDDL 2.1 numeric fluents or durative actions with continuous effects. Each limitation suggests a follow-up: RLVR on top of the supervised initialiser, smarter timeout handling, an expanded action space. There's also a wider set of frontier questions β meta-learning across orchestration tasks, compositional generalisation, the RLHF parallel for verifier-based rewards, online learning loops, inference-time tree search, the eventual convergence of Paradigm 1 and Paradigm 2 β that the paper points at but does not address.
Rather than fit all of that into the back of this post, I've broken it out into a separate Epilogue β a forward-looking companion piece that sits outside the canonical seven posts of the series. If you want the research roadmap beyond what's published, that's where to go next. The remainder of this post wraps the series itself.
Series Conclusion: The Arc, In One View
Seven posts ago we started with a simple frustration. LLM agents are confident, fluent, and broken at multi-step coordination β booking the flight without the hotel, writing the function without the dependency, double-booking the oven. These are planning failures, and there is an entire subfield of AI that has spent fifty years on exactly this problem.
The series traced the arc. Post 2 introduced PDDL. Post 3 walked through fifty years of planning algorithms, ending at heuristic-search solvers that an LLM agent should be working with, not replacing. Post 4 was the reality check: PlanBench, Mystery Blocksworld, four failure modes of self-verification. Posts 5 and 6 were the recovery β LLM-Modulo, code-generated heuristics, LMPLAN, Thought of Search, NL2Plan, agentic PDDL. The pattern throughout: LLMs provide intelligence, formal tools provide guarantees.
This post closed the loop. The orchestrator's bottleneck is structural, but the same property that creates it β a verifier at every refinement step β also gives us the supervision to fix it. Verifier-filtered trajectories provide per-step labels for a single-token classifier. A hybrid policy spends compute only where it's needed. The result is a small local orchestrator that matches a frontier model at 1% of the cost.
The Series Arc
Post 1: The roadmap. Why LLM agents fail at multi-step tasks and what 50 years of planning gives us.
Post 2: PDDL, states, actions, goals. The formal language that makes "plan" a precise object.
Post 3: STRIPS β GraphPlan β SATPlan β HSP β Fast Downward β LAMA. The solvers we're integrating with.
Post 4: LLMs alone can't plan. PlanBench, Mystery Blocksworld, four failure modes.
Post 5: LLMs help planners. LLM-Modulo (12 β 82%), code-generated heuristics, LMPLAN policies. Paradigm 1 works.
Post 6: NL β PDDL β plan. NL2Plan, agentic PDDL, the orchestrator bottleneck. Paradigm 2 works on standard domains.
Post 7: HALO β train the orchestrator. Verifier-filtered supervision + hybrid policy. Beats the teacher (GPT-5-mini) on every family at ~2 orders of magnitude lower cost.
Seven posts, one thesis. LLMs alone don't plan reliably β but the combination of LLMs and formal planning tools is extraordinarily powerful, and the verifier that makes that combination reliable also gives us the supervision to make it cheap.
The Two Paradigms, Final Score
Post 1 introduced the two paradigms as the conceptual backbone of the series. After six more posts of evidence, the picture has resolved.
Where the Two Paradigms Stand
Posts 4β5. The expert writes the formal model. LLMs help the planner β generating heuristic code, candidate policies, sound search components, or candidate plans inside a verifier-checked loop.
- LLM-Modulo: 12% β 82% on Blocksworld
- CorrΓͺa heuristics: 373/720 at 1/50th the cost
- LMPLAN portfolio: 630/900 (LAMA: 557)
- ToS: 27% states vs LATS' 3.3%, at 2 calls vs 286k
Posts 6β7. The user describes the task in English. A multi-agent pipeline formalises, validates, solves, and renders the plan back into language β under a learned orchestrator.
- NL2Plan: 100% on standard, ~35% on novel
- Agentic PDDL: 100% Blocksworld, 93% Depots
- Plan optimisation: 45.8% cost reduction
- HALO: β98% cost vs GPT-5-mini, beats teacher on every family
The two paradigms are complementary, not competing. Paradigm 1 is what you reach for when an expert has written the PDDL and you need a planner-grade answer. Paradigm 2 is what you reach for when the user is the only one who knows the task. The hybrid future β Paradigm 2 generates the PDDL, Paradigm 1 solves it β is where the two threads of this series converge.
One Pattern, Repeated at Every Layer
The clearest finding across seven posts is structural, not tactical. The same pattern keeps showing up:
LLM produces a candidate. Formal tool verifies. Disagreement becomes supervision. Post 5 has it as LLM-Modulo (LLM proposes a plan, validator checks). Post 5 again has it for heuristics (LLM writes Python, the planner's outcome rates it). Post 6 has it for PDDL itself (LLM drafts the domain, the parser validates). Post 7 has it for the orchestrator (LLM teacher proposes agent sequences, the verifier filters the accepted ones into training data). The pattern is fractal: every layer of the agentic stack that has a formal verifier somewhere in the loop can use that verifier's binary signal as supervision for the LLM component at that layer.
That fractal is the operative insight, and it generalises beyond planning. Wherever a multi-agent system has a downstream check it already trusts β a test suite, a type checker, a policy engine, a compiler β the same pattern applies. Don't ask the LLM to be right; ask it to propose, and let the formal tool be the arbiter. Then mine the arbiter's accepted traces for training data when you want to drive the cost down.
Back to Post 1
Post 1 made four claims about where the field was heading. Six posts later, the evidence is in.
- "LLMs alone cannot plan reliably." Confirmed harder than Post 1 stated. PlanBench, Mystery Blocksworld, and the obfuscation kill shot in Post 4 leave no room for argument.
- "LLMs + formal planning tools are extraordinarily powerful." Post 5's numbers are decisive: 12% β 82% on Blocksworld via LLM-Modulo, near-LAMA performance on classical benchmarks via LMPLAN, two-orders-of-magnitude cost reduction via Thought of Search.
- "Paradigm 2 is the most exciting frontier." Held. Post 6 showed NL2Plan and agentic PDDL reaching 100% on standard domains; Post 7 showed the orchestrator at the centre of these pipelines is now itself trainable.
- "Orchestration is the key unsolved problem." Held, with progress. This post is one specific cut at it. Online learning, RLVR on top of the supervised initialiser, and inference-time tree search remain open.
The thesis, restated: the planning community's fifty years are the missing piece for reliable LLM agents β not because planners replace LLMs, but because their verifiers, solvers, and formal models are exactly what's needed to give LLM agents the correctness guarantees they cannot give themselves. The trained orchestrator is one specific worked-out instance. There will be many more β at every layer of the agentic stack, wherever a formal tool sits next to a language model.
Thanks for reading.
References
- Mangannavar, R., Coalson, Z., Dugar, P., & Tadepalli, P. (2026). Training the Orchestrator: A Supervised Approach to End-to-End PDDL Planning with LLM Agents. Oregon State University. Under review (introduces HALO).
- La Malfa, E. et al. (2025). End-to-end LLM-driven PDDL planning with a multi-agent refinement framework. arXiv:2512.09629.
- Gestrin, M., Zuo, N., Stein, M., & Kambhampati, S. (2024). NL2Plan: Robust LLM-Driven Planning from Minimal Text. arXiv:2405.04215.
- Armony, R. et al. (2025). Plan repair as a deterministic toolkit: variable-swap, action-reorder, redundancy-strip, plan-truncate, subplan-fill. arXiv preprint.
- Mangannavar, V. et al. (2025). GABAR: GNN-based Action Ranking for Planning. NeurIPS 2025.
- Kambhampati, S., Valmeekam, V., & Stechly, K. (2024). LLM-Modulo: An LLM-Based Framework for Planning with Formal Verification. ICML 2024. arXiv:2402.01817.
- Valmeekam, V., Marquez, M., Olmo, A., Sreedharan, S., & Kambhampati, S. (2023). PlanBench: An Extensible Benchmark for Evaluating Large Language Models on Planning. NeurIPS 2023.
- Liu, B., Jiang, Y., Zhang, X., et al. (2023). LLM+P: Empowering Large Language Models with Optimal Planning Proficiency. arXiv:2304.11477.
- Helmert, M. (2006). The Fast Downward Planning System. JAIR, 26, 191β246.
- Howey, R., Long, D., & Fox, M. (2004). VAL: Automatic Plan Validation, Continuous Effects and Mixed Initiative Planning Using PDDL. ICTAI 2004.
- Coles, A. et al. (2010). Forward-Chaining Partial-Order Planning. ICAPS 2010 (POPF).
- Dettmers, T. et al. (2023). QLoRA: Efficient Finetuning of Quantized LLMs. NeurIPS 2023.
- Schulman, J. et al. (2017). Proximal Policy Optimization Algorithms. arXiv:1707.06347.
- Ross, S., Gordon, G., & Bagnell, D. (2011). A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning. AISTATS 2011 (DAgger).
- Katz, M., Kokel, H., & Muise, C. (2025). Planning in the Era of Language Models. NeurIPS 2025 Tutorial.