What to Learn — A Survey of Learning Objectives
Three families of methods, each picking a different target for the model to predict. The choice of target ends up mattering more than the choice of architecture.
Part 1 ended with two design axes that every neural method in Learning for Planning has to settle: what to learn (the prediction target) and how to represent state (the input encoding). This post takes the first axis seriously and walks through the literature. Part 3 will do the same for the second axis.
Three families exist. Each makes a different bet about what a neural network can usefully predict for a planning problem, and each inherits a different set of pathologies. None of them is obviously right; all of them have been pursued for over a decade. Listing them up front:
Learn a heuristic h(s) for search guidance
The model predicts an estimate of the cost-to-goal for a state. A classical search algorithm (A*, GBFS) consumes the prediction and does the actual planning. The learner accelerates search; it does not replace it.
Learn a value function V(s); act greedily
The model predicts cost-to-goal accurately enough to be used directly. At test time the policy enumerates successor states, looks up V on each, and picks the one with the lowest value. No search tree, no backtracking.
Learn a ranking — over states, or over actions
The model predicts only relative orderings rather than absolute distances. Two variants: rank states (used inside GBFS), or rank actions directly given the current state (used as a reactive policy). The action-ranking variant is where GABAR (Post 4) lives.
The three families form a progression on a single dimension: how much absolute information the model is asked to predict. Heuristics need to be roughly right and the search engine fixes the rest. Value functions need to be globally accurate. Rankings need only be locally correct. The further down the list you go, the easier the learning target — and the more you give up in correctness guarantees that the search engine used to provide.
The rest of this post walks each family with concrete papers and what they showed. By the end you should have a clear sense of what each family can do, and the specific failure modes that drove the field toward the next family.
Family 1: Learning Heuristics for Search
The earliest neural approaches to L4P inherit directly from classical heuristic search. The architecture is unchanged: A* or greedy best-first search expands states from a priority queue, ordered by a heuristic estimate of cost-to-goal. The only difference is where the heuristic comes from. Instead of being hand-designed (as hFF and hadd were for decades), it is learned from planner-solved instances.
The bet is conservative: keep all the guarantees of classical search, just replace one component. If the learned heuristic is informative, search expands fewer nodes and runs faster; if it's not, the search algorithm degrades gracefully toward exhaustive enumeration but still finds a solution. The learner is a speedup, not a replacement.
ASNets — alternating action/proposition layers
Toyer, Thiébaux, Trevizan, Xie 2020 introduced Action Schema Networks, the first neural architecture explicitly designed for generalized planning. The key insight: a planning domain has a fixed set of action schemas (e.g., move, pick, drop) regardless of how many objects exist in any particular instance. If the network has one weight set per schema and shares those weights across all groundings of that schema, the same network can process problems of any size.
ASNets stack alternating layers: action layers (one unit per ground action, weighted by which propositions are its preconditions) and proposition layers (one unit per ground proposition, weighted by which actions affect it). After several alternations, each ground-action unit has aggregated information from a fixed-depth neighborhood of the action-proposition graph. The output is interpretable as either a heuristic or an action policy, depending on training objective.
The result was the first demonstration that a neural architecture could generalize across planning instance sizes — small training instances, larger test instances — without retraining. It established the now-standard evaluation protocol the field has used since.
The limitation that subsequent work has chipped at: the fixed alternation depth bounds the network's receptive field. Long-horizon dependencies — where a useful action's effect ripples through many propositions — exceed what a small stack can model. ASNets does well on domains with local effects (Gripper, Blocks small) but struggles where chains of reasoning matter.
Family 1 on the warehouse — learned h() guides A*
The search tree still exists. The learned heuristic just orders which node A* expands next. On a 16-zone, 3-package warehouse the tree is much bigger — the heuristic stops the explosion from being uniform but doesn't eliminate it. This is the inherent limit of Family 1: the speedup is multiplicative on top of search, not a replacement for it.
STRIPS-HGN — hypergraph networks for domain-independent heuristics
Shen, Trevizan, Thiébaux 2020 introduced STRIPS-HGN, which generalizes the message-passing idea to hypergraphs. A STRIPS action has multiple preconditions and multiple effects — naturally modeled as a hyperedge connecting many propositions, not a pairwise edge. STRIPS-HGN's network operates directly on the hypergraph induced by the planning instance.
The architecture stays domain-independent: the same model trained on one set of domains transfers to held-out domains, no retraining. The output is a heuristic plugged into A*. Compared to ASNets, STRIPS-HGN handles the multi-precondition structure more naturally and shows better cross-domain transfer, at the cost of more expensive per-instance graph construction.
The same fundamental limitation applies: STRIPS-HGN is a heuristic, so search overhead remains. The model accelerates search; it doesn't eliminate the need for it.
GOOSE — modern grounded and lifted heuristic learning
Chen, Thiébaux, Trevizan 2024 released GOOSE, the current strongest GNN-based heuristic learner for classical planning. The contribution is twofold: a careful comparison of grounded vs lifted graph representations (the second-axis question Post 3 will cover), and architectural improvements that make GNN heuristics competitive with hand-engineered ones on benchmark IPC domains.
GOOSE trains on small instances (planner-solved) and integrates as a heuristic with classical search. It is the most thoroughly engineered system in this family and represents the current ceiling for "learn the heuristic, keep classical search" methods.
Every Family-1 system inherits the computational overhead of search at execution time. The learned heuristic speeds up search; it cannot replace it. For problems where search itself is the bottleneck — when the heuristic is good but the branching factor is enormous — Family 1 hits a wall that better heuristics alone cannot break. That wall is what motivates Family 2.
Family 2: Learning Value Functions for Greedy Policies
If a learned heuristic is good enough, why search at all? Just expand the immediate successors of the current state, look up the heuristic on each, and take the action leading to the lowest one. No search tree, no priority queue, no backtracking. The policy is reactive: from any state, one forward pass picks the next action.
This is the Family 2 bet. It promises a much faster planner at test time — no search overhead — at the cost of requiring the learned function to be globally accurate, not just informative. A small ranking error in a heuristic costs a few extra node expansions; the same error in a Family 2 value function picks the wrong action and the policy might never reach the goal.
GPL — unsupervised generalized policies with GNNs
Ståhlberg, Bonet, Geffner 2022a introduced GPL (Generalized Policy Learning), which trains GNNs to predict V(s) without supervision. The training signal is bootstrapped: the model's own value estimates plus the known transition function produce target values via Bellman-style updates, and the network is trained to be consistent with these. No expert planner is needed during training.
At test time the policy enumerates successor states, runs the GNN on each, and picks the action whose successor has the lowest predicted value. The system avoids search entirely. On small instances within the training distribution, GPL matches expert plan quality.
The cracks appear on larger instances. Value learning's promise rests on global accuracy: the prediction must be roughly right not just for the current state but for every state the policy will visit before reaching the goal. For long horizons, small per-step errors compound — the policy ends up favoring the wrong action when it really matters. Generalization to larger problem sizes degrades sharply because the predicted V scale itself drifts: the absolute cost-to-goal a model learned on 5-zone warehouses doesn't map well to 50-zone warehouses where the actual costs are 10× larger.
Family 2 on the warehouse — V() lookup, no search
The greedy lookahead breaks on scale. Trained on 4-zone instances where V(s) lives in [0, 5], the network simply never produces values in the [10, 20] range that 16-zone optimal plans actually need. Successor states all look indistinguishable. There's no search engine to compensate for the misranking — the policy commits to whatever action the broken values point at.
Expressive variants — pushing the architecture
Ståhlberg, Bonet, Geffner 2022b follow up with more expressive GNN architectures for general optimal policy learning, exploring what kinds of policies neural networks of bounded expressivity can represent. The work connects to logical expressiveness results Barceló et al. 2020 that bound GNN representation power to descriptions in C2 (graded modal logic, a restricted fragment of first-order). For planning problems requiring more expressive reasoning, vanilla message-passing is provably insufficient.
Ståhlberg, Bonet, Geffner 2024 extends to "Beyond C2" architectures that incorporate higher-order features. The empirical gains are real but incremental — and crucially, the expressivity story does not resolve the global-consistency problem of value learning. A more expressive model can fit small training instances better, but value generalization to larger sizes still degrades.
Value learning trades search-overhead for global-accuracy requirements. The exchange works on instances close to the training distribution. It does not work for size generalization, because the value scale itself changes with problem size: a state two steps from goal in a 4-zone warehouse looks superficially similar to a state two steps from goal in a 40-zone warehouse, but the surrounding combinatorial structure is wildly different, and the value function has to be sensitive to that difference in ways small-instance training cannot teach it.
Family 3: Learning to Rank
If global accuracy is the obstacle, weaken the requirement. Don't ask the model to predict absolute distances; just ask it to rank options correctly. This is the Family 3 bet, and it splits into two distinct sub-families: rank states (used inside a search algorithm), or rank actions (used as a reactive policy).
State ranking inside search
Garrett, Kaelbling, Lozano-Pérez 2016 introduced the idea of learning to rank states for planning, using RankSVM over hand-crafted features. Pairs of states (one closer to goal, one further) form the training data; the learned ranker decides which to expand first in GBFS. This relaxes the requirement of learning an accurate distance — only the pairwise ordering needs to be correct.
The idea sat for several years before recent revisitation. Chrestien, Edelkamp, Komenda, Pevný 2023 sharpened it: their paper's title is "Optimize planning heuristics to rank, not to estimate cost-to-goal." They show that for guiding GBFS, the loss function should directly target ranking accuracy rather than cost-to-goal regression. Models trained this way produce strictly better search guidance than the same models trained with mean-squared-error loss against ground-truth costs.
Hao, Trevizan, Thiébaux, Ferber, Hoffmann 2024 extends this to pairwise rankings for GBFS, in two variants (one is a workshop paper, the other an IJCAI extension). They train networks to predict, given two states, which is closer to the goal. GBFS expands using the predicted pairwise ordering as the priority. The empirical result is consistent: ranking-trained heuristics beat regression-trained heuristics on the same architecture and data, often by large margins on hard instances.
These methods all still use search at test time — they're heuristics, just better-trained ones. They share Family 1's search-overhead limitation. What's new is the realization that ranking is an easier learning target than regression, and that this difference translates into measurable planning performance.
Action ranking — the policy variant
The natural next step: skip search entirely. Instead of ranking states to decide which one to expand, rank actions in the current state to decide which one to execute. This collapses the "learn a policy" Family 2 idea into a ranking framing — no absolute values, no global consistency, just "which available action looks best right now."
Several lines of work fall here. Garg, Bajpai et al. 2019 (size-independent neural transfer for RDDL) learn to score actions in RDDL planning problems. Janisch, Pevný, Lisý 2020 (SR-DRL) use GNNs with autoregressive action decomposition for symbolic relational tasks. Ståhlberg, Bonet, Geffner 2023 (policy gradients for generalized policies) train action-selecting networks with policy-gradient methods. Rivlin, Hazan, Karpas 2020 apply deep RL to generalized planning.
What unifies these methods is that they predict the next action directly given the state. They differ in training signal (supervised vs RL), in architecture (GNN vs alternating layers vs custom), and in how much action information they expose to the network.
GRAPL — the closest precursor to GABAR
Karia, Srivastava 2021 (GRAPL — Generalized Relational Action Policy Learning) deserves its own treatment because it is the most direct precursor to GABAR. GRAPL ranks actions using canonical abstractions: objects with identical properties are grouped into equivalence classes, and the network reasons about classes rather than individual objects. The output is a ranking over action parameters, used to construct a complete grounded action.
The critical limitation that distinguishes GABAR from GRAPL: GRAPL selects each action parameter independently. A multi-parameter action like transport(?pkg, ?source, ?dest) in the running warehouse is decomposed into "pick the best package," "pick the best source zone," and "pick the best destination" as separate decisions. The choice of source zone does not condition on the choice of package.
This breaks on domains where parameters are coupled. In the warehouse, the correct source zone for a transport action depends on which package was selected — it has to be the zone that package is actually in. (The same coupling shows up in IPC Logistics between packages and the vehicles that must share their city.) GRAPL has no mechanism to express this dependency; the parameter decisions are made in parallel. GABAR's GRU-based decoder (Post 4) fixes this by making parameter selection sequential: the action schema is chosen first, then parameters are picked one at a time, each conditioning on what came before. Post 3's decoding figure walks this exact example.
Family 3 on the warehouse — rank, don't estimate
The action-ranking output stays the same shape regardless of warehouse size. Same warehouse with 16 zones and 3 packages produces a longer list of candidate actions, but each one still gets a score. The network's prediction target — "which of these actions is best right here" — doesn't change scale with the problem. That's the property that lets the small-training-set network rank actions on a 50-zone warehouse without recalibration.
Ranking sidesteps the global-consistency problem of Family 2 by predicting only local orderings. The action-ranking sub-family additionally sidesteps Family 1's search overhead by predicting a policy directly. The remaining question is how to represent the input so the same ranking model works across problem sizes — which is exactly the Axis-2 question Post 3 covers.
Cross-family comparison
Putting the three families side by side along the dimensions that matter most for size generalization:
The three families at a glance
| Property | Heuristic (Family 1) | Value Function (Family 2) | Ranking (Family 3) |
|---|---|---|---|
| What the model predicts | Estimate of cost-to-goal h(s) | Accurate cost-to-goal V(s) | Pairwise / total ordering only |
| What the search engine does | Full search, guided by h | One-step lookahead, no search | State-rank: still search. Action-rank: no search |
| Search overhead at test time | Yes | No | Depends on variant |
| Requires global prediction accuracy | No (graceful degradation) | Yes | No (only local ordering) |
| Degrades gracefully on size scale-up | Yes (slower search, still correct) | No (wrong action choices) | Action variants: yes; state variants: partial |
| Representative papers | ASNets, STRIPS-HGN, GOOSE | GPL, Expressive-v1/v2 | RankSVM, Chrestien et al., GBFS-rank, GRAPL, GABAR |
The progression across families is a progressive relaxation of what the learner is responsible for. Family 1 hands almost everything to the search engine and only learns guidance. Family 2 takes everything: search disappears, but global accuracy becomes mandatory. Family 3 hits a sweet spot — no search overhead, no global accuracy requirement, just "rank what's in front of you."
The cost is that Family 3 gives up the search engine's correctness guarantee. A misranked action might lead the policy into a dead end, and there's no explicit mechanism to back out. In practice, this is handled by simple termination conditions (don't revisit states; cap execution length), but it remains a real difference from Family 1's behavior.
Back to the warehouse: three families, one state
To make the three families concrete, here is the same warehouse state — robot in A, package on C, goal to deliver to D — passed to a representative of each family. They produce different outputs and use them differently.
Same warehouse state, three predictions
All three pick the same action move(A, C) on this small instance — the warehouse is too easy for the families to disagree. The disagreements start at scale: Family 1 still works but the search tree blows up, Family 2's value scale drifts and the lookahead misranks, Family 3 keeps producing valid rankings because only the local ordering matters, and that doesn't change scale with the problem.
What this means for size generalization
The original motivation for L4P is to train on small instances (where planners are fast) and deploy on large ones (where they are not). Re-reading the three families with size generalization as the criterion:
- Family 1 generalizes acceptably because the search engine compensates for heuristic errors. Even on large instances, a slightly off heuristic still produces correct plans — just slower. The bottleneck remains search.
- Family 2 generalizes poorly. The value-scale shift between small and large instances breaks the prediction, and there's no recovery mechanism: a wrong action choice is wrong, and the next state inherits the error.
- Family 3 (action ranking) generalizes well because the prediction target — "rank the actions available right here, right now" — does not change scale with the problem. A 4-zone warehouse and a 400-zone warehouse have the same kinds of actions to rank; only the candidate set is larger.
This is the empirical story the field has been converging on for several years. Family 1 hits search-overhead walls. Family 2 hits global-consistency walls. Family 3, when paired with the right input representation, doesn't hit either.
The other axis
Every paper in this post made a choice on Axis 2 as well: how to represent the input state. Most of them use GNNs over relational structures, but the specific graph construction varies widely. ASNets uses alternating action-proposition layers. STRIPS-HGN uses hypergraphs. GPL uses object-only graphs without explicit action nodes. GRAPL uses canonically abstracted graphs. GOOSE compares grounded vs lifted constructions head-to-head.
These choices are orthogonal to the learning-objective choice. The same Family-3 ranking objective can sit on top of many different graph encodings, and the encoding choice determines whether the same architecture can read instances of different sizes. Post 3 takes this seriously: it surveys the representation axis with the same care this post gave the objective axis.
Why GABAR sits where it does
One reason to read this survey before reading the GABAR paper deep-dive in Post 4: GABAR's place in the literature only makes sense once the three families and their limitations are concrete. GABAR is a Family-3 action-ranking method. It uses an action-centric graph representation (the Axis-2 choice covered in Post 3). It adds a GRU-based decoder for sequential parameter selection (the GRAPL limitation fix). And it explicitly trains the network to produce action rankings consistent with planner-solved demonstrations.
Each of those choices is a response to a specific limitation in prior work. GABAR is not "yet another GNN for planning" — it's a recipe that picks the right cell in the two-axis design space and adds the one missing piece (sequential decoding) that prior action-ranking methods lacked.
References
- Toyer, S., Thiébaux, S., Trevizan, F., & Xie, F. (2020). ASNets: Deep Learning for Generalised Planning. Journal of Artificial Intelligence Research, 68.
- Shen, W., Trevizan, F., & Thiébaux, S. (2020). Learning Domain-Independent Planning Heuristics with Hypergraph Networks. ICAPS 2020.
- Chen, D. Z., Thiébaux, S., & Trevizan, F. (2024). Learning Domain-Independent Heuristics for Grounded and Lifted Planning (GOOSE). AAAI 2024.
- Ståhlberg, S., Bonet, B., & Geffner, H. (2022a). Learning Generalized Policies Without Supervision Using GNNs. KR 2022.
- Ståhlberg, S., Bonet, B., & Geffner, H. (2022b). Learning General Optimal Policies with Graph Neural Networks: Expressive Power, Transparency, and Limits. ICAPS 2022.
- Barceló, P., Kostylev, E., Monet, M., Pérez, J., Reutter, J., & Silva, J. P. (2020). The Logical Expressiveness of Graph Neural Networks. ICLR 2020.
- Ståhlberg, S., Bonet, B., & Geffner, H. (2024). Learning General Policies for Classical Planning Domains: Getting Beyond C₂.
- Garrett, C. R., Kaelbling, L. P., & Lozano-Pérez, T. (2016). Learning to Rank for Synthesizing Planning Heuristics. IJCAI 2016.
- Chrestien, L., Edelkamp, S., Komenda, A., & Pevný, T. (2023). Optimize Planning Heuristics to Rank, not to Estimate Cost-to-Goal. NeurIPS 2023.
- Hao, M., Trevizan, F., Thiébaux, S., Ferber, P., & Hoffmann, J. (2024). Guiding GBFS through Learned Pairwise Rankings. IJCAI 2024.
- Garg, S., Bajpai, A., & Mausam (2019). Size Independent Neural Transfer for RDDL Planning. ICAPS 2019.
- Janisch, J., Pevný, T., & Lisý, V. (2020). Symbolic Relational Deep Reinforcement Learning Based on Graph Neural Networks (SR-DRL).
- Ståhlberg, S., Bonet, B., & Geffner, H. (2023). Learning General Policies with Policy Gradient Methods. KR 2023.
- Rivlin, O., Hazan, T., & Karpas, E. (2020). Generalized Planning with Deep Reinforcement Learning.
- Karia, R., & Srivastava, S. (2021). GRAPL: Generalized Relational Action Policy Learning.
- Graph Neural Network Based Action Ranking for Planning (GABAR). NeurIPS 2025.