Open Questions
What HOO-POMDP and GammaZero don't yet do, where the next decade of partially-observable planning is heading, and how this all connects to the rest of AI.
The series finished where the two papers do: HOO-POMDP shows you can shrink a POMDP through hierarchy until a principled solver handles it; GammaZero shows you can learn over a belief graph and transfer to larger problems. Both are real. Both leave a lot on the table.
This epilogue is the deliberate hand-wave at the table.
Three things even the best current methods can't do
Joint reasoning across belief and action time-scales. HOO-POMDP factors object beliefs to keep tractable, but the abstraction is hand-designed (per domain). GammaZero learns the belief representation but inherits a fixed MCTS budget. Real agents need to adaptively spend belief tracking effort and search effort, trading them off. Currently no method does both well at scale.
Continuous observation spaces with high information content. A robot looking at a kitchen counter receives an image, not a one-bit observation. POMCPOW and DESPOT-α partially handle continuous observations; GammaZero and HOO-POMDP currently assume discrete or low-dimensional ones. Closing this gap connects POMDP planning to vision-language models — the largest unsolved interface in robotics.
Learning the abstraction. HOO-POMDP works because someone hand-designed the right hierarchy (objects, rooms, room-graphs). GABAR/GammaZero work because someone chose the right relational graph encoding. The next step is to learn both the abstraction and the policy from data — without sacrificing the size-generalization guarantees. Promising work uses contrastive learning over belief states, but no method yet matches the sample efficiency of a good hand-designed abstraction.
Three threads to watch
Thread 1Foundation models as priors over plans
LLMs encode an enormous amount of weak prior knowledge about how the world tends to be arranged. "Cups go in cupboards. Books go on shelves." A POMDP solver that consumes those priors as belief seeds, then refines them through search, could dramatically reduce the number of observations needed before useful planning is possible. The interface is the open question. Planning in the Era of LLMs covers the FO version of this story; the PO version is just starting.
Thread 2Diffusion-style planners over belief space
Recent work generates plans by denoising trajectories conditioned on goals. Applied to belief space, the same idea could generate belief-trajectory plans — sequences of belief states that satisfy a goal — with the planner sampling from a learned distribution rather than searching a tree. This trades MCTS's correctness guarantees for sampling speed, which is the right trade-off when MCTS is the bottleneck (and it usually is, for large POMDPs).
Thread 3Online curriculum from real deployments
GammaZero trains on small synthetic POMDPs and transfers to larger ones, but the training distribution is hand-crafted. The next step is to fold real-world deployment data back into training — a curriculum that grows with the deployment fleet. Closely related to offline RL, but with the structure of POMDPs explicit in the loss. Operationally hard. Methodologically wide open.
Where this connects to the rest of AI
The narrow story of these four posts is "POMDPs are hard; here are two strategies that scale them." The broader story is that partial observability is the central technical obstacle to deploying capable agents in the physical world. The robots that exist today work around it — with engineered sensor suites, with closed environments, with humans in the loop. Real autonomy requires solving it, not engineering around it.
That's true for household robotics. It's true for autonomous driving. It's increasingly true for software agents (which have the digital equivalent of partial observability: a tool returns a result, but the underlying state of the external system is mostly hidden). The agents getting most of the spotlight in 2025-2026 — LLM-driven software agents — are not POMDP-native, and they pay for it in ways the field is still discovering.
This series is therefore not just about classical POMDP solving. It's about the algorithmic primitives that will underlie the next generation of agents that act under uncertainty — whether those agents are robots, software, or hybrids. The two paper anchors (HOO-POMDP, GammaZero) are concrete steps in that direction.
Where to go next from here
- If you want the FO version: Learning for Planning. Same recipe, easier setting. GammaZero (Post 4 here) is GABAR (Post 4 there) ported to belief states.
- If you want the LLM-agent angle: Planning in the Era of LLMs. How modern LLM agents fail at planning, and the modern toolkit (PDDL-as-action-space, hybrid policies) for fixing them.
- If you want the papers cold: HOO-POMDP and GammaZero, both readable standalone.
One last note
If you read both this series and the sibling Learning for Planning series end-to-end, you watched the same warehouse get a four-fold treatment: tractable / intractable / abstracted / learned, then again with fog: tractable-becomes-impossible / online-tree-search-buys-time / abstracted / learned. The fact that the same toy domain carries that much weight is not because the warehouse is special. It's because the underlying structural questions — how does the planner represent state? how does it scale? how does it handle partial information? — are the same questions whether you're delivering packages, rearranging a house, or guiding a software agent through an unfamiliar API.
Those questions are the work. Everything else is engineering.