Paolo Perrone’s survey of the 2026 open-source agent toolkit divides the stack into seven layers: orchestration, memory, tool interfaces, browser or computer use, coding agents, evaluation and observability, and inference. Its most useful conclusion is not the list of recommended projects. Tool rankings will age quickly. The durable idea is to choose each layer by the constraint that will fail first in the intended system.
This is a better starting point than selecting one framework and accepting its preferred answer at every layer.
Why a single “agent framework” is misleading
An agent system is not one replaceable component. Its layers carry different state, failure modes, and switching costs. Changing a tool protocol may alter a configuration entry. Changing orchestration can rewrite the state model, recovery behaviour, and audit trail.
The apparent convenience of an integrated framework can therefore conceal architectural coupling. A team may optimize the demo path while postponing the questions that dominate production:
- Can an interrupted run resume without repeating effects?
- Can every external action be attributed and reviewed?
- Can the model provider change without rewriting domain state?
- Does the system distinguish transient execution state from durable knowledge?
- Can evaluators reproduce the exact tool and model context of a decision?
These are contract questions, not feature-list questions.
Runtime state is not knowledge memory
The article’s distinction between runtime state and knowledge memory deserves particular attention.
Runtime state answers: where is this execution, what has already happened, and what may safely happen next? It supports checkpointing, replay, idempotency, and recovery.
Knowledge memory answers: what information should persist across executions, for whom, with what provenance, and for how long? It supports retrieval and continuity, but also introduces privacy, staleness, and contradiction problems.
Putting both into one generic “memory” store damages verification. A system may remember user preferences yet be unable to resume a failed action safely. Or it may replay a workflow accurately while retrieving stale facts as if they were current.
Select layers through their failure contracts
Before comparing tools, define what evidence each layer must provide:
| Layer | Required contract | Representative failure |
|---|---|---|
| Orchestration | explicit state transitions and recovery semantics | a retry repeats an irreversible action |
| Knowledge memory | scope, provenance, expiry, and conflict handling | stale or cross-user data influences a decision |
| Tool interface | typed inputs, permissions, effects, and error semantics | a nominally read-only tool performs a write |
| Browser or CUA | observable actions and bounded authority | visual ambiguity triggers the wrong control |
| Evaluation | versioned cases, oracles, and reproducible traces | a score improves while critical failures disappear from the sample |
| Inference | model contract, latency budget, and fallback behaviour | a provider change silently alters structured output |
The best tool for a layer is the one that satisfies the dominant contract with acceptable operational cost. Popularity and benchmark position are secondary evidence.
The seams are part of the product
Perrone observes that the layers often meet through thin seams: an import, configuration file, or HTTP call. A thin interface is helpful, but it is not automatically a strong contract. The difficult behaviour lives around the call:
- version compatibility;
- partial failure and timeout handling;
- identity and authorization propagation;
- observability across asynchronous boundaries;
- replay after side effects;
- redaction and retention of model context.
These seams need contract tests and failure injection. A production architecture should be able to replace a layer only after proving that the replacement preserves the properties the rest of the system relies on.
The practical strategy is therefore constraint-first and layer-specific. Choose the state model before the orchestration brand, the evidence model before the evaluation dashboard, and the permission boundary before the tool marketplace. The integration work is not incidental glue; it is where the system’s reliability is established or lost.