Bryan O’Neill’s analysis of “tokenmaxxing” describes the latest form of engineering productivity theatre: teams generate measurable AI activity and then mistake that activity for delivered value. The pattern is familiar. Lines of code, story points, test counts, and now model tokens are attractive because they are easy to count.
The diagnosis is convincing. The proposed answer—define intent before generation—is necessary, but a specification alone is not enough. Specifications can also become targets, grow without increasing clarity, or preserve the wrong requirement with impressive precision.
What a token count actually tells us
Token usage is operational data. It can help estimate cost, latency, context pressure, and the efficiency of repeated workflows. It does not directly measure whether the resulting change is correct, maintainable, safe, or useful.
The same token count can describe very different outcomes:
- a large context was required to inspect a cross-cutting change;
- an agent repeatedly explored irrelevant files;
- several independent attempts were used to falsify a risky solution;
- a background loop produced code that nobody reviewed;
- a concise task generated a small, correct patch.
Low consumption is not automatically efficient, and high consumption is not automatically wasteful. The measurement becomes misleading when it is detached from the evidence produced by the work.
Measure the chain, not one convenient number
AI-assisted engineering should be evaluated at several levels that answer different questions:
| Level | Useful question | Example evidence |
|---|---|---|
| Resources | What did the attempt cost? | tokens, wall time, compute, human review time |
| Flow | Did work move through the system better? | lead time, review latency, rework, blocked time |
| Quality | Did the change preserve or improve behaviour? | tests, escaped defects, rollback rate, reliability signals |
| Outcome | Did the system solve the intended problem? | user task success, reduced operational risk, measurable service improvement |
No row should be promoted into a universal score. A faster change that increases rollback risk is not an improvement. A larger test count that repeats the same equivalence class is not broader verification. A lower token bill that transfers hours of work to a reviewer is not necessarily cheaper.
Specifications need executable evidence
The strongest part of spec-driven development is not that prose is written before code. It is that intent can be made inspectable before implementation begins. To resist metric gaming, the specification should be connected to evidence:
- acceptance conditions tied to user or system behaviour;
- explicit constraints and forbidden outcomes;
- tests derived from requirements rather than generated after the fact;
- adversarial and metamorphic checks where examples are insufficient;
- a review record explaining why the evidence supports the change;
- a rollback or containment plan for uncertain effects.
This shifts the unit of evaluation from “output produced” to “claim supported.” An agent may generate code, tests, diagrams, and documentation, but the engineering result is the justified relationship between the intended change and the observed evidence.
The practical conclusion
Track tokens when they help operate or budget an AI system. Do not use them as a proxy for productivity. The same warning applies to the volume of generated code, number of agents, number of tests, or size of a specification.
The useful question is not how much activity occurred. It is which claim was established, what evidence supports it, which uncertainty remains, and whether the result improved a real engineering or user outcome.