Museum of Meaningless Metrics: Tokens Spent Joins the Collection

Lines of code, story points, and pull requests already had pedestals. Tokens spent is the newest vanity metric. Here is what governed multi-agent systems should measure instead.

Software already had a museum for vanity metrics. Lines of code got a glass case. Story points got Fibonacci cards. Pull requests got a tangled branch sculpture. The newest exhibit is Tokens Spent: a tall panel of lights and a readout big enough to impress a tour group.

The caption writes itself. "Our newest exhibit."

The joke lands because everyone in the building recognizes the pattern. Impressive numbers. Weak proxies for work that matters.

What the older exhibits got wrong

• Lines of code mistook volume for progress.
• Story points turned estimation into theater.
• Pull requests rewarded motion without proving an outcome.

None of those metrics are evil. They become meaningless when they leave the job and climb onto a pedestal. Leadership starts optimizing the display instead of the work.

Why tokens spent fits right in

Token counters feel modern. They look like cost discipline. A nine-digit total feels like proof that the AI program is "real."

A free-floating token total is still vanity. It does not tell you whether a workflow finished, whether errors dropped, whether a human held the irreversible key, or whether the next agent can restart without burning the same spend again.

Spend mistaken for value is the same old exhibit with a new label.

Cost tracking that belongs in the system

In a governed multi-agent install, cost tracking is not optional. Runs should know what they spent. Operators should see LLM cost next to the job, the handoff, and the gate. That is how you keep agents useful without letting burn rate become folklore.

The difference is attachment:

• Token vanity: a museum total with no job attached
• Operating cost: spend tied to a named run, an owner, and an outcome

One is a pedestal. The other is a control.

Measure the work, not the pedestal

When we install and operate multi-agent systems, we ask for four signals before we celebrate volume:

1. Outcome. Did cycle time, error rate, or completion move for a real workflow?
2. Ownership. Who holds approve, publish, merge, deploy, and money-move keys when agents act?
3. Cost with a job. Token spend is logged against a run id, not a hallway scoreboard.
4. Evidence. Logs, handoffs, and gates that the client's team can operate next week.

That is the operating model behind multi-agent systems, installed and operated: agents do the work; humans keep the keys; metrics stay attached to the job.

How to start

Pick one recurring workflow with clear pain. Name the success measure before you count tokens. Wire cost into the run path. Put a human gate on the irreversible step. Then grow volume only when the outcome holds.

If you want help installing that operating model, explore AI Agents & Automation or start a conversation.

Common questions

Is token spend useless?

No. Token spend is useful when it is tied to a named run, a workflow owner, and an outcome. A free-floating total on a dashboard is vanity.

What should leaders measure instead?

Outcome (cycle time, error rate, completion), ownership of irreversible keys, cost attached to a job, and operating evidence (logs, handoffs, gates).

How do Casey Key systems handle cost?

Governed multi-agent installs track LLM and run cost inside the job path, with human gates on irreversible steps. Cost is an operating signal, not a pedestal.

Where should a team start?

Pick one recurring workflow, name the success measure, attach spend to that run id, and decide who approves the irreversible step before you scale volume.