TheMcNamaraFallacyandtheAIUsageOKRTrap

Something strange is happening in boardrooms right now.

Companies that spent the last two years announcing AI transformation are now announcing AI metrics. Which sounds like progress. It isn't.

The OKRs look reasonable enough at first glance. Hours saved through AI assistance. Percentage of employees actively using approved tools. Number of prompts submitted per month. Reduction in time-to-first-draft. In some organisations, there are dashboards. In others, quarterly targets. In a few, performance reviews now include a line about AI adoption.

All of it measurable. All of it, in some important sense, beside the point.

Robert McNamara was the US Secretary of Defense during Vietnam and, before that, a celebrated systems analyst at Ford. He believed that almost anything worth understanding could be quantified. Under his direction, the military accumulated extraordinary volumes of data: body counts, sortie numbers, kill ratios. Progress, measured with considerable precision.

The problem was not the measurement. The problem was what got left out of it.

The philosophical trap that now bears his name has a simple structure.

• Step one: measure what can easily be measured.
• Step two: disregard what cannot be measured easily.
• Step three: assume that what cannot be measured is not important.
• Step four: assume that what is being measured is actually the thing that matters.

Each step feels reasonable. Together, they reliably produce confident conclusions about the wrong things.

Here is what AI usage OKRs are measuring: activity. Specifically, the visible, countable surface of activity — tool logins, prompts submitted, features touched. These are not bad things to know about. They are just not the thing that matters.

Here is what they are not measuring: whether any of it changed how decisions get made. Whether the quality of thinking inside the organisation has improved. Whether people are asking better questions, or just getting faster answers to the same old ones.

The distinction matters more than most organisations are currently willing to admit.

There is a version of AI adoption where everyone uses the tools constantly and nothing structurally changes. Teams write faster emails. Decks get produced more quickly. Summaries appear where there used to be none. Usage metrics climb. Strategy remains exactly as confused as it was before. The measurement system reports transformation. The organisation experiences optimisation.

This is not a hypothetical. It is what is happening in most places right now.

The deeper issue is that the McNamara Fallacy does its most damage not when organisations are cynical, but when they are sincere. Nobody is deliberately choosing the wrong measurement. They are choosing the available measurement — and then, gradually, beginning to confuse availability with validity.

Goodhart's Law is waiting at the end of that road. When the measure becomes the target, it ceases to be a good measure. The moment AI usage becomes a performance objective, people stop using AI because it helps them think and start using it because it produces the number. The tool survives. The purpose evaporates. The dashboard looks great.

The tell is always the same: ask what question the metric is actually answering, and then ask whether that question is the one you need to answer.

> "How many employees logged into the AI tool this week?" answers a distribution and access question. It does not answer the only question that matters: are people making better decisions?

That question is harder to measure. It is also the only one worth asking.

This is not an argument against measurement. It is an argument against the particular comfort of measuring what is easy and calling it rigour.

The organisations that will extract durable advantage from AI are not the ones with the highest usage scores. They are the ones where something has changed upstream of the output — where the nature of the question being asked has shifted, where people are applying judgement to things that previously ran on habit, where the executive team is making decisions differently rather than making the same decisions faster.

None of that appears in a usage dashboard. Most of it is not quantifiable in any straightforward sense. Which is precisely why it will be systematically undervalued by every organisation that has substituted an AI adoption metric for an AI adoption strategy.

There is a version of this problem that is almost poignant. Most of the people driving AI OKRs are not confused about what matters. In private, they will tell you that the usage numbers are a proxy — a rough signal of engagement while the organisation builds capability. They know the proxy is imperfect. They are using it anyway, because the alternative is telling their board that the most important effects of AI adoption are the ones they cannot currently measure.

That is a difficult conversation to have. It is also the honest one.

The McNamara lesson was not that measurement is bad. It was that the substitution of measurable proxies for unmeasurable realities eventually becomes indistinguishable from the thing itself. People stop asking whether the proxy is still pointing at the right thing. The proxy becomes the target. The target becomes the strategy.

Vietnam had excellent data. It had terrible strategy. The data did not cause the strategy to fail. But the confidence the data created made it harder to see the failure clearly until it was too late to change course.

The practical question, then, is not whether to measure AI adoption. It is what to measure.

The most honest leading indicator is not usage. It is the nature of the questions being asked. Are people using AI to execute the same thinking faster? Or are they using it to think about problems they previously could not afford the time to think about? The first is optimisation. The second is something more interesting.

A slightly harder but more meaningful question: has anything changed about how decisions are made? Not how fast they are made. Not how much documentation accompanies them. How they are made — the quality of the reasoning, the range of options considered, the assumptions that get challenged rather than confirmed.

That question cannot be answered with a dashboard. It requires the kind of judgement that organisations instinctively try to replace with metrics. Which is, not coincidentally, exactly the kind of judgement that AI cannot replace — and that compounds in value as AI makes everything else cheaper.

The organisations measuring AI usage are, in the most literal sense, measuring the wrong thing. They are counting the tool logins while the real question — what has changed about how your people think? — goes unanswered, because it is not answerable in the format a quarterly review requires.

McNamara would have recognised the dynamic immediately. He would probably have approved of the dashboards.

That should give us pause.

Part of The Interpretation Project. The question this piece leaves open: what would a genuinely useful AI adoption metric actually look like? That is the harder problem — and the one worth working on.