In last month’s Tech Memo I ran out of room before the most important ‘don’t’ when measuring AI returns. So, here it is, along with this month’s action item and a look ahead to November.
The rule is simple: never count adoption/usage as value.
The Illusion in One Line of Data
79% of leaders name AI as a key area of investment and 7% report established ROI.[1] The chasm between the two numbers is a reporting gap rather than a value gap. As we saw last month, real returns are showing up in bounded, high-volume work. To see the reporting gap, let’s take a look at adoption dashboards.
The Easiest AI Metrics to Track Get the Most Attention
Seats deployed, weekly active users, prompts submitted, and tokens consumed dominate executive AI reporting for a couple of reasons. First of all, they are easy to collect and come built into the platforms companies buy. In fact, vendors supply these figures as part of the service, often counting the same activity that determines the customer’s bill, so companies can assemble a report with little additional effort. Additionally, as a rollout expands, rising usage gives executives a convenient story of progress, even when it doesn’t tell the entire story.
Evaluating the investment, therefore, requires connecting AI use to what happens on the job instead of how much a tool was used. Look at how long tasks take, how much correction they require, and how well the finished work meets its purpose.
Activity vs. Accomplishment
“Botsitting”—supplying context, checking answers, and correcting errors—can consume much of the time AI saves. BambooHR’s survey estimated that employees spend roughly 47 days a year working with AI, of which about 20 are spent troubleshooting and re-prompting.[2] Workday found that corrections consumed 37% of AI-saved time, with only 14% of employees consistently achieving “net-positive outcomes from AI use.”[3] Because adoption dashboards count these repeated interactions as engagement, working to fix AI’s mistakes can register as progress.
Measuring production volume introduces a similar risk: research on autonomous coding agents found that code commits increased roughly 240%, while shipped releases rose only about 30%.[4] Evaluating productivity therefore requires accounting for correction time and following the work through to delivery.
What to Count Instead
Replace each ingestion metric with a measure of the work customers benefit from: for example, cases resolved without repeat contact, software releases shipped and used, or proposals won. To understand what it costs to deliver those outcomes, calculate cost per successful task for each workflow. Add up everything spent, including the cost of reviewing AI output and correcting errors, then divide that total by the number of completed tasks a human has delivered. Because the calculation depends on both spending and the quality of the finished work, require the business owner accountable for the workflow to review and sign off on the figure.
This Month’s Action Item for Leaders
Three moves: retire, replace, review. First, retire. Take adoption metrics off the executive dashboard entirely. Keep them where they belong, in IT capacity planning, and stop circulating them as evidence of return. Second, replace. For every funded AI workflow, name one output metric, defined jointly with the business owner, and record the baseline before the tool arrives. Third, review. Stand up a quarterly forum that looks at cost per successful task, rework included. Any use case that cannot produce that number gets re-scoped or defunded. That rule alone will tell you more about your AI portfolio in one quarter than a year of usage charts.
One Other Thing
Grant Thornton found that 78% of leaders lack confidence that the work their teams or orgs do with AI could pass an independent AI governance audit within three months.[5] Once auditors and boards start asking how a claimed AI benefit was measured, an adoption dashboard stops being merely uninformative and becomes a governance problem.
Next Month at a Glance
Better metrics give leaders a clearer view of where AI is delivering value and where the workflow needs attention. Acting on that evidence can involve removing unnecessary approval steps, reassigning decision-making authority, and replacing procedures that delay delivery. These changes feature prominently among companies seeing strong results from AI: McKinsey’s August survey found that nearly 75% of AI high performers had fundamentally redesigned their workflows, compared with 25% of everyone else.[6] Next month, we’ll examine how to redesign workflows and organizational structures so that time saved on individual tasks translates into faster delivery and better results for customers.
Sources
[3] https://www.cfobrew.com/stories/2026/01/22/to-measure-the-roi-of-ai-take-workslop-into-account
[4] https://www.nber.org/papers/w35275
[6] https://www.mckinsey.com/capabilities/quantumblack/our-insights/the-state-of-ai
Meet the Author
Mike Grandinetti is an Executive Fellow at Harvard Business School. He’s a serial tech entrepreneur, board member, AI & innovation consultant, VC EIR, and award-winning professor in the practice. A former Silicon Valley engineer and McKinsey consultant, Mike has been C-Suite leader roles across 8 tech startups, resulting in 2 NASDAQ IPOs and 7 strategic exits.
He’s led senior executive workshops for Berkeley, Brown, Carnegie Mellon, Columbia, Cornell, Harvard P&ED, NYU & Oxford. He’s been a senior advisor and organizing team member for the MIT CIO Symposium for a decade.
https://www.linkedin.com/in/mikegrandinetti
www.mikegrandinetti.com