Walk into any large company that’s been excited about AI for a year and you’ll find the same picture, and it’s a little messy.
People have signed up for five tools that mostly do the same thing. You can see the bills. You can’t see whether any of it saves time. Three teams have built the same document summariser on three platforms, and none of them know the other two exist. Someone in operations built a tool that saves twelve people an hour a day, and it will never spread, because nobody outside that team knows it’s there.
That isn’t a failure of enthusiasm. The enthusiasm is real, and so is the value. It’s scattered across personal accounts, side projects, and tools bought by whoever ran out of patience first.
A 2025 MIT study found around 95% of company AI pilots produced no measurable effect on the bottom line. The usual conclusion is that AI is overhyped. Mine is that most companies start by building, and building is exactly how you get the scattered mess in the first place.
You can’t fix a picture you’ve never seen.
What seeing it gets you
Once every AI project is on one page, four decisions become obvious that were invisible before.
| Verdict | What it means |
|---|---|
| Scale it | It works. Give it to everyone. |
| Merge it | Five versions of the same thing become one. |
| Retire it | Nobody uses it, or you’re paying for seats nobody touches. |
| Build it | A real gap. Nothing exists yet. |
Merge usually pays for the whole exercise. Retire pays for it twice. Scale is where that operations tool finally reaches the other eleven teams.
Count what exists, don’t ask people to remember
The instinct is to send a survey. A survey gives you what people remember, what they’ll admit, and what sounds impressive to their boss. Not the real picture.
Counting gives you the real picture. Look at the actual code repositories, which AI tools built each one, and whether it reached real users or quietly stopped. Look at the seat reports from every vendor, which show whose account has sat untouched for three months. Use a survey too, but for the side projects nobody wrote down, not as your main source.
Then every verdict points at something you can show someone. Not “the demo looked good.”
Four things that decide whether it works
Don’t favour one vendor. If you only count one company’s tools, you’ll conclude all your AI happens in that company’s tools. Worse than no report, because it’s confidently wrong in a flattering direction.
Keep it in the building. You’re pointing this at your code, your spending, and your staff. None of that should leave.
Replace names carefully. Swap every name for a stand-in before analysing. But use the same stand-in for the same person every time. Scramble them differently in each file and you protect privacy while destroying the most valuable finding: the same person built three of these.
The tool suggests. People decide. A tool that retires software based on how recently someone touched the code will eventually retire the thing that quietly runs payroll.
The part that isn’t technical
Announce the survey as an amnesty, not an audit. Nothing gets taken away. Good tools get proper support.
If people think you’re confiscating their side projects, they’ll hide them, and the hidden work stays hidden, which is the one thing you were trying to fix. How you frame it matters more than what you ask.
Then do it again next quarter. A one-time audit is a document. Every quarter is knowing where you stand.
What I built
I built the counting tool as a personal project and open-sourced it. It’s called AI Census: a single file you run yourself, nothing to install alongside it. It reads your repositories, your seat reports, and your survey, swaps names for consistent stand-ins, and sorts everything into the four verdicts. It runs on your own machines and sends nothing anywhere.
Copy it, run it, keep it, change it. You don’t need me for any of that. Seeing the picture is the point.
Cheers,