Measure AI use without spying on people.
A few weeks after an organization turns on an AI tool for everyone, someone discovers the admin console. It lists who logged in, how often, and how many messages each person sent. The next step feels obvious: sort by name, find the people at the bottom, and ask what's going on. That instinct is understandable, and it usually does more harm than the low numbers it was meant to fix.
Message counts say very little about the work.
Usage dashboards measure activity because activity is easy to count. A person can send two hundred prompts a week rewriting the same kind of email, one at a time. Another person might spend an afternoon building a single intake template that the whole front desk now uses, and then barely open the tool again. On the leaderboard, the first person looks like a model employee and the second looks like a holdout.
Heavy use can also point to trouble. Someone who keeps re-asking the same question may be fighting a tool that doesn't fit the task, or pasting in things that shouldn't leave the building. The count alone can't tell you which. What you want to know is whether the work is changing, and that rarely shows up in a column of numbers next to people's names.
Quiet scoring changes how people behave.
Once staff suspect they are being ranked, they start working for the ranking. Some will pad their numbers with throwaway prompts. Others will move their real AI use to a personal account on their phone, where nobody can see it and none of your data rules apply. Either way you end up with worse data and more risk than you had before you looked.
There is a fairness problem too. Judging someone on a measure they have never seen, against an expectation nobody stated, gives them no chance to meet it. People notice when a manager seems to know something about their habits that they were never told was being tracked, and they remember it long after the AI rollout is forgotten.
Decide what question the data is for.
Before anyone opens the dashboard, write down what you are trying to learn. In most organizations it comes down to a short list: whether the seats you pay for are being used enough to renew, which teams are stuck and need help, and where the work itself is starting to change. None of those questions needs a ranking of individuals.
Most of them are answered better at the team level. If the billing office barely uses the tool while the program staff use it daily, that is useful, and it points to something you can act on, like a missing example or a manager who hasn't tried it. Set a floor for group size, say no report on any group smaller than five, so a team-level number can't be read as one person's. Leave the content of prompts alone. Reading what people typed is a security matter for the rare investigation your policy already covers, and it should stay there.
If you wouldn't be comfortable showing a person the report you are reading about them, you shouldn't be reading it.
Tell people exactly what you look at.
Write a short note to staff and keep it current. It should say what the admin view shows, who in the organization can see it, what you use it for and what you will never use it for. A town office or a school district can do this in half a page. Most people relax once it is in writing, since the worry was mostly about what they couldn't see.
If you do look at an individual's data, show it to that person first, and do it in a one-on-one, never in a meeting. Give them the context with it: what the numbers include, what they leave out, and what you were hoping to see. A label like "light user" with no explanation reads as a verdict. The same information, walked through together, gives the two of you something to work on.
Use low and high numbers to open a conversation.
Low use usually has an ordinary cause. The tool may not connect to the system the person spends their day in. They may have tried it once on the wrong task, got a bad answer and decided it wasn't for them. Or nobody above them has ever mentioned it, so it reads as optional. Each of these has a fix, and none of them calls for a warning.
High use deserves a question as well. Ask what it is producing and who relies on it. Sometimes the answer is a set of prompts that the rest of the team should be using too. Sometimes it is a scheduled job that runs all day and that nobody has read in weeks. You only find out by asking, and people answer honestly when they know the question isn't a trap.
A one-page usage charter.
Put the rules in writing before you report anything, and share the page with everyone it covers. Fill in each line in plain language.
- The question. What you are trying to learn from usage data, in one or two sentences.
- The data. Which numbers you collect (logins, active days, features used) and which you never look at, starting with prompt content.
- The level. The smallest group you will report on, and the rule that stops a team number from pointing at one person.
- The audience. Who can see the dashboard and who receives the reports, by role.
- The limits. What the data will never be used for, such as discipline, rankings or pay decisions made on usage alone.
- The individual view. How a person can see their own numbers, and the promise that they see them before anyone discusses them.
- The review date. When you will check the charter against what you actually did, and who signs off on changes.
Review it on that date, and if you found yourself wanting to break one of its rules in the meantime, change the page openly before you change what you do.
