Scorecards That Measure the Wrong Thing, and Why AI Raises the Stakes

Since the beginning of this century, “scorecard” has become one of the most common words in business. Departments have scorecards. Managers review them weekly. Employees are rated on them. Dashboards glow with green, yellow, and red.

But a scorecard is only as useful as what it measures. If the measures are not aligned with what actually makes the organization successful, a company can hit every target on the page and still lose ground to competitors. It ends up managing performance indicators that are not key to winning.

Artificial intelligence has raised the stakes. Anyone can now ask an AI tool to build a scorecard and receive a polished, professional-looking result in seconds. That makes scorecards easier than ever to create, and easier than ever to get wrong. When a scorecard takes seconds to produce, the hard thinking about what it should measure is the first thing to be skipped.

Where the scorecard came from

The modern business scorecard traces back to 1992, when Robert Kaplan and David Norton introduced the Balanced Scorecard in the Harvard Business Review. Their concern was that companies relied too heavily on financial results, which only show what has already happened. They proposed balancing financial measures with three other views of the business: customers, internal processes, and learning and growth.

The original idea was strategic. Over the following years, Kaplan and Norton described the scorecard as a tool for describing, communicating, and carrying out strategy. Every measure was supposed to connect to how the organization intended to win.

Somewhere along the way, many organizations kept the word and lost the idea. Scorecards became collections of whatever was easy to count, inherited from a previous manager or copied from an industry template, with little connection to the organization’s strategy.

Alignment is the whole point

The first question for any scorecard is not “What can we measure?” It is “What makes us successful, and how would we know if we were getting better at it?”

Consider three companies in the same industry. One competes on price, one on speed, and one on outstanding service. They should not be using the same scorecard. A measure that is critical for one may be irrelevant, or even harmful, for another. Yet many organizations use the same generic measures as everyone else, then wonder why they look just like everyone else.

What happens when measures and strategy drift apart

Researchers have warned about this for a long time.

In 1975, management scholar Steven Kerr published a classic article titled “On the Folly of Rewarding A, While Hoping for B.” His point was simple: people respond to what is actually measured and rewarded, not to what leaders say they want. In one of his examples, insurance claims staff were judged partly on customer complaints. Underpayments produced complaints, while overpayments rarely did, so employees learned an unofficial rule: when in doubt, pay it out. The measure quietly worked against the company’s interests.

Economists describe a related idea known as Goodhart’s law, often summarized as: when a measure becomes a target, it ceases to be a good measure. Once people are pushed to hit a number, the number can improve while the thing it was meant to represent gets worse.

The most painful modern example came in 2016. Regulators found that Wells Fargo employees, under pressure to meet aggressive sales goals, had opened more than two million accounts without customers’ approval. The bank paid $185 million in penalties, about 5,300 employees were fired, and the company eliminated sales goals for its retail bankers. The measure, products sold per customer, had become more important than the customer relationship it was supposed to reflect.

Common misalignments, department by department

You don’t need a scandal to have a misaligned scorecard. Most are simply measuring activity instead of results:

• Contact centers: Rewarding short call times while customers have to call back two or three times to get a problem solved. Or quality forms that score script wording and tone more heavily than whether the issue was resolved.
• Sales: Counting new deals while ignoring whether those customers are profitable or stay beyond the first year.
• Warehouses and distribution: Measuring how fast orders ship while mistakes, returns, and credits climb.
• Human resources: Celebrating how quickly positions are filled rather than how well the new hires perform and how long they stay.
• IT and support: Counting tickets closed instead of problems permanently fixed.

In each case the number looks good, the team works hard to hit it, and the organization drifts away from what actually creates its competitive edge.

A new risk: scorecards built in seconds

Artificial intelligence adds a new twist. Today, almost anyone on a team can ask an AI tool to build a scorecard or dashboard and get a polished result in seconds, complete with measures, targets, and color-coded charts.

That speed is useful, but it creates a real risk. An AI tool builds from patterns in what it has seen before, which usually means generic, industry-standard measures. It does not know your strategy, your customers, or what makes your organization different. The result can look complete and professional while measuring exactly the wrong things.

The danger is a shift from “here it is, and here is what it can do” to simply “here it is.” A scorecard gets delivered without anyone being able to explain why each measure was chosen, how it connects to the strategy, or what behavior it will encourage. Because it looks finished, it is less likely to be questioned.

AI can be a helpful starting point. But the thinking behind a scorecard, deciding what success means and how people will respond to being measured, still has to come from leaders who understand the business. The tool can draft the scorecard; it cannot own it.

How to build a scorecard that measures the right things

1. Start with the strategy, not the data. Write down how the organization intends to win, then choose measures that show progress toward it.
2. Ask the competitive question. For every measure, ask: “If this number improved, would we be more competitive?” If the answer is unclear, the measure may not belong.
3. Keep it short. A scorecard with thirty measures has no priorities. Focus on the few that matter most.
4. Pair measures that balance each other. Speed with accuracy, volume with quality, cost with customer satisfaction. Pairs make it harder to win one number by sacrificing another.
5. Measure outcomes, not just activity. Activity is easy to count; outcomes are what customers and owners care about.
6. Involve the people being measured. Front-line employees know how a measure can be gamed and what it misses.
7. Require the “why” behind every measure. Whether a scorecard is built by a person or an AI tool, whoever presents it should be able to explain how each measure connects to the strategy and what behavior it will drive.
8. Review the scorecard regularly. Strategies change. A measure that made sense three years ago may now point people in the wrong direction.

The bottom line

Scorecards are powerful because people pay attention to what is measured. That is also what makes them dangerous. A scorecard aligned with strategy focuses the whole organization on what makes it successful. A scorecard that has drifted away from strategy can keep everyone busy, and green, while the competitive edge slips away. And as AI makes scorecards easier than ever to produce, the discipline of asking why each measure belongs matters more than ever.

Before asking whether your team is hitting its numbers, ask a more important question: are these the right numbers?

Further reading

• Robert S. Kaplan and David P. Norton, “The Balanced Scorecard: Measures That Drive Performance,” Harvard Business Review (1992)
• Steven Kerr, “On the Folly of Rewarding A, While Hoping for B,” Academy of Management Journal (1975)
• Related: The Filter Behind the Thread, on how personal bias can shape quality scorecards