Monthly blog by senior staff on public policy + updates.
Enter your email address.
Evidence-based decision making is best when data are properly used, when good judgment hasn’t been trumped by bad numbers, or good numbers twisted to support an inappropriate conclusion. Just collecting data, tying it to outcomes and using it to make comparisons isn’t enough. Numbers must be accurate and they need context. Data are essential to great leaders, but numbers don’t tell the whole story.
Let’s say I wanted to invest in a technology company. Being a devoted iPhone user, I am predisposed to liking Apple products. With a few clicks of my mouse I find Apple has earnings per share (EPS) of $13.87. Is this good? I quickly search for Google and learn its EPS is at $8.75. Should I be convinced that Apple’s numbers make it a better investment?
As a good consumer I should ask: Are the metrics accurate? EPS is a well-known measure of corporate profitability and my source was reputable. Are the metrics comparable? In this case, yes, but only after I checked that the EPS numbers covered the same time period. What do the metrics tell me? They give me insight into Apple’s and Google’s profitability.
Clearly, that is not enough. I’d like to know whether their EPS numbers met Wall Street expectations. I need to track historical performance; read some forecasts; learn about planned products and global expansion, etc. EPS is a good metric, but it only starts me on the path to making an informed decision.
In my day job, I often compare the performance of one local government against another. Using CGR’s “Govistics” tool, I discover that the City of Rochester spends $376 per capita on police services, Syracuse spends $331 and Buffalo $291. So do I vote in the next mayoral election for someone who is committed to bringing police costs in line with our peer upstate communities? Or, conversely, do I vote for the candidate who promises to maintain the city’s strong commitment to safe streets?
Again, are the metrics accurate? The data are the most up to date (2009) that let you compare communities across the U.S. The data source is also well documented and reliable. Are the metrics comparable? The site says the data are from the same source and calculated the same way. What do the data tell us? They’re about total costs relative to the population.
But again, this just starts me down a path. I now want to know more about the demographics of each city; study their crime rates, policing strategies (what is driving the number of police officers?), or the average age of the workforce. Bottom line: Benchmarking with data is what starts good investigations and helps direct you to where to dig deeper.
Benchmarks are useful as conversation starters because they:
- Point out differences and similarities;
- Provide context for fiscal/operational goals; and
- Provide accountability for performance.
Benchmarks cannot:
- Definitively identify what is right or wrong;
- Identify appropriate fiscal policy or staffing levels; or
- Tell the whole story.
In today’s sound bite society, we are bombarded by information. Benchmarking can help you figure out what’s noise and what’s not. But, great leaders use data and benchmarks to start conversations, not finish them.