You Can Measure Software Developer Productivity
But should you?
McKinsey recently released a report about their efforts to impose productivity tracking onto software engineering teams. The article is filled with assertions that aren’t backed by facts, like a claim of a 20 percent reduction in escaped bugs and a 60-point improvement in customer satisfaction ratings, correlated to the measurement framework they’ve established but likely not caused by it. It’s clear that McKinsey sees a need here — management in companies that employ software engineers want hard data about how individual contributors are performing.
Having a number to rely on is comforting to management and feels like it’s objective. A number is tangible and actionable, and that makes the complicated, messy job of managing humans so much clearer. However, this objectification undermines trust between managers and developers and reduces their human relationship to a transaction based on the number of widgets delivered.
The need for measuring performance is certainly real, but this report places the focus in the wrong place. Performance can be measured as the outcome of what developers have built in concert with their product owner, designers, customer service representatives, and everyone else who has some input into the software. Performance shouldn’t be measured as the number of widgets built or the amount of time spent on a given task.
Dumb Ideas
The report has a plethora of assertions and assumptions that aren’t grounded in any particular set of facts. For example, the report assumes that quantifying individual developer productivity is a desirable goal and a ‘critical task.’ This seems like it’s probably based on the myth of needing to measure something in order to manage it.
A myth, you say? Yes–as much as I love data and numbers, measurements in management are really fraught. The notion that you need to measure something to improve it dates back to Lord Kelvin, but he was talking about thermodynamics and how to know and communicate something well, not about managing people. Peter Drucker often gets the credit for the pithy “you can’t manage what you can’t measure,” but he never said that and it’s wrong. W. Edwards Deming is sometimes credited with the saying, but he actually said,
“It is wrong to suppose that if you can’t measure it, you can’t manage it — a costly myth.”
Nevertheless, the fear that you can’t manage what you can’t measure persists in the modern workplace, and McKinsey plays into that quite well. They also throw some AI scare tactics in there, implying that developers will just be using ChatGPT to create their code and not really sitting in their seats and typing. Nothing is more anathema to management than the idea of someone getting paid while they aren’t toiling away at 100% efficiency. If only you can get maximum productivity out of your human beings, maybe you can improve profitability just enough to get your bonus or your promotion. But you’ll burn out your developers and eventually cost your company more money.
McKinsey goes a step further into their misunderstandings, though. Beyond assuming that you must measure every aspect of how developers produce code, they assume that typing code while in their seats is the true measure of productivity. There have been a number of studies that show that writing code is a tiny part of what developers do. Estimates for time spent writing code range from 37 minutes per day to 3.3 hours per day. At the core of each of those studies was the notion that developers are only productive when they are actively coding. I think that’s nonsense.
In my experience, software engineers spend most of their time talking to people and trying to understand their problems so they can build a repeatable process to solve that problem. This is what McKinsey calls the ‘outer loop.’
An inner loop comprises activities directly related to creating the product: coding, building, and unit testing. An outer loop comprises other tasks developers must do to push their code to production: integration, integration testing, releasing, and deployment.
McKinsey thinks that minimizing the amount of time that developers spend understanding the problem they need to solve will lead to greater productivity. This is probably the most dangerous conceit of the whole article: the notion that time spent coding==productivity==desirable outcomes. However, there’s no metric in there that measures the amount of churn that developers go through or the lines of code that get thrown away or the amount of time spent building the wrong thing.
Ultimately, this is a report for managers who have never written software written by consultants who have never written software. Use at your own risk.
Better Ideas
I’ll give McKinsey some points because they at least are trying to do something, even if that something is idiotic. There’s a real need in engineering teams to understand what’s being built, if the thing being built is valuable, and whether or not you’re moving forward quickly enough to deliver some value to customers. Badgering individual developers is not the answer, though.
Eighty-five percent of the reasons for failure are deficiencies in the systems and process rather than the employee. The role of management is to change the process rather than badgering individuals to do better. — W. Edwards Deming
Process often gets a bad rap because of the way some managers impose it on their engineers. When engineers hear that there’s a New Process coming, they sigh inwardly and resign themselves to listen politely while the manager describes how everything will be better now. If the manager is one of those Agile Zealots, they sigh outwardly and glare. But there’s always a process, whether you’ve defined one or not.
The answer to a bad process shouldn’t be chaos–it should be a better process. Software is (almost) always built by teams, and teams, by definition, are two or more draft animals harnessed to the same vehicle or implement… wait. I mean, teams are groups of people that are working toward a common goal. Anyway, teams, whether they’re pulling a farm implement or building software, need some process to understand which way to go and who goes where. That’s where management needs to step in–not to whip the poor ox who’s going slower, but to make sure that both oxen are moving at the same speed so you can get the field plowed. (Sorry I beat that metaphor to death.)
So, to get the field plowed–to get the software built, you need to measure your process, not the progress of the individuals. If you’re not making sure the team is moving together, it’s your fault as a manager, not the fault of the individuals. You aren’t ever going to find a group of disparate people that will all move at exactly the same speed, because they’re people. Measuring them as if they were motors that you can adjust to spin at exactly the same rates makes these human beings feel like a) they aren’t human beings; b) they aren’t trusted; and c) you don’t care about them as people.
Manage your team as a team. Manage your process to understand if that team is producing the outcomes that you expect and they’ve agreed to. Measure that progress and communicate those measurements back to your team. Get them to agree to continuously improve their process and those measurements. Don’t whip them and terrify them with individual measurements that will eventually get used on performance reviews to determine if they stay or go or whether they get a raise or not.
