The case for tracking the performance of individual developers

In the world of agile software development, it is widely considered to be a fool’s errand to attempt to track the performance of individual developers. Even though more and more data can be collected and analyzed on an individual’s actions and results, most companies choose to ignore it.

This post will make the case that more often than not, this willful blindness is holding back the potential of development teams. It’s time for leaders and managers to step up their game and embrace the promise of data-driven learning in the context of agile software development.

The virtuous cycle of a strong meritocracy

Few people have a fundamental problem with the idea that success should await those who are most deserving. It dates back a few millennia to ancient China, and it has permeated most aspects of society. The belief that anyone can succeed, as long as they work hard is a philosophy that carries a strong sense of justice and equality.

But these high-minded principles are not what catapulted this mindset to almost every corner of the world. Societies and businesses built on meritocratic values simply performed better. When individuals believe they can expect a better future if they work harder today, it kickstarts a virtuous cycle of personal self-improvement that contagiously spreads throughout the community around them.

The need for measurement

Self-improvement – or any form of progress for that matter – is necessarily a relative concept. We must know where we began in order to make the comparison. Which is why it is imperative to measure indicators of progress. Given the competitive pressures that most businesses face, it is not surprising that most industries have developed ways of measuring the performance of individuals.

In the world of agile software development, however, individual performance tracking is rarely used, and it is even wholeheartedly reviled in many corners. The underlying belief is not only that tracking individuals is unhelpful, but that it is actually counterproductive.

Arguments against tracking individuals

The main concerns of the detractors of individual performance measurement can be summarized as follows:

1. Developers could game the system

When performance evaluations are based on a few metrics, developers can find ways of modifying their work habits in order to maximize that performance metric. Those modifications often come at the expense of another behavior that is not measured, but also valuable.

2. Competition could create divisions within the team.

Modern software development is written in teams, and making performance evaluation about the individual could break the team spirit and the desire to cooperate.

3. Knowledge work is difficult to measure

Software developers are knowledge workers, and like most knowledge workers, the optimal solution is rarely known in advance. That makes performance difficult to measure objectively.
Some activities like coaching a new developer, or refactoring some code to make it reusable, could be very valuable for the company, but may not show on the performance metrics. Strictly looking at objective measures could paint a misleading picture that could lead to the wrong conclusions.

These concerns are not simply speculative, they have been voiced by smart, experienced, well-meaning managers. Nevertheless, the question is not whether these risks are real, but whether they are so intractable that companies should deprive themselves – out of principle – from the promise of data driven learning.

Promise of Data-Driven Learning

For decades, many industries and countless businesses have been using data and visualization tools to uncover meaningful trends and patterns. Simply drawing on experiences and anecdotes, it is too easy to miss the underlying trends, or even worse, spot and act on patterns that were never really there.

A particularly entertaining example was the 2011 movie, Moneyball (starring Brad Pitt). Based on a true story, the movie shows how the new manager of a major league baseball team with a low budget ignored the recommendations of experienced scouts, and based his player decisions on historical data. The resulting team went on to set records and it fundamentally changed the way the league’s teams scout for players.

Similarly, the field of medicine has made major breakthroughs by embracing the complexity of our bodies and the diseases that attempt to harm them. By compiling and analyzing the data on an individual’s diet, genes, physical activity, stress levels and other measurable influencers of health outcomes, doctors and researchers are furthering our understanding on how all these variables interact.

There are certainly journalists (and even doctors) that may oversimplify findings or overstate effects to the point where they bear little resemblance to a study’s findings. But that is hardly a good enough reason to shun the data.

Is Team Performance Enough?

An alternative to tracking individual performance is to monitor team performance. This approach is vastly more common as a method for gauging the health of a team and its members. However, a modern software development team, just like a human body, is a complex organism with many interdependent parts. Teams are composed of members with different roles and varying experience levels. There are too many steps and variables in the development of software that make “team performance” too large a unit of study to effectively identify the largest opportunities for improvement.

Breaking down the team into its individual components opens up the possibility to unearth patterns, correlations, and even causations that enable managers to foster better performance outcomes.

It is true that some of the aforementioned risks vanish or are greatly minimized by simply choosing to monitor team performance over individual performance, but this simple solution establishes an artificial ceiling over what can be learned from each team.

But before looking at the other ways managers can mitigate the unintended consequences of tracking individual developers, it would be useful to look at what can be gained with such a policy.

Potential of Individual-Level Analytics

The process of software development is one of the most data-rich processes executed by knowledge workers today. The status and activity of every participant in the process can be measured, tracked over time, and analyzed across different outcomes.

It is undoubtedly true that there are many conversations, decisions, and actions that are impossible to objectively measure, and yet contribute meaningfully to the team. But compiling and visualizing even incomplete information allows you to uncover patterns and trends that a collection of anecdotes (coupled with a fuzzy memory) simply cannot.

Here are some of the benefits of tracking individuals:

1. Faster Responsiveness and Timely Feedback:

It’s easy to underestimate story sizes or forget that one story depends on another’s completion. Closely monitoring individual workloads makes managers better equipped to quickly identify and diagnose slowdowns, blockages or bottlenecks.

Additionally, daily or weekly breakdowns of a team’s accomplishments provides managers with a wealth of material to start early conversations with individuals in their team. Whether the trends appear to be positive or negative, it’s often very helpful to ask questions earlier rather than later.

2. Better Informed Sprint Planning:

Visualizing historical data on individual team members enables managers to take a more granular approach to planning. When managers set unattainable targets for a sprint, it erodes the trust in the manager as well as the motivating effect of reaching that goal.

3. Increased Team Efficiency:

Having more visibility into an individual’s task discipline (and helping them maintain it) has the effect of increasing the trust that everyone places in their project management tools. When everyone knows that the tools they use closely mirror reality, they become more confident and autonomous in their decisions, which can drive team productivity.

4. More Objective Performance Appraisal

Despite the fascinating complexity and capacity of the human brain, it is still highly vulnerable to many biases, especially when assessing the performance of others. When these biases erroneously impact a manager’s impression of an employee, it could have pernicious effects on their career.

Using historical activity and performance to inform the assessment of individuals effectively fights favoritism and injects a healthy dose of objectivity into a process that can be heavily influenced by subconscious biases.

5. Powerful Motivator for Many:

When an individual is able to view charts and metrics that aggregate their actions over a period of time, it not only sparks their curiosity, but it establishes an unspoken challenge: can I improve?
It is true that some people will feel the increased visibility into their work as burdensome, but tracking and displaying an individual’s progress usually has a positive effect on an individual’s motivation to improve, which can directly impact their productivity.

6. Easier to Retain your top players and help the ones that need it

Even the most ardent team-players agree that they would like their careers (and salaries) to evolve over time. Therefore, managers have to make decisions about raises or promotions that can’t simply be based on how long they have been at the company. It is critical to know whether a team member has greatly improved in a short amount of time and deserves to be promoted or reclassified. Concealing or deliberately ignoring data that reflects important elements of a developer's evolution in the company is not only silly, but unfair to them.

With the right tools, these benefits are easily within the grasp of any team. Nevertheless, the real challenge becomes whether or not they can be captured while minimizing the risks mentioned at the top.

Fortunately, it is only one person that can almost entirely influence whether or not tracking individual developers will be the right decision for the company.

The Manager

Managers are the most consequential members of a team. They strongly influence the work environment, and, as a consequence, have a huge impact on team cohesion and overall morale. But whether they are loved or hated, they are often responsible for establishing the direction, setting expectations, and implementing new policies.
So let’s identify the qualities a manager must have in order to ensure that tracking of individuals will be positive for the company and the team members alike.

1. Desire to coach

The underlying motivation of the manager must be to help the team improve as opposed to finding shortcuts to the job of managing. The complexity of the software development process and the multitude of variables that could affect the quality of the resulting code make it impossible to isolate unidimensional measures of productivity. Likewise, it is important to acknowledge that data alone will never paint a full picture of the individual’s performance or contribution to their team.

Google’s Project Oxygen concluded that the best managers are great coaches. They build a strong team by focusing on the development and success of the individuals that are compose it.

Just like the world’s top athletes have coaches that measure and track every movement their athletes make, equipping good managers with a wealth and granularity of data can enable them to be more proactive in the face of difficulties and more creative in their search for solutions. And when people believe their managers are truly trying to help, they don’t try to game the system.

2. Solid Understanding of the Process

For managers to be able to interpret metrics about individual developers, they must be intimately familiar with the complexities and nuances of good software development. Understanding the consequences of developing technical debt and how it may impact future productivity and flexibility is crucial.

Like most jobs, it’s not difficult to cut corners as a way to save time. But unlike most jobs, it’s very difficult to tell upon first sight if hastily written code will cause lots of problems in the future. This is referred to as technical debt. And a lack of awareness of this speed/quality tradeoff may lead to the wrong conclusions by simply looking at the data. The corollary is that many valuable things like making the code easy to understand and reuse may not be reflected in quantitative metrics.

Additionally, many challenges developers face have no optimal solution that is well documented. In these cases, taking the time to research and test potential options is a necessary investment of time that may not be reflected in the performance metrics.

3. Appreciation for team dynamics

Agile software development is a team sport. And just like any sport, healthy teams are composed of individuals in a variety of roles with different skillsets. As such, each individual has their own particular strengths and weaknesses that cannot be reliably compared to one another.

Given that Individual trends are far more important that single point measures, managers should use analytics to compare individuals to themselves, and not directly to their peers. Nevertheless, managers shouldn’t abandon tracking overall team performance. Focusing entirely on the individual may harm the team’s overall productivity by straining team cohesion and by devaluing the positive team contributions that might not be reflected in the individual performance metrics.

Trust is a two-way street

This debate can cause strong reactions, especially from developers themselves. Many are concerned that there is an abundance of bad managers that will misuse the information. But more importantly, they argue that, as a matter of principle, they should be trusted, and without that foundation, there is a sense of loss of freedom, a lack of privacy and an erosion of trust.

Inherent in their position, however, is a lack of trust in their manager, and what they will do with that information. There is a deeply-rooted presumption that managers are inexperienced or ill-intentioned. But it would be a mistake to presume that all managers are like that. In fact, the Agile manifesto asks us to presume just the opposite (Principle 8: “Projects are built around motivated individuals, who should be trusted”). The reality is that anyone concerned about the dangers of exposing individual metrics would not make the mistakes they fear. Presuming the worst in others is unnecessarily pessimistic.

Changing Expectations

The act of managing has long been considered an art form. Decisions are often made on gut-feeling, and are difficult to objectively evaluate or critique. But with the tools we have at our disposal, this no longer has to be the case.

It is true that giving managers a deep and comprehensive view of the activities of their subordinates grants them a lot of power, but it also leaves behind the precursors of their reasoning.

Every year, we expect more and more of developers: to be at the cutting edge of technology, to have a good sense for UI, to have a solid grasp of the backend, to know what builds business value….the list goes on.

It is time we raise the expectations for managers. In the vast majority of cases, I am confident that they will rise to the occasion.