The curious case of Developer Productivity

With Developers working in an Agile team, where collaboration and collective ownership are cultural traits, can we really measure developer productivity?

Measuring developer productivity is the elephant in the room for many managers responsible for performance reviews, mentor individuals, help them grow. Some managers come with their way of measuring individuals, while others try to keep it more based on their intuitions, usability, response time, etc. Whichever way you use, can we measure individual developer productivity in an Agile Team?

Manager: You were not productive last week.

Developer: Not sure; what does this mean? Could you help me understand?

Manager: I don’t see any pull request from you last week.

Developer: Yes, as I was helping others solve a critical design problem and doing some exploration work for our product.

Manager: I measure your productivity by Pull requests you raise and based on data, your productivity is going down

Developer: #?%&&*@

What is Developer Productivity anyway?

Productivity is commonly defined as a ratio between the output volume and the volume of inputs.

Some people take this meaning literally about productivity and apply this to define developer productivity as the amount of output an individual developer is producing on a day-to-day basis based on his seniority level or pay range. In contrast, others come up with metrics to identify trends about an individual, their performance and ultimately determine productivity.

This literal meaning of productivity could be relevant in the simple or complicated work environment; however, software and product development is complex work with varying input and output relationships.

Developer productivity could be defined as the impact of efforts put by a developer, where efforts could be further defined as the commitment to the mission, product, and roadmap.

Developer productivity could be defined as the impact of efforts

Many developers are working in Agile teams, where the focus is on collaboration and build over each other’s ideas, efforts to have a multiplier impact on outcome and successful delivery of Product.

Can we measure individual Developer Productivity then?

If developers collaborate and problem-solve together, can we really measure how much impact each developer has in that collaborative problem-solving?

At the same time, it will be unfair for all team members if all fingers are considered equal, as there will be no differentiation of efforts, individual contribution, and in turn, individual aspirations.

So how can we balance individuality while encouraging shared ownership, collaboration, and trust among the team?

Some managers come up with many individual output metrics to assess performance and, in turn, rate people and place them in the bell curve.

What are these matrices?

1. Number of Pull Requests
2. Some people link more pull requests to increased productivity, believing that a person is adding more value by writing more code, in turn, more pull requests.
3. Number of code review comments
4. Some managers believe that if you add more comments in code review, you are involved in developing and ensuring code quality.
5. The number of issues closed.
6. There is a perception that more defects you are closing mean you are putting more effort and are more productive.
7. The number of story points closed.
8. Story points are a variation of closed issues, where managers consider individual velocity in story points closed in a sprint.
9. Line of codes written
10. Another individual metric is how many code developers write daily, weekly, and monthly to determine how productive an individual developer is.
11. And, Many more such metrics.

The issue with Individual output measures?

“You are what You Measure.”

Suppose we measure individual output metrics; we may improve on output from developers in terms of metrics improvement. However, in a complex work environment, more output does not necessarily mean more outcomes, impact. Even in some cases it more output may mean less outcome.

Consider a case where you measure several pull requests as a measure of productivity. The developer raises too many pull requests as his initial pull request resulted in multiple defects, resulting in additional work with more pull requests. In this case, the output is multiplied; however, the outcome has been reduced. You may have left with a few unhappy customers.

Output metrics could easily tell you a false picture of paradise, where you are improving on pull requests, line of codes, ticket closed, etc., while constantly impacting outcomes negatively such as bad quality, technical debt, defects.

So What?

Suppose we can’t measure individual productivity, output. Then what? Should we maintain the status quo and keep working without any measurable way of knowing how we are doing?

It would be insane to suggest that we should not measure. If we can’t measure our development productivity and how our team members contribute to success, we will not improve and help individuals with their goals.

We should focus on measuring Development efficiency as a team, where we empower each team member for collective ownership, decision-making for continuous improvement of efficiency.

What is Development efficiency?

In science class, you may have learned that efficiency is defined as “Output Energy/Input Energy.”

Development efficiency could be defined as:

Where you set some expectations of measures with the team, which are realistic while challenging, the team improves their tool, process, way of working, and collaboration to improve their efficiency against expectation. As the team matures, the expected state would also change once they reach 100% efficiency.

But what to measure?

The question remains as to what is the measure for development efficiency, so if we can’t measure output, individual productivity, then how to measure team development efficiency.

A couple of metrics could help us measure development efficiency and work as the center stone for continuous improvement and optimization. However, there should be some ground rules for measuring development efficiency.

1. As efficiency is a current state vs. expected state, One team’s data would be entirely irrelevant for another team’s data.
2. The team is measuring against themselves and focusing on continuous improvement.
3. This is not a comparison tool among teams. However, learning could be encouraged about more efficient teams and what they are doing differently; are they have low value for the expected state, or have some learnings, which helped them improve their efficiency.

Measure for development efficiency

1. Cycle Time

If you want to measure one thing, then measure cycle time. Cycle time is defined as the total time it takes for work to flow from a particular state of the development cycle. Various cycle times could be measured for the team and reviewed to see what is causing low development efficiency.

• Story Cycle time
• Pull Request cycle time
• Deployment cycle time
• Any other cycle time, which makes sense for an individual team’s context

2. Defect

Achieving zero defect is never the goal, as it would result in an unnecessary cost of quality, which will not have any significant impact.

Defect efficiency could be measured as the Number of trivial defects divided by total defects during the measurement cycle.

3. Change Success Rate

Change Success means how many changes were successfully delivered to the user without reversing change or a defect fix.

It is a useful measure of inbuild quality and outcome in successful value creation for the user with new changes.

4. BurnDown

Burndown is another important measure, which could help you find development efficiency as a team. Are we focusing as a team to close top priority items first rather than starting work as individuals in the team?

There could be few more measures for development efficiency based on your context. However, it is important to see it as Team development efficiency while giving them autonomy for continuous improvement.

What about Individual Developer Productivity or Performance?

Not all fingers are equal. If we didn’t measure individual performance in the name of an Agile team, we would be doing a great disservice to individual aspiration, recognization of their contribution, and career growth.

While we focus on Development efficiency as a team and measure each individual on the team for that efficiency, we should also identify the right quantitative or qualitative measures to help individuals in the team understand what their role in the team’s success was and how they could reach their personal goals by identifying improvement areas.

Peer feedback and observation could be key factors in measuring individual effectiveness in team contribution. Some of the ways we can measure individual contributions on a scale are:

1. Showing leadership at critical moments
2. Challenging the status quo in team
3. Active contribution during team meetings, design discussions, Code reviews, etc
4. Strong customer advocate for the product
5. Innovative ideas for products or get things moving when issues faced

For all the managers and leaders who want to measure developer productivity. It could be a combination of

What was this long write-up?

Developer productivity is a very touchy topic for many agile teams if you are still scratching your head. While managers need to find a way to measure individual contribution to help with people’s growth, it is no go subject for agile enthusiasts.

If we measure all team members on the same scale, it would be inappropriate for many team members to work while also wanting some individuality, goals, and aspirations.

At the same time, individual performance is hard to measure when collaborative effort help in building a successful product with acceptable quality and an environment of trust.

To measure individual performance, it is critical to consider team performance. At the same time, we need to combine it with an individual contribution towards creating a high-performing team to assess developer productivity.