AWS Builder Center

Stop Rating Engineers by Commits and Pull Requests: Measure Outcomes Across Levels

A dashboard that ranks engineers by commit count does not measure productivity; it measures how easy a number was to collect. This article argues that single activity metrics — commits, merged pull requests, lines of code, hours, or editor time — are invalid as individual performance ratings, and that the defensible alternative is a multi-dimensional, outcome-oriented set of measures applied across the individual, team, and system levels rather than one "metric that matters."

Series: developer-tools (8 articles)

  1. …
  2. 3
    Stop Rating Engineers by Commits and Pull Requests: Measure Outcomes Across Levels This article

  1. The argument
The temptation is understandable. Activity data is already in version control and issue trackers, so it is cheap to aggregate into a leaderboard. The problem is that the number describes input, not value, and it breaks in two distinct ways: it is gameable, and it is blind to the work that matters most.
When a measure becomes a target, it stops being a good measure. Industry analyses of git analytics describe the predictable response: developers split pull requests artificially, inflate story points, and optimize the number rather than the underlying value [4]. The same dashboards cannot see mentoring, architectural guidance, brainstorming, or the time spent stopping a bad design — activities the peer-reviewed SPACE research explicitly lists as essential yet intractable to measure [1]. Rewarding what is visible quietly punishes the people doing the invisible work.
  1. What the peer-reviewed research actually says
The SPACE framework, published in Communications of the ACM in 2021, was built to dismantle the "one metric that matters" assumption. Its central claim is blunt: productivity cannot be reduced to a single dimension or metric [1]. It organizes measurement across five dimensions that sit in tension with one another:
  • Satisfaction and well-being
  • Performance (outcomes, not output)
  • Activity
  • Communication and collaboration
  • Efficiency and flow
Two distinctions matter for engineering managers. First, performance should be evaluated as outcomes — reliability, service health, customer impact — rather than output, because a large volume of code is not the same as correct or valuable code [1]. Second, activity is not worthless; the authors treat it as a useful but limited "way-point" that must never be used in isolation to judge an individual or team [1]. Practical guides to the framework recommend covering at least three dimensions and reading metrics at the individual, team, and system levels rather than collapsing everything onto one person [6].
A concrete example from the same paper shows why a single signal misleads. During the shift to mandatory remote work, some activity measures — commits and speed to merge pull requests — went up, while qualitative data showed well-being falling and some engineers approaching burnout [1]. A commit-only dashboard would have declared the situation an unqualified success.
  1. System-level results can diverge from individual activity
Individual activity rising tells you nothing about whether the system delivers better. The DORA program's 2024 report, published by Google Cloud, finds that AI adoption significantly increases individual productivity, flow, and job satisfaction — yet negatively affects software delivery stability and throughput [2]. That is the precise failure mode an activity leaderboard cannot detect: people feel and look more productive while the team's delivery outcomes worsen.
DORA measures delivery rather than personal effort, using throughput and stability together. The current set covers deployment frequency, change lead time, and failed-deployment recovery time on the throughput side, and change fail rate plus deployment rework rate (added in 2024) on the stability side [5]. The framework is explicitly team- and system-level; these measures describe how quickly and safely a unit ships changes, and they are not designed to rank the individuals inside it.
A complementary peer-reviewed model, DevEx (2023), narrows the experience to three friction-focused dimensions — feedback loops, cognitive load, and flow state — and reports that organizations deploying more frequently with shorter lead times are about twice as likely to exceed their performance goals [3]. The "twice as likely" figure is a correlational finding from survey-based research, not a randomized effect, so it should be read as an association rather than a guaranteed causal lever.
  1. A metric selection framework
If you are tempted to count…
How it fails
Better measure
Source dimension
Commits per engineer
Splits into trivial commits; gameable
Reliability and impact of shipped work
SPACE Performance
Merged PR count
Rewards volume over review quality
Quality and speed of review; flow of work
SPACE Activity + Efficiency
Lines of code
Punishes concise, deleted, or reused code
Customer adoption, retention, service health
SPACE Performance
Hours / editor time
Confuses presence with progress; blind to collaboration
Perceived flow, interruptions, handoffs
SPACE Efficiency & flow
Deployment frequency alone
Can hide instability
Pair with change fail rate and rework rate
DORA throughput + stability
Team velocity alone
Misses burnout and cognitive load
Feedback loops, cognitive load, flow state
DevEx
The selection rule that follows is simple. Pick measures across at least three SPACE dimensions, keep them at the level where they are valid (system metrics for the system, not the person), and never use a single activity count to make a hiring, promotion, or compensation decision about an individual.
  1. The counter-argument and the limits
Activity metrics are not fraud; they are useful signals when kept in context and combined with other dimensions, which is exactly what SPACE does with its Activity category [1]. Removing all lightweight telemetry would also leave managers with nothing but anecdote. The evidence base has its own boundaries: DORA and DevEx draw largely on large surveys and correlation, not controlled experiments, and the industry gaming examples come from vendor blogs rather than peer-reviewed studies. Measurement also carries a cost — too many instruments change behavior and create the distrust the frameworks are meant to avoid. Measure enough to understand the system, not enough to surveil the people in it.


Next steps: audit your current engineering dashboard and remove any single activity metric used to rate an individual. Replace it with a small set spanning at least three SPACE dimensions, keep DORA throughput and stability metrics at the team level, and add one perceptual measure on well-being or flow. Review the combination quarterly and watch for divergence between individual activity and system delivery — that gap is where the real story lives.

References

  1. Forsgren, N. et al., The SPACE of Developer Productivity, Communications of the ACM, 2021 —  https://cacm.acm.org/practice/the-space-of-developer-productivity/ 
  1. DORA / Google Cloud, Accelerate State of DevOps 2024 —  https://dora.dev/research/2024/dora-report/ 
  1. Storey, M. et al., DevEx: What Actually Drives Productivity, ACM, 2023 —  https://dl.acm.org/doi/fullHtml/10.1145/3595878 
  1. Keypup, Goodhart's Law in Action: Why Your Dev Metrics Are Being Gamed (industry analysis) —  https://www.keypup.io/blog/goodharts-law-in-action-why-your-dev-metrics-are-being-gamed-and-how-to-fix-it/ 
  1. Spacelift, DevOps Metrics: KPIs Including the Five DORA Metrics (secondary reference) —  https://spacelift.io/blog/devops-metrics 
  1. The SPACE Framework: A Complete Guide (practical guide) —  https://space-framework.com/ 

Series: developer-tools (8 articles)

  1. …
  2. 3
    Stop Rating Engineers by Commits and Pull Requests: Measure Outcomes Across Levels This article
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article