When Output Stops Being a Signal
It used to be easier to tell who the strongest engineers were on a team, often just based on output. They were the ones shipping the most, moving the fastest, and getting the most done.
AI is making that harder to do. It made it easier for everyone to produce output, even non-engineers, which means output got a lot noisier as a signal.
To be clear, I don’t think output is or has ever been our best signal. I’ve never loved using Jira metrics as a proxy for performance. I’ve always cared more about whether we shipped what we intended, when we intended to.
But AI makes it harder to ignore.
On one of my teams, an engineer systematically cleared a backlog of low-level tech debt in a single sprint. That was real, valuable work. But when I looked at the metrics alongside an engineer who spent the same sprint on one high-impact change, the numbers showed a major disparity even though the impact to the team was huge in both cases. Output has never been a great signal. AI just made that harder to pretend otherwise.
Some of the most impactful engineers I work with don’t have the highest Jira velocity. They spend time building internal tools, reviewing others’ work, and keeping the team aligned to the technical direction. Their impact shows up in everyone else’s output, not just their own. If I only looked at tickets closed, I’d have undervalued them, which means I’d be optimizing for the wrong things.
It has also become increasingly possible to have two engineers shipping similar amounts of work and operating at completely different levels. One who uses AI to move faster but validates decisions and simplifies the design. The work holds up. The other who uses AI to generate most of the implementation. It passes tests, gets approved, and ships. A few weeks later, it’s the source of follow-up work, edge cases, and confusion. On paper, they look equally productive.
I think a lot of people are focused on “AI slop” and the obviously bad work being passed off as good. I’d argue that while that is a real concern, the bigger risk isn’t even the obviously bad code. It’s the work that looks reasonable - solid PRs, passing tests, sensible structure - but work that’s built on shallow understanding.
It’s easier to skip the deep digging that used to build real intuition, and that gap shows up later in edge cases and design decisions. You can get to something that works without fully internalizing why it works.
I’ve even felt this myself. Early in my own AI usage, I once relayed an inflated estimate to product leadership based on that kind of exploration, only to realize later that most of it wasn’t relevant. It felt so convincing in the moment.
We’re all learning how to use AI. Mistakes are bound to happen. I use these tools too, but as a manager, I’m often in situations where I have to rely not only on my own ability to use them but on my direct reports’. I’m one degree removed, which puts a lot of trust in a skill we’re all still figuring out.
For a long time, output was an easy “out”. A rough proxy for capability. It was never perfect, but you could argue it sort of worked.
It works even less reliably now.
We need to avoid hyper-focusing on output and spend more time evaluating the things that are harder to measure: how engineers make decisions, whether they can explain why something is the right approach, whether they improve the system around them, not just their own output.
None of this is about AI being good or bad.
It’s about what it changed. Output is easier to produce, which means it’s less useful as a signal.
The difference between engineers hasn’t gone away. It’s just harder to see at first and more expensive to ignore later.
That means more direct conversation about decisions, more investment in code review as a teaching tool, and more weight given to the engineers whose impact multiplies others rather than just their own.
When output stops being a signal, judgment is what we need to pay closer attention to.
