How Did You (Or Your Robot) Test This?

AI and engineering is a volatile mix these days. You read about how some engineering shops have gone full AI while others are just starting to implement tools. Where I work, the thing we've been struggling with is how the lines have blurred so when a PR from an engineer who isn't part of the main squad appears, it's not so much that AI was used, it's more how do they trust the engineer even understood the issue.

Let me pause and say what a PR is, in case you don't spend your days in this world. A pull request (everyone just says PR) is the bundle of code changes someone wants to add to a project. Before it gets merged in, another engineer is supposed to look it over and, ideally, actually run the thing to make sure it does what it claims. That last part, the "I ran it, it works," is the whole game.

We build an application delivery system. Other companies run their businesses on our platform, which means the code we ship isn't just ours. A bad change doesn't mean a bug report we put in a queue and fix in 48 hours. It means somebody's application is down, and somebody's customer is angry, and somebody somewhere is having a genuinely terrible afternoon. When that's the situation, trust is not a nice-to-have. Trust is the product.

AI makes it easy to produce a PR that looks completely ready. Confident, well-formatted, tidy little green checkmarks. You can review a change like that and feel like you've done your job without ever answering the only question that matters, which is whether a human actually ran this and watched it work. And when you don't fully trust the person who submitted it, you start reviewing everything from scratch anyway. Which is slow, demoralizing, and defeats the whole point of the AI helping.

So we're doing something low-tech. Every PR now has to come with an answer to one question: how did you test this? You or your agent, we don't care which, but document it in the PR. "I ran it locally and checked the output." "The agent executed the test suite and here's the result." "Honestly, I read the diff and it looks right, but I haven't run it." All of those are acceptable answers.

This isn't super fancy or cutting edge, that's for sure. No clever tool or automation scoring that can be made into fancy graphics for a LinkedIn post. You just cannot merge until you've said, in writing, how you know your PR works.

I don't think the text box builds trust on its own. What it does is make the invisible thing visible. Standards only work if you can see them. Right now the standard is "we test our work," and the way you show that is you write down how you tested it, and then your teammates can trust it, and trust is what lets us ship fast without breaking our customers' stuff.