{"url": "https://sub.xiliary.com/p/brendancodykenny/posts/14/an-llm-writing-code-is-as-good-a-software-engineer-as-the", "author": {"handle": "brendancodykenny", "name": "Brendan Cody-Kenny", "headline": "Technical Due Diligence for Software M&A", "url": "https://sub.xiliary.com/p/brendancodykenny"}, "post": {"id": 14, "url": "https://sub.xiliary.com/p/brendancodykenny/posts/14/an-llm-writing-code-is-as-good-a-software-engineer-as-the", "kind": "update", "summary": "An LLM writing code is as good a software engineer as the test suite it runs against. SBSE found this in 2009.", "details": ["GenProg (Weimer, Nguyen, Le Goues, Forrest, ICSE 2009) repaired real bugs by mutating a program and keeping the variants that passed the tests.", "Qi, Long, Achour and Rinard (ISSTA 2015) showed most accepted patches passed by deleting the functionality the failing test exercised.", "Smith, Barr, Le Goues and Brun (FSE 2015) named it overfitting to the test suite and measured it getting worse as the suite got thinner.", "A coding agent is the same loop with a better mutation operator. It proposes, the tests judge, it keeps what passes. The judge did not change.", "The amount an agent can safely do to a codebase is set by the suite. A team that wants more from the agent gets it by writing tests."], "links": ["https://doi.org/10.1109/ICSE.2009.5070536", "https://doi.org/10.1145/2771783.2771791", "https://doi.org/10.1145/2786805.2786825", "https://doi.org/10.1016/S0950-5849(01)00189-6", "https://doi.org/10.1109/TEVC.2017.2693219"], "tags": ["ai-coding", "genetic-improvement", "sbse", "testing"], "happened_on": null, "created": "2026-09-29T09:40:01.302127Z", "edited": "2026-09-29T09:45:54.655081Z"}}