While obviously a LLM might miss functional problems, it feels extremely well suited for catching “stupid mistakes”.
I don’t think anyone is advocating for LLMs merging and approving PRs on their own, they can certainly provide value to the human reviewer.
"Computer already looked at it so I only need to glance at it"
No.
Linters are extremely well suited for catching stupid mistakes.
LLMs are extremely well suited for the appearance of catching stupid mistakes.
Linters will catch things like this because they can go through checking and evaluating things logically:
> if (
> isValid(primaryValue, "strict") || isValid(secondaryValue, "strict") ||
> isValid(primaryValue, "loose" || isValid(secondaryValue, "loose"))
> //...............................^^^^ Did we forget a closing ')'?
> ) {
> ...
> }
LLMs will only highlight exact problems they've seen before, miss other problems that linters would immediately find, and hallucinate new problems altogether.
AI for example can catch the fact that `phone.match(/\d{10}/)` might break because of spaces, while a linter has no concept of a correct "regex" as long as it matches the regex syntax.
I don't think anyone is arguing that replacing linters with AI is the answer, instead a combination of both is useful.
Especially helpful if you're a team with where there's a wide variance in competency/experience levels.
Still won't be perfect, but you'll definitely get to a point where it's a net positive overall - especially with frontier models.
Is this just a non interactive version of this kind of agent?
1. You use Aider to complete a novel task you're actively working on. Patchwork completes repetitive tasks passively without bothering you. For e.g. updating a function v/s fixing linting errors.
2. Aider is agentic, so it figures out how to do a task itself. This trades accuracy in favor of flexibility. With patchwork, you control exactly how the task is done by defining a patchflow. This limits the set of tasks to those that you have pre-defined but gives much higher accuracy for those tasks.
While the demo shows CLI use, the ideal use case patchwork is as part of your CI or even a serverless deployment triggered via event webhooks. Hope this helps? :)
Do you accept PRs yourself :-)
It's a common noun which works really well for patch-based offerings I guess, and we chose it because we built a 'framework to patch code'. But we'll think more about this - thanks for bringing it up.
When I saw your submission title I thought it was that Patchwork.