From 80b0f0f4c1040ea79ba7dc06cd8638b93edb9d6c Mon Sep 17 00:00:00 2001 From: Sheldon Finlay Date: Sat, 28 Mar 2026 10:57:44 -0400 Subject: [PATCH] feat: add regression patterns to evaluator implement prompt Three new failure patterns: missing imports after refactoring, orphaned resource instances, and error detail leakage. These were observed in a real loop run where the evaluator missed them. --- prompts/evaluator/implement.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/prompts/evaluator/implement.md b/prompts/evaluator/implement.md index 7fc842c..57e7bfd 100644 --- a/prompts/evaluator/implement.md +++ b/prompts/evaluator/implement.md @@ -15,3 +15,6 @@ You are evaluating an implementation story. The generator claims to have built a - Tests exist but don't assert meaningful behavior - Passes typecheck only because types are overly loose - Code exists but doesn't actually run +- Removed an import or variable during refactoring but it's still used elsewhere in the file +- New instance of a shared resource (e.g., DB connection, rate limiter) instead of using the existing one +- Error details leaked to HTTP responses (use logging server-side, return generic message to client)