fix: tighten skill specs — exact prd.json schema, explicit scaffold, validation
This commit is contained in:
@@ -1,43 +1,47 @@
|
||||
---
|
||||
name: run
|
||||
description: Execute the generator-evaluator loop interactively inside Claude Code. Dispatches subagents with full visibility and intervention capability. Run /agent-loop:init and /agent-loop:plan first.
|
||||
description: Execute the generator-evaluator loop interactively inside Claude Code. Dispatches subagents with full visibility. Run /agent-loop:init and /agent-loop:plan first.
|
||||
---
|
||||
|
||||
# /run — Execute Agent Loop Inside Claude Code
|
||||
|
||||
Run the generator-evaluator loop natively in Claude Code using subagents. Unlike `loop.sh` (headless), this gives you full visibility into each agent's work and the ability to intervene at any point.
|
||||
Run the generator-evaluator loop natively in Claude Code. You see every tool call and can intervene at any point.
|
||||
|
||||
## Usage
|
||||
|
||||
```
|
||||
/agent-loop:run # Run until all stories pass or max iterations
|
||||
/agent-loop:run 3 # Run at most 3 iterations
|
||||
/agent-loop:run --skip-eval # Skip evaluator (generator marks stories done)
|
||||
/agent-loop:run --skip-eval # Skip evaluator pass
|
||||
/agent-loop:run --story US-003 # Run only a specific story
|
||||
```
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- `.loop/config.json` exists (run `/agent-loop:init` first)
|
||||
- `.loop/prd.json` exists with stories (run `/agent-loop:plan` first)
|
||||
|
||||
## Instructions
|
||||
|
||||
When the user invokes `/loop-run`, follow this orchestration sequence exactly.
|
||||
Follow this orchestration sequence exactly.
|
||||
|
||||
### Step 0: Parse Arguments
|
||||
### Step 0: Validate Prerequisites
|
||||
|
||||
- If a number is provided, use it as max iterations. Otherwise read `maxIterations` from `.loop/config.json`.
|
||||
- If `--skip-eval` is provided, skip the evaluator pass.
|
||||
- If `--story <ID>` is provided, only work on that specific story.
|
||||
1. Check `.loop/config.json` exists. If not: tell user to run `/agent-loop:init` and stop.
|
||||
2. Check `.loop/prd.json` exists. If not: tell user to run `/agent-loop:plan` and stop.
|
||||
3. **Validate prd.json schema.** Read the file and verify:
|
||||
- Has a `userStories` array (NOT `sprints`, `stories`, or `tasks`)
|
||||
- Each story has: `id`, `title`, `passes`, `priority`
|
||||
- If validation fails, show the error and stop. Do NOT attempt to fix it automatically.
|
||||
4. Check prompts exist. Look for `.loop/prompts/generator/_base.md` in these locations (first match wins):
|
||||
- `.loop/prompts/` (local project copy)
|
||||
- `${CLAUDE_PLUGIN_ROOT}/prompts/` (plugin install)
|
||||
- `~/.claude/plugins/cache/agent-loop/agent-loop/*/prompts/` (plugin cache)
|
||||
|
||||
### Step 1: Load State
|
||||
Save the resolved prompt base path for later use. If no prompts found, tell user to run `/agent-loop:init` and stop.
|
||||
|
||||
1. Read `.loop/config.json` — get `mode`, `maxIterations`, `evalRetries`, `scopeBudgets`
|
||||
2. Read `.loop/prd.json` — get the story list and their statuses
|
||||
3. Check `.loop/progress.md` exists; if not, create it from `.loop/templates/progress.md.template`
|
||||
### Step 1: Parse Arguments and Load State
|
||||
|
||||
Report to the user:
|
||||
- Parse arguments: number → max iterations, `--skip-eval`, `--story <ID>`
|
||||
- Read `.loop/config.json` for defaults
|
||||
- Read `.loop/prd.json` for story list
|
||||
|
||||
Report:
|
||||
|
||||
> **Loop Ready**
|
||||
> - Mode: {mode}
|
||||
@@ -45,7 +49,7 @@ Report to the user:
|
||||
> - Max iterations: {N}
|
||||
> - Eval: {on/off}
|
||||
>
|
||||
> Starting loop. You can interrupt me at any time to adjust course.
|
||||
> Starting. Interrupt me at any time.
|
||||
|
||||
### Step 2: Iteration Loop
|
||||
|
||||
@@ -53,45 +57,42 @@ For each iteration (1 to max iterations):
|
||||
|
||||
#### 2a. Find Next Story
|
||||
|
||||
Find the highest-priority story in `prd.json` where `passes` is `false` and `blocked` is not `true`. If `--story` was specified, use that story instead.
|
||||
Find the story with the lowest `priority` number where `passes` is `false` and `blocked` is not `true`. If `--story` was specified, use that story.
|
||||
|
||||
**If no actionable story remains:**
|
||||
- If all stories have `passes: true` → report success and stop
|
||||
- If some stories are `blocked: true` → report which are blocked and suggest `/agent-loop:triage`
|
||||
- All `passes: true` → report success and stop
|
||||
- Some `blocked: true` → report which and suggest `/agent-loop:triage`
|
||||
- Stop the loop
|
||||
|
||||
#### 2b. Report Iteration Start
|
||||
|
||||
Tell the user:
|
||||
> **Iteration {N}/{max} — {story.id}: {story.title}**
|
||||
|
||||
If the story has `[REJECTED]` entries in its `notes` field, summarize the previous feedback so the user has context.
|
||||
If the story has `[REJECTED]` in its `notes`, summarize the feedback.
|
||||
|
||||
#### 2c. Assemble Generator Prompt
|
||||
|
||||
Read these files and concatenate them with `---` separators:
|
||||
1. `.loop/prompts/generator/_base.md`
|
||||
2. `.loop/prompts/generator/{mode}.md`
|
||||
Read and concatenate with `---` separator:
|
||||
1. `{prompt_base_path}/generator/_base.md`
|
||||
2. `{prompt_base_path}/generator/{mode}.md`
|
||||
|
||||
Then substitute these template variables in the assembled text:
|
||||
- `{{MAX_FILES_TO_READ}}` → from `config.scopeBudgets.{mode}.maxFilesToRead`
|
||||
- `{{MAX_LINES_TO_WRITE}}` → from `config.scopeBudgets.{mode}.maxLinesToWrite`
|
||||
- `{{MAX_FILES_TO_MODIFY}}` → from `config.scopeBudgets.{mode}.maxFilesToModify`
|
||||
- `{{MODE}}` → the mode
|
||||
- `{{ITERATION}}` → current iteration number
|
||||
Substitute template variables:
|
||||
- `{{MAX_FILES_TO_READ}}` → from config scopeBudgets
|
||||
- `{{MAX_LINES_TO_WRITE}}` → from config scopeBudgets
|
||||
- `{{MAX_FILES_TO_MODIFY}}` → from config scopeBudgets
|
||||
- `{{MODE}}` → mode from config
|
||||
- `{{ITERATION}}` → current iteration
|
||||
- `{{MAX_ITERATIONS}}` → max iterations
|
||||
- `{{LOOP_DIR}}` → path to `.loop/` directory
|
||||
- `{{PROJECT_ROOT}}` → project root path
|
||||
- `{{CURRENT_STORY_ID}}` → the story ID being worked on
|
||||
- `{{LOOP_DIR}}` → absolute path to `.loop/`
|
||||
- `{{PROJECT_ROOT}}` → project root absolute path
|
||||
- `{{CURRENT_STORY_ID}}` → story ID
|
||||
|
||||
#### 2d. Capture Pre-Generator Git State
|
||||
|
||||
Run `git rev-parse HEAD` and save it. This is needed for the evaluator's diff.
|
||||
Run `git rev-parse HEAD` and save the SHA.
|
||||
|
||||
#### 2e. Dispatch Generator Agent
|
||||
|
||||
Use the **Agent tool** to launch the generator:
|
||||
|
||||
```
|
||||
Agent(
|
||||
prompt: <assembled generator prompt>,
|
||||
@@ -101,32 +102,28 @@ Agent(
|
||||
)
|
||||
```
|
||||
|
||||
**IMPORTANT:** Use `mode: "auto"` so the user can see tool calls but isn't prompted for every action. If the user has expressed a preference for more control, use `mode: "default"` instead.
|
||||
|
||||
Wait for the agent to complete. The Agent tool returns the generator's final output.
|
||||
Wait for completion.
|
||||
|
||||
#### 2f. Check for Completion Signal
|
||||
|
||||
If the generator output contains `<promise>COMPLETE</promise>`, report all stories complete and stop.
|
||||
If output contains `<promise>COMPLETE</promise>`, report all stories complete and stop.
|
||||
|
||||
#### 2g. Skip Evaluator (if configured)
|
||||
|
||||
If `--skip-eval` was specified or `config.skipEval` is true, skip to step 2j.
|
||||
If `--skip-eval` or `config.skipEval` is true, skip to 2j and treat as PASS.
|
||||
|
||||
#### 2h. Assemble Evaluator Prompt
|
||||
|
||||
Read these files and concatenate them:
|
||||
1. `.loop/prompts/evaluator/_base.md`
|
||||
2. `.loop/prompts/evaluator/{mode}.md`
|
||||
Read and concatenate:
|
||||
1. `{prompt_base_path}/evaluator/_base.md`
|
||||
2. `{prompt_base_path}/evaluator/{mode}.md`
|
||||
|
||||
Substitute the same template variables as the generator, plus:
|
||||
- `{{PRE_GENERATOR_SHA}}` → the git SHA captured in step 2d
|
||||
- `{{CURRENT_STORY_ID}}` → the story ID
|
||||
Substitute same variables plus:
|
||||
- `{{PRE_GENERATOR_SHA}}` → SHA from step 2d
|
||||
- `{{CURRENT_STORY_ID}}` → story ID
|
||||
|
||||
#### 2i. Dispatch Evaluator Agent
|
||||
|
||||
Use the **Agent tool** to launch the evaluator:
|
||||
|
||||
```
|
||||
Agent(
|
||||
prompt: <assembled evaluator prompt>,
|
||||
@@ -136,29 +133,24 @@ Agent(
|
||||
)
|
||||
```
|
||||
|
||||
Wait for completion. Parse the verdict from the output:
|
||||
Parse the verdict:
|
||||
- `<verdict>PASS</verdict>` → PASS
|
||||
- `<verdict>REJECT</verdict>` → REJECT, extract `<rejection_reason>...</rejection_reason>`
|
||||
- No verdict tag → REJECT (fail-safe)
|
||||
|
||||
- Look for `<verdict>PASS</verdict>` → story passes
|
||||
- Look for `<verdict>REJECT</verdict>` → story rejected; extract reason from `<rejection_reason>...</rejection_reason>`
|
||||
- No verdict tag found → treat as REJECT (fail-safe)
|
||||
#### 2j. Update State
|
||||
|
||||
#### 2j. Update State Based on Verdict
|
||||
|
||||
**On PASS (or skip-eval):**
|
||||
1. Update `.loop/prd.json` — set `passes: true` for the story
|
||||
2. Report to user: ✓ **{story.id} PASSED**
|
||||
**On PASS:**
|
||||
1. Read `.loop/prd.json`, set `passes: true` for the story, write it back
|
||||
2. Report: **{story.id} PASSED**
|
||||
|
||||
**On REJECT:**
|
||||
1. Update `.loop/prd.json`:
|
||||
- Keep `passes: false`
|
||||
- Increment `rejections` count
|
||||
- Append `[REJECTED] {reason}` to `notes`
|
||||
2. Report to user: ✗ **{story.id} REJECTED** — {reason}
|
||||
3. Check if `rejections` >= `evalRetries` from config:
|
||||
- If yes: set `blocked: true` in prd.json, append `[BLOCKED]` to notes
|
||||
- Report: ⚠ **{story.id} BLOCKED** — rejected {N} times, needs human review
|
||||
1. Read `.loop/prd.json`, increment `rejections`, append `[REJECTED] {reason}` to `notes`, write back
|
||||
2. Report: **{story.id} REJECTED** — {reason}
|
||||
3. If `rejections >= evalRetries`: set `blocked: true`, append `[BLOCKED]` to notes
|
||||
- Report: **{story.id} BLOCKED** — rejected {N} times, needs human review
|
||||
|
||||
#### 2k. Append Progress Entry
|
||||
#### 2k. Append Progress
|
||||
|
||||
Append to `.loop/progress.md`:
|
||||
|
||||
@@ -171,33 +163,22 @@ Verdict: {PASS/REJECT/SKIP-EVAL}
|
||||
---
|
||||
```
|
||||
|
||||
#### 2l. Report Iteration Summary
|
||||
#### 2l. Report and Continue
|
||||
|
||||
Show current story counts: `{passed}/{total} stories complete`
|
||||
Show: `{passed}/{total} stories complete`
|
||||
|
||||
If there are more iterations and more stories, continue to the next iteration.
|
||||
Continue to next iteration.
|
||||
|
||||
### Step 3: Loop Exit
|
||||
|
||||
When the loop ends (all stories done, max iterations, or all remaining blocked), report:
|
||||
|
||||
> **Loop Complete**
|
||||
> - Iterations used: {N}
|
||||
> - Stories: {passed}/{total} complete, {blocked} blocked
|
||||
> - {Suggest `/agent-loop:triage` if anything is blocked or incomplete}
|
||||
|
||||
If incomplete, suggest `/agent-loop:triage`.
|
||||
|
||||
### Error Handling
|
||||
|
||||
- If an Agent subagent fails or returns empty output, log a warning and continue to the next iteration. Do NOT stop the loop for a single agent failure.
|
||||
- If `prd.json` cannot be parsed, stop immediately and report the error.
|
||||
- If the user interrupts (denies a tool call, says "stop", etc.), gracefully end the loop and report current status.
|
||||
|
||||
### Key Differences from loop.sh
|
||||
|
||||
| Feature | loop.sh | /loop-run |
|
||||
|---------|---------|-----------|
|
||||
| Execution | Headless (`claude --print`) | Visible in Claude Code |
|
||||
| Intervention | Kill the process | Deny tool calls, chat mid-loop |
|
||||
| Permissions | `--dangerously-skip-permissions` | User-controlled |
|
||||
| Context | Fresh process per agent | Fresh Agent subagent per agent |
|
||||
| State updates | Shell functions | Claude Code reads/writes files directly |
|
||||
- Agent fails or empty output → warn and continue to next iteration
|
||||
- prd.json unparseable → stop immediately
|
||||
- User says "stop" → end loop, report current status
|
||||
|
||||
Reference in New Issue
Block a user