review-bench

A/B test two planning questions by applying each to one or more plans, judging which question produces a better plan revision. Priority chain: quality > input tokens (question size) > time. AUTOMATICALLY INVOKE when user mentions: - "compare questions", "which question is better", "A/B test questions" - "evaluate questions", "test these questions against plans" - "question efficiency", "question comparison" - Two planning questions to compare against plan(s) STRONGLY RECOMMENDED for: - Optimizing review-plan question quality - Reducing question token cost while maintaining effectiveness - Choosing between alternative question phrasings - Validating new questions against existing ones Position bias mitigated via randomized ordering per test case — judge sees A/B in random order, results remapped before aggregation.

Iteratively researches real software project failures and wins, extracts key planning questions via 5-whys analysis, validates them against synthetic test plans, judges their effectiveness via a parallel judge agent, refines them, and persists to a growing questions library. Builds a curated, language/system-agnostic question library that prevents known failure modes when applied during software planning. Supports multiple resumable runs: reads existing question inventory on startup, tracks researched failure domains, avoids near-duplicating existing questions. AUTOMATICALLY INVOKE when user mentions: - "derive questions", "derive planning questions", "research failure questions" - "build question library", "question library from failures" - "mine post-mortems", "extract planning questions"

Systematically optimize review-plan evaluator questions for token efficiency while maintaining quality. For each target question: generates 3 token-optimized variants (STRUCTURAL, SEMANTIC, RADICAL) in parallel, selects the best one (optionally synthesizing strengths from others), A/B tests it against the original via compare-questions using a dedicated test plan, and updates QUESTIONS.md if the variant wins. Retries up to N times with judge feedback. AUTOMATICALLY INVOKE when user mentions: - "optimize questions", "compress questions", "reduce question tokens" - "token-optimize review-plan", "make questions more concise" - "question efficiency", "shrink evaluator prompts" STRONGLY RECOMMENDED for: - Reducing review-plan token consumption - Iterative question quality improvement - Validating question rewrites against calibration plans

Optimize or refine the Sheets Chat ClaudeConversation system prompt. Analyze, compress, refine, A/B test, and deploy system prompt changes with automated quality gates. **AUTOMATICALLY INVOKE** when: - User mentions "system prompt", "optimize prompt", "compress prompt" - User wants to change ClaudeConversation system behavior - User asks about prompt token usage or context budget - User says "prompt too long", "reduce prompt", "prompt efficiency" - User says "refine system prompt", "update system prompt", "add to system prompt" - User wants to adapt the system prompt for a new model version - **(--mode ideate)** User says "ideate system prompt", "generate prompt ideas", "hypothesize prompt changes", "explore prompt improvements", "test new prompt ideas" **NOT for:** General prompt engineering, non-GAS prompts, one-off prompt writing. Default mode is user-directed refinement/compression. `--mode ideate` is autonomous hypothesis generation + benchmarking (was the standalone `/ideate-system-prompt` skill).

End-to-end validation of review-plan question effectiveness. Runs a multi-stage pipeline against the bench bank of realistic plans to determine which questions create material value, which overlap, and which should be removed. Produces a Question Effectiveness Report with per-question KEEP/MERGE/CONDITIONAL/DROP dispositions backed by empirical ablation data. Orchestrates: question-bench (ablation engine), overlap investigation (targeted single-plan ablation for suspect pairs), cluster/layer analysis, and final synthesis. AUTOMATICALLY INVOKE when user mentions: - "validate questions", "audit questions", "question accuracy" - "are the questions working", "question validation", "question audit" - "revalidate review-plan", "check question effectiveness" NOT for: running a single ablation experiment (use /question-bench), reviewing a plan (use /review-plan)