eval-ladder

v0.0.1

Design and audit an agent system's eval ladder — the cheapest rung that catches each regression, the blind spot beside every green, judges validated by TPR/TNR, pass^k for irreversible actions. Use when designing, auditing, or defending a test/eval strategy for an agent, skill, or prompt; when adding an eval tier or LLM judge; or when a suite is all-green and you cannot say what it would catch.

Claude Code1 Skill

By Jordan RichlenLicense: MIT1 GitHub starsUpdated 2 hours ago

Directory evidence

Runtimes: Claude Code
Parsed components: 1 skill or MCP entry
Source updated: Sep 24, 2026
Manifest status: Canonical path parsed

The directory validates manifest shape and source location. It does not execute the plugin or provide a security endorsement. Review the indexing methodology →

Install eval-ladder for Claude Code

Installs for the current user

claude plugin marketplace add IchenDEV/agent-plugin-mkt
claude plugin marketplace update agent-plugin-marketplace
claude plugin install eval-ladder@agent-plugin-marketplace

Paste and run these commands in a terminal with Claude Code. They add and refresh the PluginsMP catalog, then install this plugin.

The installer fetches third-party code from the source repository shown on this page. This directory validates manifest structure and source location, but does not perform a security audit; review the manifest, components, and source before installing.

Get the source manually

$ git clone https://github.com/JRichlen/agent-plugins

Clone the source repository, then follow its setup instructions to add the plugin to a compatible client. The plugin root is plugins/eval-ladder/.

Plugin files

plugins/eval-ladder/

├── .claude-plugin/plugin.json
└── skills/eval-ladder/SKILL.md

Included Skills1

eval-ladderskills/eval-ladder/SKILL.md

Design and audit the eval ladder for an agent system: build tiers bottom-up from observed failures, pick the cheapest rung that can catch a given regression, name what each rung structurally cannot prove, validate every LLM judge against human labels (TPR and TNR separately, never raw agreement), and score irreversible-action scenarios pass^k rather than by majority. Use when designing, auditing, or defending a test/eval strategy for an agent, skill, or prompt; when adding an eval tier or LLM judge; when a suite is all-green and you cannot say what it would catch; or on phrases like "assess our testing strategy", "do we have eval coverage", "is this judge trustworthy", "pass@k or pass^k", "what does this tier actually prove".

Plugin manifests1

plugins/eval-ladder/.claude-plugin/plugin.json

{
  "name": "eval-ladder",
  "version": "0.0.1",
  "description": "Design and audit an agent system's eval ladder — the cheapest rung that catches each regression, the blind spot beside every green, judges validated by TPR/TNR, pass^k for irreversible actions. Use when designing, auditing, or defending a test/eval strategy for an agent, skill, or prompt; when adding an eval tier or LLM judge; or when a suite is all-green and you cannot say what it would catch.",
  "author": {
    "name": "Jordan Richlen"
  },
  "repository": "https://github.com/JRichlen/agent-plugins",
  "homepage": "https://github.com/JRichlen/agent-plugins/tree/main/plugins/eval-ladder",
  "license": "MIT",
  "keywords": [
    "evals",
    "testing-strategy",
    "llm-as-judge",
    "coverage-audit",
    "pass-at-k",
    "test-tiers"
  ],
  "skills": "./skills/",
  "commands": "./commands/"
}

Browse the “evals” tag →

autorefine

by Surah Li

Iteratively improve any skill using eval-grounded autoresearch and Hamel's Three Gulfs methodology.

CodexClaude Code

9Updated 2 months ago

claude-app-engineering

by Matt Corbett

Claude app-engineering specialist team — agents (claude-solution-architect, prompt-and-context-engineer, mcp-and-server-tools-engineer, agent-sdk-engineer, eval-engineer, claude-app-ops-engineer) for building production applications on the Claude API, the Claude Agent SDK, and MCP. Covers the build-surface decision (Messages API / Agent SDK / Managed Agents), model right-sizing (Opus 4.8 / Sonnet 4.6 / Haiku 4.5, 1M context), prompt caching, tool use + structured output, MCP server authoring + Anthropic-hosted server tools (computer use, code execution, Files API, memory), evals (golden sets, LLM-as-judge), and FinOps + reliability (batch, model routing, backoff, observability). dated-and-cited knowledge bank, templates, an advisory hook (house opinions). Seams: prompt-as-artifact, security, and cross-domain architecture escalate to ravenclaude-core. Requires ravenclaude-core@>=0.7.0.

Claude Code7 Skills

7Updated last week

llm-evaluation-engineering

by Matt Corbett

LLM-evaluation engineering team — agents (eval-strategy-lead, eval-harness-engineer) for the question every team shipping an AI feature has to answer: is it getting better or worse? Eval design (task-grounded metrics, offline-vs-online split, sample size, a ship-gate decided as a number before the run), and the harness that proves it (frozen golden sets with provenance, LLM-as-judge rubrics with a bias audit and human calibration, CI regression gates, guardrail/red-team suites). skills, a decision-tree knowledge bank (an eval-method Mermaid tree + a dated 2026 tooling/method map), best-practices. Engineering decision-support, not a safety certification; model/judge/tooling specifics are volatile — retrieval-dated + verify-at-use; no eval-data PII. Seams: retrieval quality -> ai-rag-engineering; model selection -> ai-coding-model-guidance; classical-ML training/metrics -> ml-engineering; product A/B -> experimentation-growth-engineering. Requires ravenclaude-core@>=0.7.0.

Claude Code3 Skills

7Updated last week

For maintainers

If you maintain this plugin, link to this source-backed listing from your README so users can review its manifest and indexed components.

[eval-ladder on Agent Plugins Marketplace](https://pluginsmp.com/plugins/eval-ladder)