evals plugins

by Surah Li

Iteratively improve any skill using eval-grounded autoresearch and Hamel's Three Gulfs methodology.

by Matt Corbett

Claude app-engineering specialist team — agents (claude-solution-architect, prompt-and-context-engineer, mcp-and-server-tools-engineer, agent-sdk-engineer, eval-engineer, claude-app-ops-engineer) for building production applications on the Claude API, the Claude Agent SDK, and MCP. Covers the build-surface decision (Messages API / Agent SDK / Managed Agents), model right-sizing (Opus 4.8 / Sonnet 4.6 / Haiku 4.5, 1M context), prompt caching, tool use + structured output, MCP server authoring + Anthropic-hosted server tools (computer use, code execution, Files API, memory), evals (golden sets, LLM-as-judge), and FinOps + reliability (batch, model routing, backoff, observability). dated-and-cited knowledge bank, templates, an advisory hook (house opinions). Seams: prompt-as-artifact, security, and cross-domain architecture escalate to ravenclaude-core. Requires ravenclaude-core@>=0.7.0.

by Matt Corbett

LLM-evaluation engineering team — agents (eval-strategy-lead, eval-harness-engineer) for the question every team shipping an AI feature has to answer: is it getting better or worse? Eval design (task-grounded metrics, offline-vs-online split, sample size, a ship-gate decided as a number before the run), and the harness that proves it (frozen golden sets with provenance, LLM-as-judge rubrics with a bias audit and human calibration, CI regression gates, guardrail/red-team suites). skills, a decision-tree knowledge bank (an eval-method Mermaid tree + a dated 2026 tooling/method map), best-practices. Engineering decision-support, not a safety certification; model/judge/tooling specifics are volatile — retrieval-dated + verify-at-use; no eval-data PII. Seams: retrieval quality -> ai-rag-engineering; model selection -> ai-coding-model-guidance; classical-ML training/metrics -> ml-engineering; product A/B -> experimentation-growth-engineering. Requires ravenclaude-core@>=0.7.0.

by Goodeye Labs

MCP server and agent skills for the Truesight AI quality platform. Score inputs, build evaluations, analyze errors, and review results through natural language.

by zerlinpi

Amazon Ads AI Agent skills, playbooks and eval fixtures for auditing, monitoring, causal diagnosis, growth discovery, experiments, weekly reviews, post-change evaluation, optimization memory and guarded decisions.

by Jordan Richlen

Design and audit an agent system's eval ladder — the cheapest rung that catches each regression, the blind spot beside every green, judges validated by TPR/TNR, pass^k for irreversible actions. Use when designing, auditing, or defending a test/eval strategy for an agent, skill, or prompt; when adding an eval tier or LLM judge; or when a suite is all-green and you cannot say what it would catch.

by AER Engineering

Adaptive repository-aware engineering control plane for coding, debugging, research, POCs, reviews and safe changes, with capability-aware providers, durable memory, procedural skills, bounded delegation, secure tools, verification and evidence-backed learning.

Career agents for anyone: sync LinkedIn, CV and personal website from one source of truth, review and score all three, find and apply to jobs, track the pipeline, prep and mock every interview stage, write posts with visuals, grow reach, go multi-platform and multilingual, review offers, see it all on a dashboard, and run it unattended on a schedule with email, calendar and digest integrations. Automation modes draft/assisted/auto: LinkedIn MCP for reads and messages, a browser operator agent for profile edits, posts and applications.

by Carlos Araujo

Planeja e revisa arquiteturas LangGraph com dois agentes isolados topo de linha: Architect e Reviewer.

by Carlos Araujo

Prompt Engineering, engenharia e auditoria de evals agenticos LangSmith-first: prompts, datasets, evaluators, experiments, backtests e gates de promocao.

by Itay Glick

Requirements tracking for Claude Code. One living spec per product: every intent is classified against everything already agreed, a contradiction stops the work instead of merging, and each release regenerates its own guide and announcement.