Agent Plugins Marketplace
← All plugins

eval

v0.4.0

Grades a real run of a skill against the contract that skill committed to, keeps only the findings it can cite, and turns the ones that will recur into permanent eval cases.

CodexClaude Code1 Skill

By Nate SwensonLicense: MIT3 GitHub starsUpdated 4 days ago

Directory evidence

Runtimes
Codex and Claude Code
Parsed components
1 skill or MCP entry
Source updated
Sep 19, 2026
Manifest status
Canonical path parsed

The directory validates manifest shape and source location. It does not execute the plugin or provide a security endorsement. Review the indexing methodology →

Install eval for Codex and Claude Code

Installs for the current user
codex plugin marketplace add natejswenson/claude-skills
codex plugin marketplace upgrade claude-skills
codex plugin add eval@claude-skills

Paste and run these commands in a terminal with Codex. They add and refresh the claude-skills catalog, then install this plugin.

Compatibility: the page URL and API slug “eval” remain stable.

  • Codex: eval@agent-plugin-marketplace → eval@claude-skills

The installer fetches third-party code from the source repository shown on this page. This directory validates manifest structure and source location, but does not perform a security audit; review the manifest, components, and source before installing.

Get the source manually
git clone https://github.com/natejswenson/claude-skills

Clone the source repository, then follow its setup instructions to add the plugin to a compatible client. The plugin root is skills/eval/.

Plugin files

skills/eval/
├── .codex-plugin/plugin.json
├── .claude-plugin/plugin.json
└── skills/eval/SKILL.md

Included Skills1

evalskills/eval/SKILL.md

Grade a real run of a skill against its own committed contract, then turn each confirmed failure into a permanent eval case. Use when the user says "evaluate this run", "grade this skill", "eval this run", "how good was that run", "did the skill follow its own rules", "audit this transcript", "write evals for my skill", "deepen the evals", or "my evals are decorative" — or wants a session transcript checked for results it claimed but never observed.

Plugin manifests2

skills/eval/.codex-plugin/plugin.json
{
  "name": "eval",
  "version": "0.4.0",
  "description": "Grades a real run of a skill against the contract that skill committed to, keeps only the findings it can cite, and turns the ones that will recur into permanent eval cases.",
  "author": {
    "name": "Nate Swenson"
  },
  "homepage": "https://github.com/natejswenson/claude-skills/tree/main/skills/eval",
  "license": "MIT",
  "skills": "./skills/",
  "interface": {
    "displayName": "eval",
    "shortDescription": "Grades a real run of a skill against the contract that skill committed to, keeps only the findings it can cite, and turn",
    "longDescription": "Grades a real run of a skill against the contract that skill committed to, keeps only the findings it can cite, and turns the ones that will recur into permanent eval cases.",
    "developerName": "Nate Swenson",
    "category": "Productivity",
    "capabilities": [],
    "defaultPrompt": [
      "Use $eval to help me."
    ]
  }
}
skills/eval/.claude-plugin/plugin.json
{
  "name": "eval",
  "version": "0.4.0",
  "description": "Grades a real run of a skill against the contract that skill committed to, keeps only the findings it can cite, and turns the ones that will recur into permanent eval cases.",
  "author": {
    "name": "Nate Swenson"
  },
  "homepage": "https://github.com/natejswenson/claude-skills/tree/main/skills/eval",
  "license": "MIT"
}

For maintainers

If you maintain this plugin, link to this source-backed listing from your README so users can review its manifest and indexed components.

[eval on Agent Plugins Marketplace](https://pluginsmp.com/plugins/eval)