Agent Plugins Marketplace
← All plugins

plugin-eval

v0.1.2

Evaluate Codex skills and plugins from chat with a beginner-friendly start command, local-first reports, token budget explanations, and guided benchmarking.

Codex5 Skills

By OpenAI CodexLicense: MIT7.1k GitHub starsUpdated 5 days ago

Directory evidence

Runtimes
Codex
Parsed components
5 skill or MCP entries
Source updated
Sep 18, 2026
Manifest status
Canonical path parsed

The directory validates manifest shape and source location. It does not execute the plugin or provide a security endorsement. Review the indexing methodology →

Install plugin-eval for Codex

Installs for the current user
codex plugin marketplace add openai/plugins
codex plugin marketplace upgrade openai-curated
codex plugin add plugin-eval@openai-curated

Paste and run these commands in a terminal with Codex. They add and refresh the openai-curated catalog, then install this plugin.

Compatibility: the page URL and API slug “plugin-eval-2” remain stable.

  • Codex: plugin-eval-2@agent-plugin-marketplace → plugin-eval@openai-curated

The installer fetches third-party code from the source repository shown on this page. This directory validates manifest structure and source location, but does not perform a security audit; review the manifest, components, and source before installing.

Get the source manually
git clone https://github.com/openai/plugins

Clone the source repository, then follow its setup instructions to add the plugin to a compatible client. The plugin root is plugins/plugin-eval/.

Plugin files

plugins/plugin-eval/
├── .codex-plugin/plugin.json
├── skills/evaluate-plugin/SKILL.md
├── skills/evaluate-skill/SKILL.md
├── skills/improve-skill/SKILL.md
├── skills/metric-pack-designer/SKILL.md
└── skills/plugin-eval/SKILL.md

Included Skills5

evaluate-pluginskills/evaluate-plugin/SKILL.md

Evaluate a local Codex plugin in engineer-friendly language. Use when the user says "evaluate this plugin", "audit this plugin", "why did this score that way", "what should I fix first", "help me benchmark this plugin", or asks for a plugin-wide report before comparing versions.

evaluate-skillskills/evaluate-skill/SKILL.md

Evaluate a local Codex skill in engineer-friendly terms. Use when the user says "evaluate this skill", "give me an analysis of the game dev skill", "audit this skill", "why did this score that way", "what should I fix first", or asks for a skill-specific report before benchmarking it.

improve-skillskills/improve-skill/SKILL.md

Turn plugin-eval findings into a concrete rewrite brief for a Codex skill. Use when the user already evaluated a skill and now wants Codex to improve it, especially after asking what to fix first.

metric-pack-designerskills/metric-pack-designer/SKILL.md

Design custom metric packs for plugin-eval so teams can add local evaluation rubrics that emit schema-compatible checks and metrics. Use when the user wants their own evaluation criteria or visualizations.

plugin-evalskills/plugin-eval/SKILL.md

Help engineers evaluate a local skill or plugin, explain why it scored that way, show what to fix first, measure real token usage, benchmark starter scenarios, or decide what to run next. Use when the user says things like "evaluate this skill", "give me an analysis of the game dev skill", "why did this score that way", "what should I fix first", "measure the real token usage of this skill", or "what should I run next?".

Plugin manifests1

plugins/plugin-eval/.codex-plugin/plugin.json
{
  "name": "plugin-eval",
  "version": "0.1.2",
  "description": "Evaluate Codex skills and plugins from chat with a beginner-friendly start command, local-first reports, token budget explanations, and guided benchmarking.",
  "author": {
    "name": "OpenAI Codex",
    "email": "[email protected]",
    "url": "https://openai.com/"
  },
  "homepage": "https://openai.com/",
  "repository": "https://github.com/openai/plugins",
  "license": "MIT",
  "keywords": [
    "codex",
    "plugin",
    "skill",
    "evaluation",
    "quality",
    "budget"
  ],
  "skills": "./skills/",
  "interface": {
    "displayName": "Plugin Eval",
    "shortDescription": "Start from chat, then evaluate or benchmark locally",
    "longDescription": "Ask Codex to evaluate a plugin or skill, give you a full analysis of a named plugin such as game-studio, explain why it scored that way, show what to fix first, explain its token budget, measure real token usage, benchmark a plugin, or tell you what to run next. Plugin Eval keeps the path engineer-friendly: start with a natural chat request, then use the local `plugin-eval start` entrypoint or the routed workflow command it recommends.",
    "developerName": "OpenAI",
    "category": "Developer Tools",
    "capabilities": [
      "Interactive",
      "Write"
    ],
    "websiteURL": "https://openai.com/",
    "privacyPolicyURL": "https://openai.com/policies/privacy-policy/",
    "termsOfServiceURL": "https://openai.com/policies/terms-of-use/",
    "defaultPrompt": [
      "Give me an analysis of the game studio plugin.",
      "Evaluate this plugin.",
      "Why did this score that way?"
    ],
    "brandColor": "#0F766E",
    "composerIcon": "./assets/plugin-eval.svg",
    "logo": "./assets/plugin-eval.svg",
    "screenshots": []
  }
}

For maintainers

If you maintain this plugin, link to this source-backed listing from your README so users can review its manifest and indexed components.

[plugin-eval on Agent Plugins Marketplace](https://pluginsmp.com/plugins/plugin-eval-2)