web-scraping-data-extraction

v0.1.2

Web-scraping & data-extraction engineering team — agents (extraction-architect, scraper-implementation-engineer) answering 'how do we acquire this web data legally, robustly, and at the right cost?': API-vs-scrape source choice, a legal/ethical gate (robots.txt, ToS, public-vs-authenticated, rate, PII/copyright) BEFORE any code, fetch strategy (HTTP vs headless, JSON-endpoint-first), resilient parsing (structured data over brittle selectors, validate-to-schema), change-detection & re-crawl scheduling, and the extract→validate→dedup→store pipeline with provenance. skills, a knowledge bank (a Mermaid decision tree + a dated 2026 tooling/legal reference). Distinct from data-orchestration/data-streaming (move data that ALREADY exists) and backend-engineering — this team owns ACQUIRING web data. Legality-first: anti-bot is a stop-signal, never evasion for abuse; ToS/robots specifics volatile + jurisdictional, retrieval-dated + verify-at-use. Needs ravenclaude-core.

Claude Code3 Skills

By Matt CorbettLicense: MIT7 GitHub starsUpdated last week

Directory evidence

Runtimes: Claude Code
Parsed components: 3 skill or MCP entries
Source updated: Sep 15, 2026
Manifest status: Canonical path parsed

The directory validates manifest shape and source location. It does not execute the plugin or provide a security endorsement. Review the indexing methodology →

Install web-scraping-data-extraction for Claude Code

Installs for the current user

claude plugin marketplace add IchenDEV/agent-plugin-mkt
claude plugin marketplace update agent-plugin-marketplace
claude plugin install web-scraping-data-extraction@agent-plugin-marketplace

Paste and run these commands in a terminal with Claude Code. They add and refresh the PluginsMP catalog, then install this plugin.

The installer fetches third-party code from the source repository shown on this page. This directory validates manifest structure and source location, but does not perform a security audit; review the manifest, components, and source before installing.

Get the source manually

$ git clone https://github.com/mcorbett51090/RavenClaude

Clone the source repository, then follow its setup instructions to add the plugin to a compatible client. The plugin root is plugins/web-scraping-data-extraction/.

Plugin files

plugins/web-scraping-data-extraction/

├── .claude-plugin/plugin.json
├── skills/crawl-scheduling-and-pipeline/SKILL.md
├── skills/legal-ethical-and-fetch-strategy/SKILL.md
└── skills/resilient-extraction-and-parsing/SKILL.md

Included Skills3

crawl-scheduling-and-pipelineskills/crawl-scheduling-and-pipeline/SKILL.md

Design the crawl schedule, change-detection, and the extraction-to-storage pipeline — a re-crawl cadence matched to data volatility, incremental crawls via conditional GET / ETag / sitemap lastmod, a polite rate-limit/backoff budget, and storage with provenance (source URL + fetch timestamp) and dedup. Traverses the schedule/pipeline branch of the web-scraping decision tree. Reach for this when the user asks 'how often should we re-crawl?', 'how do we detect changes?', 'set up the extraction pipeline', or 'add rate-limiting so we don't get blocked'. Used by scraper-implementation-engineer (primary) and extraction-architect.

legal-ethical-and-fetch-strategyskills/legal-ethical-and-fetch-strategy/SKILL.md

Run the legality/ethics gate BEFORE any code (robots.txt, ToS, public-vs-authenticated, rate, PII/copyright), prefer an API/feed/export over scraping, then choose the fetch strategy (HTTP + parser vs headless, JSON-endpoint-first). Traverses the top of the web-scraping decision tree. Reach for this when the user asks 'is it OK to scrape this?', 'should we scrape or is there an API?', 'do we need a headless browser?', or 'how do we do this without getting blocked/sued?'. Legality-first; never evasion for abuse. Used by extraction-architect (primary) and scraper-implementation-engineer.

resilient-extraction-and-parsingskills/resilient-extraction-and-parsing/SKILL.md

Extract web data defensively — prefer structured data (JSON-LD / microdata / __NEXT_DATA__ / JSON API) over brittle CSS/XPath selectors, anchor selectors on stable attributes with fallbacks, and validate every record to a schema so breakage is DETECTED not silently wrong. Traverses the parse branch of the web-scraping decision tree. Reach for this when the user asks 'build a robust extractor', 'this scraper keeps breaking', 'how do I parse this reliably?', or 'why is my scraped data wrong?'. Used by scraper-implementation-engineer (primary) and extraction-architect.

Plugin manifests1

plugins/web-scraping-data-extraction/.claude-plugin/plugin.json

{
  "name": "web-scraping-data-extraction",
  "version": "0.1.2",
  "description": "Web-scraping & data-extraction engineering team — agents (extraction-architect, scraper-implementation-engineer) answering 'how do we acquire this web data legally, robustly, and at the right cost?': API-vs-scrape source choice, a legal/ethical gate (robots.txt, ToS, public-vs-authenticated, rate, PII/copyright) BEFORE any code, fetch strategy (HTTP vs headless, JSON-endpoint-first), resilient parsing (structured data over brittle selectors, validate-to-schema), change-detection & re-crawl scheduling, and the extract→validate→dedup→store pipeline with provenance. skills, a knowledge bank (a Mermaid decision tree + a dated 2026 tooling/legal reference). Distinct from data-orchestration/data-streaming (move data that ALREADY exists) and backend-engineering — this team owns ACQUIRING web data. Legality-first: anti-bot is a stop-signal, never evasion for abuse; ToS/robots specifics volatile + jurisdictional, retrieval-dated + verify-at-use. Needs ravenclaude-core.",
  "author": {
    "name": "Matt Corbett"
  },
  "homepage": "https://github.com/mcorbett51090/RavenClaude",
  "license": "MIT",
  "keywords": [
    "web-scraping",
    "data-extraction",
    "crawling",
    "robots-txt",
    "headless-browser",
    "html-parsing",
    "rate-limiting",
    "change-detection",
    "etl",
    "data-pipeline"
  ],
  "requires": {
    "plugins": [
      "ravenclaude-core@>=0.7.0"
    ]
  }
}