Agent Plugins Marketplace
← All plugins

databricks-lakehouse-engineering

v0.1.3

Databricks lakehouse engineering team — agents (lakehouse-architect, databricks-platform-engineer) answering 'how do we build this on Databricks correctly and affordably?': medallion (bronze/silver/gold) architecture, Delta Lake table design (partitioning, liquid clustering, OPTIMIZE/Z-ORDER, VACUUM), Unity Catalog governance (catalogs/schemas/grants, lineage), Spark & PySpark job design and the shuffle/skew/spill failure modes, Structured Streaming & Auto Loader, DLT pipelines, Photon, Jobs/Workflows orchestration, and cluster/SQL-warehouse sizing & cost control (DBUs, autoscaling, spot, serverless). Engineering judgment, not a benchmark; DBR/runtime/pricing specifics are volatile — every version carries a retrieval date + [verify-at-use]. Distinct from microsoft-fabric (Fabric/OneLake), data-platform (generic ETL), data-orchestration (Airflow/Dagster), analytics-engineering (dbt/semantic layer), and ml-engineering (classical MLOps). Needs ravenclaude-core.

Claude Code2 Skills

By Matt CorbettLicense: MIT7 GitHub starsUpdated last week

Directory evidence

Runtimes
Claude Code
Parsed components
2 skill or MCP entries
Source updated
Sep 15, 2026
Manifest status
Canonical path parsed

The directory validates manifest shape and source location. It does not execute the plugin or provide a security endorsement. Review the indexing methodology →

Install databricks-lakehouse-engineering for Claude Code

Installs for the current user
claude plugin marketplace add IchenDEV/agent-plugin-mkt
claude plugin marketplace update agent-plugin-marketplace
claude plugin install databricks-lakehouse-engineering@agent-plugin-marketplace

Paste and run these commands in a terminal with Claude Code. They add and refresh the PluginsMP catalog, then install this plugin.

The installer fetches third-party code from the source repository shown on this page. This directory validates manifest structure and source location, but does not perform a security audit; review the manifest, components, and source before installing.

Get the source manually
git clone https://github.com/mcorbett51090/RavenClaude

Clone the source repository, then follow its setup instructions to add the plugin to a compatible client. The plugin root is plugins/databricks-lakehouse-engineering/.

Plugin files

plugins/databricks-lakehouse-engineering/
├── .claude-plugin/plugin.json
├── skills/design-medallion-lakehouse/SKILL.md
└── skills/tune-spark-and-costs/SKILL.md

Included Skills2

design-medallion-lakehouseskills/design-medallion-lakehouse/SKILL.md

Design a Databricks lakehouse from the query and freshness SLO backward: the medallion layering (what earns bronze/silver/gold), the Delta table & partitioning strategy per layer (low-cardinality partition vs liquid clustering, OPTIMIZE/Z-ORDER/VACUUM cadence, MERGE/CDC vs append/overwrite write pattern), the batch-vs-streaming call (scheduled batch is the default; Auto Loader / Structured Streaming / DLT only for a real sub-hour SLO), the Unity Catalog governance layout (catalog/schema, managed vs external, grants to groups, PII tagging), and the compute + DBU cost envelope. Reach for this when the ask is 'how should we structure this on Databricks?', 'bronze/silver/gold + table layout', 'batch or streaming?', or 'Unity Catalog setup'. Used by `lakehouse-architect` (primary).

tune-spark-and-costsskills/tune-spark-and-costs/SKILL.md

Diagnose a slow or failing Databricks/Spark job from the EVIDENCE (the Spark UI stages/task-skew/shuffle-spill/GC and the query plan) rather than guessing, then apply the fix the evidence points to — AQE skew-join or key salting for a hot key, broadcast for a small-dim join, repartition/coalesce for partition sizing, OPTIMIZE/compaction for the small-file problem, and writing-instead-of-collecting for driver OOM — and bring the DBU cost down (auto-termination, jobs-vs-all-purpose compute, right-sized warehouses, Photon where it pays). Reach for this when the ask is 'this job is slow/spilling/OOMing', 'why is this taking hours?', or 'our Databricks bill is too high'. Used by `databricks-platform-engineer` (primary).

Plugin manifests1

plugins/databricks-lakehouse-engineering/.claude-plugin/plugin.json
{
  "name": "databricks-lakehouse-engineering",
  "version": "0.1.3",
  "description": "Databricks lakehouse engineering team — agents (lakehouse-architect, databricks-platform-engineer) answering 'how do we build this on Databricks correctly and affordably?': medallion (bronze/silver/gold) architecture, Delta Lake table design (partitioning, liquid clustering, OPTIMIZE/Z-ORDER, VACUUM), Unity Catalog governance (catalogs/schemas/grants, lineage), Spark & PySpark job design and the shuffle/skew/spill failure modes, Structured Streaming & Auto Loader, DLT pipelines, Photon, Jobs/Workflows orchestration, and cluster/SQL-warehouse sizing & cost control (DBUs, autoscaling, spot, serverless). Engineering judgment, not a benchmark; DBR/runtime/pricing specifics are volatile — every version carries a retrieval date + [verify-at-use]. Distinct from microsoft-fabric (Fabric/OneLake), data-platform (generic ETL), data-orchestration (Airflow/Dagster), analytics-engineering (dbt/semantic layer), and ml-engineering (classical MLOps). Needs ravenclaude-core.",
  "author": {
    "name": "Matt Corbett"
  },
  "homepage": "https://github.com/mcorbett51090/RavenClaude",
  "license": "MIT",
  "keywords": [
    "databricks",
    "lakehouse",
    "delta-lake",
    "unity-catalog",
    "apache-spark",
    "pyspark",
    "structured-streaming",
    "auto-loader",
    "delta-live-tables",
    "photon",
    "medallion-architecture",
    "dbu-cost"
  ],
  "requires": {
    "plugins": [
      "ravenclaude-core@>=0.7.0"
    ]
  }
}

For maintainers

If you maintain this plugin, link to this source-backed listing from your README so users can review its manifest and indexed components.

[databricks-lakehouse-engineering on Agent Plugins Marketplace](https://pluginsmp.com/plugins/databricks-lakehouse-engineering)