coco-databricks

Create, configure, deploy, and run Databricks Declarative Automation Bundles (DAB, formerly Databricks Asset Bundles). Use when: initializing bundle projects, defining jobs, pipelines, Python wheels, Scala JARs, MLOps stacks, or Databricks apps as code, deploying bundles to workspaces, running bundle workflows, validating configurations, generating bundle config from existing resources, binding resources, creating custom templates, or managing the full bundle lifecycle (init → validate → deploy → run → destroy). Triggers: bundle, DAB, asset bundle, declarative automation bundle, bundle init, bundle deploy, bundle run, bundle validate, bundle destroy, databricks.yml, job bundle, pipeline bundle, python wheel bundle, scala jar bundle, mlops stacks, bundle template, CI/CD databricks, infrastructure as code databricks.

Run Databricks CLI commands for day-to-day workspace operations. Use when: managing clusters, running jobs, listing or cancelling job runs, managing SQL warehouses, working with workspace files and notebooks, managing secrets and scopes, managing users/groups/service-principals/permissions, querying serving endpoints, interacting with ML experiments and model registry, using the filesystem (DBFS/Volumes), executing arbitrary REST API calls, or any general databricks CLI operation not covered by the bundle, install, UC, or ETL skills. Triggers: databricks cli, clusters, jobs, job runs, pipelines, SQL warehouse, workspace, notebooks, secrets, fs, dbfs, serving endpoints, model serving, experiments, model registry, users, groups, service principals, permissions, grants, alerts, queries, instance pools, libraries, repos, git credentials, account, api, REST.

Install, update, and configure the Databricks CLI. Use when the user wants to install databricks cli, set up databricks cli, update databricks cli, configure databricks authentication, databricks auth login, brew install databricks, troubleshoot databricks cli installation issues, connect to a new databricks workspace, new workspace onboarding, getting started with databricks, or first time databricks setup. NOTE: If the user wants to install Databricks SKILLS or MCP tools for Cortex Code (AI Dev Kit), route to the databricks-setup skill instead.

Audit and optimize Databricks costs across compute, storage, and workloads. Use when: reduce Databricks cost, save money, cost optimization, expensive cluster, DBU usage, spot instances, autoscaling, auto termination, compute policy, right-size cluster, serverless vs classic, Photon cost-benefit, streaming cost, always-on streaming, tagging, cost attribution, chargeback, budget alerts, cost monitoring, system tables billing, billing usage, system.billing.usage, job compute vs all-purpose, SQL warehouse sizing, instance type selection, GPU cost, unnecessary GPU, compute policies, fleet instances, cost audit, cost dashboard, reduce spend Databricks, model serving cost, ML training cost, Delta optimization cost.

Databricks SQL (DBSQL) advanced features and SQL warehouse capabilities. Use when the user mentions: DBSQL, Databricks SQL, SQL warehouse, SQL scripting, stored procedure, CALL procedure, materialized view, CREATE MATERIALIZED VIEW, pipe syntax, pipe operator, geospatial, H3, ST_, spatial SQL, collation, COLLATE, ai_query, ai_classify, ai_extract, ai_gen, ai_analyze_sentiment, ai_similarity, ai_forecast, ai_mask, ai_fix_grammar, ai_parse_document, vector_search, AI function, http_request, remote_query, read_files, Lakehouse Federation, recursive CTE, WITH RECURSIVE, multi-statement transaction, temp table, temporary view, data modeling best practices on Databricks, Liquid Clustering, star schema Databricks, SCD Type 2 Databricks.

Build, deploy, and run end-to-end dbt pipelines on Databricks using Declarative Automation Bundles (DAB) and the Databricks CLI. Use when: dbt pipeline, dbt project, dbt Databricks, dbt-databricks, dbt task, dbt workflow, dbt job, dbt run, dbt seed, dbt test, dbt deps, dbt build, dbt models, dbt sources, dbt transformations, profiles.yml Databricks, dbt SQL warehouse, dbt serverless, dbt-core Databricks, dbt bundle, dbt-sql template, medallion dbt, dbt deploy, dbt schedule, dbt CI/CD, dbt DAB, dbt production, dbt development workflow.

Build, deploy, and orchestrate ETL pipelines on Databricks using PySpark notebooks (.ipynb). Use when: creating ETL pipelines, building medallion architecture (silver/gold), writing PySpark transformations in notebooks, scheduling notebook-based data pipelines, deploying ETL jobs with Declarative Automation Bundles, configuring serverless or classic compute for ETL workloads, writing to Delta or Iceberg tables. Assumes source data already exists as tables in Unity Catalog. Triggers: ETL, ELT, extract transform load, PySpark, spark notebook, medallion, silver gold, data pipeline, transform, aggregate, data engineering, Delta Lake, Iceberg, data lakehouse, notebook job, ETL job, Delta merge, SCD.

Generate pytest suites for Databricks PySpark code that run locally without a cluster. Handles dbutils mocking (fs, secrets, widgets), SparkSession fixtures, display() stubs, and DataFrame assertions. Use when: writing unit tests, testing locally, mocking dbutils, pytest databricks, local testing, CI/CD testing, test pyspark, mock secrets, mock widgets, test without cluster, assertDataFrameEqual, conftest fixtures.

Refactor monolithic Databricks notebooks into modular Python packages. Extracts business logic into testable .py modules, replaces %run chains with imports, parameterizes hardcoded values, and creates thin orchestrator notebooks. Use when: refactor notebook, modularize code, extract functions, %run migration, notebook to package, monolith notebook, code organization, split notebook, clean up notebook, production-ready code, move to modules, notebook too big, organize notebook code.

Install, update, and manage the Databricks AI Dev Kit skills for Cortex Code. Handles prerequisites, installer execution, and profile management. Use when: install databricks skills, set up databricks tools, setup databricks, update ai dev kit, databricks ai dev kit, databricks setup, dev kit install, add databricks skills, reinstall databricks tools, change skill profile, list available skills.

Diagnose and fix Spark job performance bottlenecks on Databricks. Use when: slow Spark job, shuffle optimization, data skew, spill to disk, broadcast join, AQE tuning, Adaptive Query Execution, Photon evaluation, partition tuning, Spark UI interpretation, stage bottleneck, OOM, out of memory, GC pressure, task duration skew, shuffle partition count, spark.sql.shuffle.partitions, autoBroadcastJoinThreshold, redundant shuffle, unnecessary repartition, double shuffle, why is my Spark job slow, optimize Spark, performance tuning Databricks, slow stage, slow query Databricks.

Browse and discover data in Databricks Unity Catalog via the CLI. Use when: listing catalogs, schemas, tables, volumes, viewing table metadata, exploring UC hierarchy, finding data assets, checking grants/permissions, or referencing data objects for downstream work. Triggers: unity catalog, UC, list catalogs, list schemas, list tables, describe table, browse data, data discovery, catalog.schema.table, three-level namespace, grants, permissions, volumes.