majestic-data

Design and evaluate anomaly detection for tabular, multivariate, or time-series data using context-appropriate statistical or model-based methods. Use when identifying unusual records, monitoring metric deviations, tuning alert thresholds, or investigating suspected outliers.

Diagnose and recover messy delimited files without silently dropping or corrupting records. Use when CSV or TSV files have uncertain encoding, delimiters, headers, quoting, malformed rows, locale-specific values, schema drift, or excessive size.

Design reliable batch or incremental data pipelines with explicit grain, load strategy, idempotency, checkpoints, late-data handling, backfills, storage layout, and reconciliation. Use when planning ETL, ELT, CDC, file-based, or PostgreSQL-backed data movement before implementation.

Design tests and fixtures for data pipelines across transformation, contract, integration, incrementality, replay, and reconciliation behavior. Use when verifying ETL or ELT code, dbt models, schema changes, backfills, late data, idempotency, or failure recovery.

Define and operate data quality controls, service levels, scorecards, drift monitoring, and incident response for a data product. Use when establishing trust requirements, freshness or volume alerts, quality gates, ownership, or ongoing monitoring across pipelines and datasets.

Assess an unfamiliar data source by profiling its schema, grain, volume, change behavior, quality baseline, and extraction constraints. Use when evaluating a database, API, CSV, JSON, Parquet dataset, file drop, or event stream before pipeline integration.

Design executable data contracts and validation behavior across records, DataFrames, warehouse models, and pipeline boundaries. Use when enforcing schemas, nullability, uniqueness, ranges, relationships, business rules, schema evolution, quarantine, or validation failure policy.

Build and review dbt projects with clear model layers, explicit grain, sources, materializations, incremental behavior, tests, documentation, and scoped verification. Use when creating or changing dbt models, snapshots, macros, sources, data tests, unit tests, or project structure.