redpanda

Redpanda's Agentic Data Plane: governance infrastructure for building, running, and governing AI agents and MCP servers behind a credential-injecting AI Gateway for LLM providers, operated through the `rpk ai` CLI and the ADP UI at ai.redpanda.com. Use when creating or managing managed or self-managed AI agents and their triggers with `rpk ai agent` and `rpk ai trigger`, configuring remote or managed MCP servers with `rpk ai mcp-server` (including code mode and user-delegated OAuth), setting up LLM providers or discovering models with `rpk ai llm-provider` and `rpk ai model`, writing Cedar access policies with `rpk ai policy`, reading conversation transcripts or the audit log with `rpk ai agent transcript` and `rpk ai auditlog`, or working with budgets, guardrails, and cost reporting in the ADP UI. For the separate rpk cloud mcp control-plane MCP server, see `/redpanda:rpk-cloud`.

Provisions and manages Redpanda Cloud BYOC (Bring Your Own Cloud) clusters via the Control Plane API (api.redpanda.com) and the `rpk cloud byoc` plugin, where your VPC, IAM roles, and storage stay in your own AWS, GCP, or Azure account. Use when creating or tearing down BYOC clusters and Networks; wiring customer-managed IAM roles, buckets, or subnets into a cluster; setting up private connectivity (AWS PrivateLink, GCP Private Service Connect, Azure Private Link, VPC peering, or Transit Gateway egress); configuring dual listener mode (a public and a private listener per service); registering cross-account AWS access; managing Shadow Link cross-cluster DR; running `rpk cloud byoc apply`; or enabling Enterprise features (Tiered Storage, Cloud Topics, Iceberg Topics, Continuous Data Balancing, RBAC, and more) on a BYOC cluster. For Serverless clusters, see `/redpanda:cloud-serverless`; for fully Redpanda-managed Dedicated clusters, see `/redpanda:cloud-dedicated`.

Provisions and manages Redpanda Cloud Dedicated clusters via the Control Plane API (api.redpanda.com): fully Redpanda-managed single-tenant clusters that run in Redpanda's own AWS, GCP or Azure account. Use when creating, updating or deleting Dedicated clusters or Networks; choosing a region or throughput tier; configuring public or private connectivity such as PrivateLink; scripting the provisioning lifecycle with curl and OAuth2 client-credentials auth; or using `rpk cloud` against a Dedicated cluster. Also covers Enterprise features included on Dedicated such as Tiered Storage, Iceberg Topics and RBAC. For Serverless clusters, see `/redpanda:cloud-serverless`; for customer-cloud-account BYOC clusters, see `/redpanda:cloud-byoc`.

Provisions and manages Redpanda Cloud Serverless clusters via the public Control Plane API (api.redpanda.com): fully managed Kafka-compatible clusters in Redpanda's own account reached through a per-cluster Data Plane API. Use when creating or deleting Serverless clusters; authenticating with OAuth2 client credentials; picking a ServerlessRegion; polling the create/delete Operation state machine; managing topics, ACLs or secrets on the data-plane URL; or enabling Iceberg Topics, schema ID validation or RBAC. For BYOC clusters, see `/redpanda:cloud-byoc`; for Dedicated clusters, see `/redpanda:cloud-dedicated`; for the rpk CLI equivalent, see `/redpanda:rpk-cloud`.

Teaches how to build streaming data pipelines with Redpanda Connect (formerly Benthos): declarative YAML configs with the input/pipeline/output model, processors, and Bloblang mappings for transforming messages. Use when creating or editing a Connect pipeline config, wiring inputs and outputs (Kafka topics, S3, HTTP, databases), writing or debugging Bloblang, or running pipelines with `rpk connect run`. For diagnosing a failing pipeline (lint errors, metrics, tracing, dry runs), see `/redpanda:connect-debugging`; for change-data-capture connectors, see the per-database `/redpanda:connect-cdc-*` skills.

Guides setup and operation of the aws_dynamodb_cdc input in Redpanda Connect, which streams change data capture from AWS DynamoDB into Redpanda or Kafka using DynamoDB Streams. Use when configuring aws_dynamodb_cdc, enabling DynamoDB Streams, setting up IAM policies and the checkpoint table, choosing snapshot_and_cdc or snapshot_only, auto-discovering tables by tag, or handling the 24-hour stream retention window. Also covers Redpanda Enterprise features that pair with the destination topic, such as Iceberg Topics, Tiered Storage, Remote Read Replicas, Shadow Linking, and server-side Schema ID Validation.

Streams change data capture from MongoDB into Redpanda or Kafka using Redpanda Connect's mongodb_cdc input, built on MongoDB Change Streams over a replica set or sharded cluster. Use when configuring mongodb_cdc, setting up Change Streams, enabling an initial snapshot before live streaming, choosing between update_lookup and pre_and_post_images document modes, or checkpointing with resume tokens in a cache resource. Also covers Redpanda Enterprise destination features like Iceberg Topics, Tiered Storage, and server-side Schema ID Validation, all of which require a Redpanda Enterprise license.

Streams change data capture from MySQL or MariaDB into Redpanda or Kafka using the mysql_cdc input in Redpanda Connect, which replicates row-level changes via binlog. Use when configuring mysql_cdc, setting up binlog replication and an initial snapshot, wiring a checkpoint_cache resource, using AWS RDS or Aurora with IAM auth, or routing per-table CDC events to separate topics. Also covers Redpanda Enterprise destination features such as Iceberg Topics, Tiered Storage, and server-side Schema ID Validation, since mysql_cdc itself is an Enterprise connector requiring a Redpanda Enterprise license.

Streams change data capture from Oracle Database into Redpanda or Kafka using the oracledb_cdc input in Redpanda Connect, which reads redo logs via LogMiner. Use when configuring oracledb_cdc, enabling ARCHIVELOG mode and supplemental logging, granting LogMiner privileges, tuning SCN checkpointing, capturing LOBs, monitoring a pluggable database (CDB/PDB), or choosing a snapshot_mode. Also covers Redpanda Enterprise destination features such as Iceberg Topics, Tiered Storage, and server-side Schema ID Validation, all of which require a Redpanda Enterprise license.

Streams change data capture from PostgreSQL into Redpanda or Kafka using Redpanda Connect's postgres_cdc input, which reads the WAL through logical replication (pgoutput). Use when configuring postgres_cdc (formerly pg_stream), setting up wal_level=logical, replication slots, and publications, enabling an initial snapshot before streaming, or troubleshooting slot growth and WAL accumulation. Also covers Redpanda Enterprise destination features such as Iceberg Topics, Tiered Storage, and server-side Schema ID Validation, since postgres_cdc itself requires a Redpanda Enterprise license.

Streams change data capture and platform events from Salesforce into Redpanda or Kafka using Redpanda Connect's salesforce_cdc input, built on the Salesforce Pub/Sub gRPC API with OAuth Client Credentials auth. Use when configuring salesforce_cdc, subscribing to the CDC firehose or custom Platform Events, setting up a Connected App, enabling Change Data Capture for sObjects, or persisting replay-ID checkpoints in a durable cache resource. Also covers Redpanda Enterprise destination features such as Iceberg Topics, Tiered Storage, Cloud Topics, and server-side Schema ID Validation, since salesforce_cdc itself requires a Redpanda Enterprise license.

Streams change data capture from Google Cloud Spanner into Redpanda or Kafka using Redpanda Connect's gcp_spanner_cdc input, built on Spanner change streams with partition-aware watermarked delivery. Use when configuring gcp_spanner_cdc, creating a Spanner change stream, setting up GCP service-account credentials, filtering mod types, or tuning watermark and heartbeat settings. Also covers Redpanda Enterprise destination features such as Iceberg Topics, Tiered Storage, Cloud Topics, Remote Read Replicas, and Shadowing, since gcp_spanner_cdc itself requires a Redpanda Enterprise license.

Streams change data capture from Microsoft SQL Server into Redpanda or Kafka using Redpanda Connect's microsoft_sql_server_cdc input, which reads native CDC change tables in LSN order. Use when enabling SQL Server CDC capture instances, configuring microsoft_sql_server_cdc, setting up an initial snapshot alongside live streaming, tuning LSN checkpoint storage, or troubleshooting LSN gaps and missing Agent jobs. Also covers Redpanda Enterprise destination features such as Iceberg Topics, Tiered Storage, and server-side Schema ID Validation, since microsoft_sql_server_cdc itself requires a Redpanda Enterprise license.

Guides setup and operation of the tigerbeetle_cdc input in Redpanda Connect, which streams change data capture from a TigerBeetle financial transactions database into Redpanda or Kafka. Use when configuring tigerbeetle_cdc, connecting to a TigerBeetle cluster, choosing a persistent progress_cache for checkpointing, filtering settled vs pending transfers, routing by ledger, or resolving cgo-build or duplicate-event issues. Unlike the other CDC inputs, tigerbeetle_cdc is a certified community connector and does not require an Enterprise license, though it can still land data on Enterprise destination features like Iceberg Topics and Tiered Storage.

Diagnoses, validates, and monitors Redpanda Connect pipelines. Use when a Redpanda Connect pipeline fails, stalls, drops messages, or won't start; when linting a config with rpk connect lint; tuning log level or format; reading Connect metrics (Prometheus, statsd, json_api) or traces (OpenTelemetry); checking /ready and /ping health endpoints; running a dry-run connection test; or diagnosing connector failures including auth errors, TLS errors, Enterprise license errors, backpressure, and checkpoint/cache issues. Covers Redpanda Connect enterprise features and their config keys: enterprise connectors including all CDC inputs (postgres_cdc, mysql_cdc, mongodb_cdc, oracledb_cdc logminer block), connector allow/deny lists, secrets management URNs, the redpanda: configuration service block, and FIPS compliance — all requiring an Enterprise license.

Install and configure the rpk CLI, the single binary for all Redpanda operations, including installing it on macOS, Linux, and Docker, and setting up profiles (rpk.yaml) and connection settings. Use when installing rpk, creating or switching rpk profiles, configuring broker/admin/registry endpoints or SASL/TLS credentials, using -X flags or RPK_ environment variables, connecting to self-hosted or Redpanda Cloud clusters, or checking/applying an Enterprise license. This is the hub skill for command-group discovery; for topic/cluster/security/cloud/registry/transform/debug work, see the matching /redpanda:rpk-* skill.

Authenticate and manage Redpanda Cloud from the CLI with the `rpk cloud` command group: SSO or client-credential login, cloud auth management, cluster selection, resource groups, BYOC plugin provisioning, and the MCP server that exposes Redpanda Cloud to AI agents. Use when logging in to Redpanda Cloud, running `rpk cloud cluster select`, managing resource groups or BYOC installs, or configuring `rpk cloud mcp` for Claude Desktop or Claude Code. For enterprise data features on a selected Cloud cluster, such as Tiered Storage, Cloud Topics, Iceberg, or RBAC, see /redpanda:rpk-cluster. Related details also live in /redpanda:rpk-topic and /redpanda:rpk-security.

Operate a Redpanda cluster's health, brokers, configuration, and maintenance from the CLI using the `rpk cluster` command group, the front-end to the Admin API. Use when checking cluster health or metadata, decommissioning or recommissioning brokers, getting or setting cluster config properties, balancing or moving partitions, enabling maintenance mode, managing client quotas, running self-tests, or managing the Enterprise license. Also covers enabling Enterprise differentiators through cluster config: Continuous Data Balancing, Tiered Storage, and Iceberg Topics. It also manages Shadow Linking DR, Remote Read Replicas, Audit Logging, and Leadership Pinning. For single-node process lifecycle (start/stop, tuning, redpanda.yaml), see /redpanda:rpk-redpanda; for diagnostics bundles, see /redpanda:rpk-debug.

Collect local and remote Redpanda diagnostics bundles and gather broker process info using the `rpk debug` command group. Use when collecting a debug bundle for Redpanda support or self-triage, generating a cluster-wide bundle via the Admin API with `rpk debug remote-bundle`, or troubleshooting a cluster that needs captured logs, metrics, and profiles. Also covers triaging Enterprise features from a bundle, such as Tiered Storage, RBAC, and Schema ID Validation, plus checking license status. Applies to self-managed deployments only (Linux hosts and Kubernetes); Redpanda Cloud clusters don't support debug bundles — for client-side triage there, see /redpanda:rpk-cluster and /redpanda:rpk-topic instead.

Use rpk group to list and describe consumer groups, inspect lag and members, reset or seek consumer group offsets, delete groups, and delete committed offsets. Use when: inspecting consumer group lag and membership; resetting or seeking consumer group offsets (to earliest/latest/timestamp/specific offset or to another group's commits); stopping consumers and rewinding after a bad deploy; deleting a consumer group; deleting committed offsets for specific topic-partitions; reading CURRENT-OFFSET, LOG-END-OFFSET, and LAG columns; spotting stuck or over-lagged consumer groups; using rpk group list, describe, seek, offset-delete, or delete subcommands; verifying or repairing consumer group offsets after a Shadow Linking disaster-recovery failover (Enterprise offset-preserving replication, consumer_offset_sync_options, group_filters, offset clamping); and authorizing rpk group operations via GROUP-resource ACLs, Enterprise RBAC roles (rpk security role / --allow-role), or Enterprise GBAC OIDC Group: principals.

Operate a self-managed Redpanda broker process and node from the CLI using the `rpk redpanda` command family and `rpk iotune`: node lifecycle (start/stop), production vs. development mode, kernel autotuning, node configuration (redpanda.yaml), and per-node Admin API operations including broker decommission/recommission. Self-managed only — these commands operate the broker process on the node itself and don't apply to Redpanda Cloud clusters. Use when starting or stopping a broker, bootstrapping redpanda.yaml, switching node mode, running the autotuner, benchmarking disk I/O with iotune, or decommissioning a broker via the admin listener. For cluster-wide operations (cluster config, partition movement, maintenance mode, cluster health), see /redpanda:rpk-cluster.

Manage schemas, subjects, compatibility levels, modes, and contexts in the Redpanda Schema Registry via the `rpk registry` CLI (alias `rpk sr`). Use when registering or fetching Avro/Protobuf/JSON schemas, listing or deleting subjects and schema versions, setting BACKWARD/FORWARD/FULL/NONE compatibility levels, changing registry mode (READONLY/READWRITE/IMPORT), or working with schema contexts for namespace isolation. Also covers Enterprise-licensed registry features: Schema Registry Authorization and server-side Schema ID Validation. For granting ACLs or RBAC on registry resources, see /redpanda:rpk-security; for producing or consuming records with schema encoding/decoding, see /redpanda:rpk-topic.

Manage Redpanda authentication and authorization from the CLI using the `rpk security` command group (`user`, `acl`, `role`, `secret` subcommands; `rpk acl` is a deprecated alias). Use when creating or deleting SASL/SCRAM users, granting or revoking Kafka ACLs on topics, groups, the cluster, or transactional IDs, creating or assigning RBAC roles (Enterprise), managing secrets for Redpanda Cloud clusters, or configuring SASL/PLAIN, OAUTHBEARER/OIDC, GSSAPI/Kerberos, or mTLS principal mapping. Also covers Enterprise security config keys like Audit Logging, GBAC, Schema ID Validation, and FIPS mode, plus bootstrapping a new cluster with a superuser. For Schema Registry-specific ACL patterns, see /redpanda:rpk-registry; for the Enterprise license itself, see /redpanda:rpk-cluster.

Manage Redpanda topics and produce or consume records from the CLI using the `rpk topic` command group. Use when creating or deleting topics with custom partitions, replication, or configs; altering topic configs or retention; adding partitions; trimming or deleting records; or using `rpk topic produce`/`consume` with the percent-escape format language, keys, headers, offsets, and schema-registry encoding/decoding. Also covers Enterprise topic properties, including Tiered Storage, Cloud Topics, Iceberg Topics, Remote Read Replicas, Leader Pinning, and server-side Schema ID Validation. Includes Redpanda Cloud notes on auth and per-tier limits; for Cloud login and profile setup, see /redpanda:rpk-cloud, and for Schema Registry details, see /redpanda:rpk-registry.

Build, deploy, and manage Redpanda Data Transforms, WebAssembly functions that run inside the broker to transform records from an input topic to one or more output topics. Use when writing or deploying a transform in Go (TinyGo), Rust, JavaScript, or TypeScript; using `rpk transform init/build/deploy/list/logs/pause/resume/delete`; enabling the `data_transforms_enabled` cluster config; or wiring transform output to Enterprise features like Iceberg Topics, Tiered Storage, or Schema ID Validation. Runs on BYOC and Dedicated Cloud clusters, not Serverless. For multi-record joins or aggregations, use Redpanda Connect instead; for enabling the cluster config itself, see /redpanda:rpk-cluster.

Redpanda SQL (Oxla), the distributed PostgreSQL-wire-compatible columnar analytical database, for writing and running SQL queries. Use when connecting via psql or a PostgreSQL driver, creating tables and schemas, loading data with COPY (CSV/Parquet/ORC, S3), running analytical queries with joins, aggregate and window functions, CTEs, or set operations, creating views, explaining a query plan with EXPLAIN, or working with Oxla's data types and role-based grants. Also covers Oxla's external-source integration for querying Redpanda/Kafka topics and Apache Iceberg tables directly with SQL, including CREATE REDPANDA/KAFKA CATALOG, CREATE ICEBERG CATALOG, CREATE STORAGE, and mapping PostgreSQL SQL features to what Oxla supports.

Configures and operates an Oxla cluster: YAML config and OXLA__ environment-variable overrides, node ports, roles, and leader election; storage backends (local/S3/GCS/Azure); memory limits; access control; and TLS/OIDC, plus the HTTP-based ConnectRPC admin service and Prometheus metrics endpoint. Also covers Oxla's lakehouse and streaming differentiators: Apache Iceberg REST catalogs, transparent Redpanda/Kafka integration (CREATE REDPANDA CATALOG), object-storage connections, and security features such as SCRAM passwords and secret encryption. Use when configuring an Oxla cluster, choosing a storage backend, setting up Iceberg catalogs or querying Redpanda/Kafka topics as tables, enabling TLS/OIDC, changing log levels via the admin API, or deploying Oxla with Docker.

Diagnose and observe an Oxla distributed analytical database using system catalog tables (system_nodes/system_queries/system_transactions/system_storage/system_execs), Prometheus metrics, runtime log-level changes, and troubleshooting workflows for slow queries, node health, and memory/OOM pressure. Also covers debugging Oxla's external data sources — the Redpanda/Kafka ingestion path and the Apache Iceberg catalog Oxla reads — including correlating stale-data or missing-row symptoms with the upstream Redpanda Iceberg Topics integration. Use when troubleshooting Oxla query performance or node health, auditing storage connections and active transactions, or diagnosing Kafka/Redpanda ingestion stalls and stale or missing rows in Iceberg tables.

Query external data from Oxla — Kafka topics via catalogs, Apache Iceberg tables, and S3/GCS/Azure parquet/ORC files — alongside native Oxla tables. Use when querying Kafka topics with CREATE KAFKA CATALOG or CREATE REDPANDA CATALOG, reading Apache Iceberg tables with the catalog=>path.table syntax, loading or exporting parquet/ORC files with COPY FROM/TO, joining external data with native tables, or inspecting external metadata via system.kafka_connections, system.kafka_sources, system.iceberg_catalogs, and system.iceberg_tables. Also covers the Redpanda Enterprise features that produce the data Oxla reads — Iceberg Topics, Tiered Storage, and Server-Side Schema ID Validation.

Teaches how to use the Redpanda broker through its Kafka-compatible API: producing and consuming data, consumer groups, idempotent producers, exactly-once semantics and transactions, topic creation and configuration, Kafka client compatibility, and tiered storage. Use when producing to or consuming from Redpanda, creating or configuring topics, working with consumer groups or offsets, choosing a Kafka client library (any Kafka client), understanding Kafka protocol compatibility, or troubleshooting acks, compression, batching, follower fetching, or rack awareness. Also covers Redpanda Enterprise topic/broker features requiring a license: Iceberg Topics, Cloud Topics, Continuous Data Balancing, Shadow Linking for cross-cluster disaster recovery, Remote Read Replicas, Leader Pinning, and Schema ID Validation.

Operate a Redpanda cluster via its HTTP Admin API on port 9644 (base path /v1) using Basic, Bearer, or mTLS authentication. Covers endpoint groups for brokers/partitions, cluster config/health, features/licensing/transactions, cloud storage, security/RBAC, and debug bundles, plus ConnectRPC endpoints. Also covers configuring Redpanda's Enterprise features through the Admin API: Tiered Storage/Cloud Topics, Iceberg Topics/Continuous Data Balancing, Shadow Linking for cross-cluster disaster recovery, Remote Read Replicas/Topic Recovery/Audit Logging/RBAC, OIDC/OAuthBearer/Kerberos authentication, FIPS mode/Schema ID Validation, and Leadership Pinning (most require an Enterprise license). Use when calling the Admin API directly over HTTP, decommissioning or recovering brokers/partitions, checking cluster health, managing licensing or security, or scripting cluster operations without rpk.

Diagnoses a Redpanda broker or cluster using debug bundles, metrics endpoints, logs, CPU profiling, partition and raft health checks, and triage playbooks. Use when a broker is unhealthy, crashing/lagging/under-replicated, or out of disk; when collecting a debug bundle for Redpanda support; when reading /public_metrics or /metrics; when triaging cluster health/partition movement or raft recovery issues; when consumer lag is growing unexpectedly; or when running rpk debug bundle or rpk debug remote-bundle. Also covers debugging Enterprise features and their health signals — Tiered Storage/Continuous Data Balancing, Cloud Topics/Iceberg Topics, Shadow Linking/Remote Read Replicas, Leadership Pinning, and Audit Logging — and checking license status via rpk cluster license info.