filesystem-pipeline
v0.1.1Load files (CSV, Parquet, JSONL, or custom) from local disk, S3, GCS, Azure, or SFTP into a destination using dlt's filesystem source.
By ScaleVector GmbHLicense: https://github.com/dlt-hub/dlthub-ai-workbench/blob/master/LICENSE59 GitHub starsUpdated 5 days ago
Directory evidence
- Runtimes
- Claude Code
- Parsed components
- 3 skill or MCP entries
- Source updated
- Sep 18, 2026
- Manifest status
- Canonical path parsed
The directory validates manifest shape and source location. It does not execute the plugin or provide a security endorsement. Review the indexing methodology →
Install filesystem-pipeline for Claude Code
claude plugin marketplace add IchenDEV/agent-plugin-mkt
claude plugin marketplace update agent-plugin-marketplace
claude plugin install filesystem-pipeline@agent-plugin-marketplacePaste and run these commands in a terminal with Claude Code. They add and refresh the PluginsMP catalog, then install this plugin.
The installer fetches third-party code from the source repository shown on this page. This directory validates manifest structure and source location, but does not perform a security audit; review the manifest, components, and source before installing.
Get the source manually
git clone https://github.com/dlt-hub/dlthub-ai-harnessClone the source repository, then follow its setup instructions to add the plugin to a compatible client. The plugin root is workbench/filesystem-pipeline/.
Plugin files
├── .claude-plugin/plugin.json├── skills/add-incremental-loading/SKILL.md├── skills/create-filesystem-pipeline/SKILL.md└── skills/optimize-filesystem-performance/SKILL.md
Included Skills3
Add incremental loading to a dlt filesystem pipeline — filter files by modification date, optionally filter records by a timestamp column, and switch to merge with a primary key to deduplicate updated records, so each run only reads new or modified files. Use after create-filesystem-pipeline produces a working replace-mode pipeline, or when the user wants to set up merge/dedup or move a pipeline from replace to incremental. For speed/memory tuning (faster reader, chunked streaming, parallel reads, glob narrowing) use optimize-filesystem-performance instead.
Create a dlt filesystem pipeline that reads files (CSV, Parquet, JSONL, or custom) from local disk, S3, GCS, Azure, or SFTP into a destination. Use for the filesystem core source. Not for REST APIs (rest_api) or databases (sql_database).
Speed up a dlt filesystem pipeline. Use when reading files (CSV, Parquet, JSONL) from local disk, S3, GCS, Azure, or SFTP is slow or memory-heavy and the user wants to optimize it — choose a faster reader, read files in parallel, narrow the glob, or stream in chunks. For first-time incremental/merge setup, or to skip files already loaded in previous runs, use add-incremental-loading instead.
Plugin manifests1
{
"name": "filesystem-pipeline",
"description": "Load files (CSV, Parquet, JSONL, or custom) from local disk, S3, GCS, Azure, or SFTP into a destination using dlt's filesystem source.",
"version": "0.1.1",
"author": {
"name": "ScaleVector GmbH"
},
"homepage": "https://dlthub.com/docs",
"repository": "https://github.com/dlt-hub/dlthub-ai-workbench",
"license": "https://github.com/dlt-hub/dlthub-ai-workbench/blob/master/LICENSE",
"keywords": [
"dlthub",
"filesystem",
"s3",
"gcs",
"azure",
"sftp",
"csv",
"parquet",
"jsonl"
]
}For maintainers
If you maintain this plugin, link to this source-backed listing from your README so users can review its manifest and indexed components.
[filesystem-pipeline on Agent Plugins Marketplace](https://pluginsmp.com/plugins/filesystem-pipeline)