Skip to content
This repository was archived by the owner on Apr 3, 2026. It is now read-only.

BlackRoad-Labs/blackroad-data-pipeline

ulackroad data pipeline

Part of BlackRoad OS — Sovereign Computing for Everyone

BlackRoad OS BlackRoad-Labs

ulackroad data pipeline is part of the BlackRoad OS ecosystem — a sovereign, distributed operating system built on edge computing, local AI, and mesh networking by BlackRoad OS, Inc.

BlackRoad Ecosystem

Org Focus
BlackRoad OS Core platform
BlackRoad OS, Inc. Corporate
BlackRoad AI AI/ML
BlackRoad Hardware Edge hardware
BlackRoad Security Cybersecurity
BlackRoad Quantum Quantum computing
BlackRoad Agents AI agents
BlackRoad Network Mesh networking

Website: blackroad.io | Chat: chat.blackroad.io | Search: search.blackroad.io


ETL data pipeline framework

Part of the BlackRoad OS ecosystem — BlackRoad-Labs


blackroad-data-pipeline

CI Python License BlackRoad OS BlackRoad Labs

BlackRoad Data Pipeline — ETL data pipeline framework for the BlackRoad OS platform, built by BlackRoad-Labs.

Part of the BlackRoad infrastructure ecosystem: BlackRoad AI · BlackRoad Cloud · BlackRoad OS · BlackRoad Security · BlackRoad Quantum · BlackRoad Foundation · BlackRoad Education · BlackRoad Gov · BlackRoad Hardware · BlackRoad Interactive · BlackRoad Media · BlackRoad Studio · BlackRoad Ventures · BlackRoad Archive · Blackbox Enterprises

Features

  • Sources: CSV, JSON, SQLite, inline data
  • Transforms: filter, map, aggregate, join, sort, deduplicate, select, rename
  • Sinks: stdout, JSON file, CSV file, SQLite database
  • Dependency graph execution with ordered transform chaining
  • Schema validation — auto-infer field types from live data

Usage

python main.py create "My Pipeline"
python main.py run <pipeline-id>
python main.py list
python main.py validate <source-id>

Testing

python -m pytest test_data_pipeline.py -v

Directory

The index.html in this repository is a reusable, SEO-optimized BlackRoad Infrastructure Directory page listing the full BlackRoad GitHub enterprise, all 15 GitHub organizations, and all 19 registered domains.

BlackRoad GitHub Enterprise: github.com/enterprises/blackroad-os

BlackRoad Organizations (15): Blackbox-Enterprises · BlackRoad-AI · BlackRoad-Archive · BlackRoad-Cloud · BlackRoad-Education · BlackRoad-Foundation · BlackRoad-Gov · BlackRoad-Hardware · BlackRoad-Interactive · BlackRoad-Labs · BlackRoad-Media · BlackRoad-OS · BlackRoad-Security · BlackRoad-Studio · BlackRoad-Ventures

BlackRoad Domains (19): blackboxprogramming.io · blackroad.company · blackroad.io · blackroad.me · blackroad.network · blackroad.systems · blackroadai.com · blackroadinc.us · blackroadqi.com · blackroadquantum.com · blackroadquantum.info · blackroadquantum.net · blackroadquantum.shop · blackroadquantum.store · lucidia.earth · lucidia.studio · lucidiaqi.com · roadchain.io · roadcoin.io


BlackRoad OS, Inc. — Delaware C-Corp

About

ulackroad data pipeline — Part of the BlackRoad OS ecosystem. Sovereign computing, edge AI, mesh networking. blackroad.io

Topics

Resources

License

Code of conduct

Contributing

Security policy

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors