Zerobus

[amotl]

About

Zerobus Ingest is a push-based ingestion API that writes data directly into Unity Catalog Delta tables. It is a serverless connector that automatically scales to handle incoming connections. It does not require configuring partitions or managing brokers. With Zerobus Ingest, your "scaling strategy" is to open more connections. This streamlines ingestion workflows by eliminating the need for message bus infrastructure.

Evaluation

Can Zerobus only be used with Databricks, or is it a technology that can also be used independently?

References

• Delta Lake #104
• https://docs.databricks.com/aws/en/ingestion/zerobus-overview
• https://github.com/databricks/zerobus-sdk
• examples/sync_example_json.py
• feat(databricks): Support ingestion via Zerobus dlt-hub/dlt#3874
• feat: Databricks Zerobus loading dlt-hub/dlt#3904
• Feature: Databricks Zerobus insert_api support Glitni/dlt-saga#39