Skip to content

Commit 55f0ca1

Browse files
committed
feat: Implement support for a new Azure Blob connector
Issue - #1957 Summary - Expanded the Azurite seed script to cover multi-container seeding, Office document formats (docx, xlsx, pptx, csv, html), and an in-place `--update` mode for re-ingestion / change-detection testing. - Standardized bucket/container restriction label copy across S3, IBM COS, and Azure Blob settings forms. - Improved `.env.example` Azurite config docs and removed redundant echo output from the `azurite-up` Makefile target. Developer Tooling — `scripts/connectors/azure/seed_azurite.py` - Split single `openrag-test` container into two: `openrag-test-1` (plain text, markdown, PDF) and `openrag-test-2` (Office/web formats). - Added pure-Python OOXML builders (`_make_docx`, `_make_xlsx`, `_make_pptx`) using `zipfile` — no external dependencies. - Added `BLOBS_1_V2` fixture set (VERSION_2 tagged content) to exercise re-ingestion and change-detection flows. - Added `--update` CLI flag to overwrite specific blobs in place without recreating containers. - Refactored seeding logic into `_seed_container` and `_update_blobs` helpers. Configuration — `.env.example` / `Makefile` - Expanded Azurite account-key mode docs with explicit per-variable format for both host-side and in-compose backends. - Removed redundant configuration echo lines from `azurite-up` target (now documented in `.env.example`). Frontend — Connector Settings Forms - Renamed label "Restrict to Specific Buckets/Containers" → "Restrict Ingestion to Buckets/Containers" across S3, IBM COS, and Azure Blob settings forms. - Shortened helper text to "(optional)" (removed the redundant "leave all unchecked" parenthetical).
1 parent d6596e5 commit 55f0ca1

6 files changed

Lines changed: 492 additions & 39 deletions

File tree

.env.example

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -180,9 +180,14 @@ AWS_SECRET_ACCESS_KEY=
180180
# AZURE_STORAGE_CONNECTION_STRING=
181181
#
182182
# Account-key mode (set account name + key; endpoint optional):
183-
# - Azurite (host-side backend): endpoint http://127.0.0.1:10000/devstoreaccount1
184-
# - Azurite (in-compose backend): endpoint http://azurite:10000/devstoreaccount1
185-
# (use the well-known Azurite key: Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KnufldCDs==)
183+
# - Azurite (host-side backend):
184+
# - AZURE_STORAGE_ACCOUNT_NAME=devstoreaccount1
185+
# - AZURE_STORAGE_ACCOUNT_KEY=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==
186+
# - AZURE_STORAGE_ENDPOINT=http://127.0.0.1:10000/devstoreaccount1
187+
# - Azurite (in-compose backend):
188+
# - AZURE_STORAGE_ACCOUNT_NAME=devstoreaccount1
189+
# - AZURE_STORAGE_ACCOUNT_KEY=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==
190+
# - AZURE_STORAGE_ENDPOINT=http://azurite:10000/devstoreaccount1
186191
# AZURE_STORAGE_ACCOUNT_NAME=
187192
# AZURE_STORAGE_ACCOUNT_KEY=
188193
# AZURE_STORAGE_ENDPOINT=

Makefile

Lines changed: 0 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -671,12 +671,6 @@ azurite-up: ## Start Azurite (local Azure Blob emulator) for connector testing
671671
@echo "$(YELLOW)Starting Azurite (Azure Blob emulator)...$(NC)"
672672
$(COMPOSE_CMD) --profile azurite up -d azurite
673673
@echo "$(PURPLE)Azurite started on http://localhost:10000 (account: devstoreaccount1).$(NC)"
674-
@echo " $(CYAN)Host-side backend: AZURE_STORAGE_CONNECTION_STRING=\"UseDevelopmentStorage=true\"$(NC)"
675-
@echo " $(CYAN)In-compose backend: use account name + key + endpoint mode:$(NC)"
676-
@echo " $(CYAN) AZURE_STORAGE_ACCOUNT_NAME=devstoreaccount1$(NC)"
677-
@echo " $(CYAN) AZURE_STORAGE_ACCOUNT_KEY=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KnufldCDs== # well-known Azurite key$(NC)"
678-
@echo " $(CYAN) AZURE_STORAGE_ENDPOINT=http://azurite:10000/devstoreaccount1$(NC)"
679-
@echo " $(CYAN)Set OPENRAG_DEV_AZURE_BLOB=true in .env to enable the connector without IBM_AUTH_ENABLED$(NC)"
680674

681675
azurite-down: ## Stop Azurite emulator
682676
@echo "$(YELLOW)Stopping Azurite...$(NC)"

frontend/app/settings/_components/s3-settings-form.tsx

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -170,9 +170,9 @@ export function S3SettingsForm({
170170
<div className="space-y-2">
171171
<div className="flex items-center justify-between">
172172
<Label className="text-sm font-medium">
173-
Restrict to Specific Buckets
173+
Restrict Ingestion to Buckets
174174
<span className="ml-1 text-muted-foreground font-normal">
175-
(optional — leave all unchecked to allow all buckets)
175+
(optional)
176176
</span>
177177
</Label>
178178
{buckets.length > 1 && (

frontend/enhancements/connectors/azure-blob/settings-form.tsx

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -217,9 +217,9 @@ export function AzureBlobSettingsForm({
217217
<div className="space-y-2">
218218
<div className="flex items-center justify-between">
219219
<Label className="text-sm font-medium">
220-
Restrict to Specific Containers
220+
Restrict Ingestion to Containers
221221
<span className="ml-1 text-muted-foreground font-normal">
222-
(optional — leave all unchecked to allow all containers)
222+
(optional)
223223
</span>
224224
</Label>
225225
{containers.length > 1 && (

frontend/enhancements/connectors/ibm-cos/settings-form.tsx

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -275,9 +275,9 @@ export function IBMCOSSettingsForm({
275275
<div className="space-y-2">
276276
<div className="flex items-center justify-between">
277277
<Label className="text-sm font-medium">
278-
Restrict to Specific Buckets
278+
Restrict Ingestion to Buckets
279279
<span className="ml-1 text-muted-foreground font-normal">
280-
(optional — leave all unchecked to allow all buckets)
280+
(optional)
281281
</span>
282282
</Label>
283283
{buckets.length > 1 && (

0 commit comments

Comments
 (0)