feat: Ingestion logs enhancement#1598
Open
ricofurtado wants to merge 42 commits into
Open
Conversation
github-actions
Bot
added
the
backend
ricofurtado
changed the title
Contributor
WalkthroughThis PR adds ingestion-jobs FastAPI endpoints and Pydantic schemas, implements connector orphan reconciliation and sync-preview endpoints, introduces a Docling two‑phase ingest flow with a polling coordinator, adds operator CRD and deletion/finalizer handling plus env-hash rollout triggers, rewires many frontend upload/sync UI flows (preview/confirm, duplicate handling, assistant feedback), and expands test coverage and CI/workflows. ChangesIngestion Jobs API
(Other notable checkpoints: connector orphan-reconciliation and preview endpoints in Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
Suggested reviewers
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
⚔️ Resolve merge conflicts
|
github-actions
Bot
added
enhancement
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 5
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/api/v1/ingestion_jobs.py`:
- Around line 546-587: Add pagination to the list_ingestion_jobs endpoint:
modify list_ingestion_jobs to accept limit and offset Query params (e.g., limit
with default 50 and bounds, offset default 0), compute total =
len(filtered_jobs) after applying filters on MOCK_JOBS, then slice jobs =
jobs[offset: offset + limit], build summaries from the sliced jobs and return an
IngestionJobListResponse containing total and the paginated jobs (and optionally
include limit/offset/has_more fields in the response model if needed); update
references in the function (list_ingestion_jobs, MOCK_JOBS, IngestionJobSummary,
IngestionJobListResponse) and keep get_api_key_user_async dependency unchanged.
- Around line 546-587: The endpoint list_ingestion_jobs currently returns a raw
JSONResponse and lacks a declared response model; update the FastAPI route
signature to include response_model=IngestionJobListResponse, and return the
Pydantic model instance (IngestionJobListResponse(total=..., jobs=...)) directly
instead of JSONResponse(response.model_dump()); keep the filtering logic over
MOCK_JOBS and construction of IngestionJobSummary unchanged, but ensure the
function signature still depends on get_api_key_user_async so OpenAPI and
response validation use IngestionJobListResponse and IngestionJobSummary types.
- Around line 546-599: Both endpoints accept user: User =
Depends(get_api_key_user_async) but never enforce ownership; update
list_ingestion_jobs to filter MOCK_JOBS by job.user_id == user.user_id before
building IngestionJobSummary (so only the authenticated user's jobs are
returned) and update get_ingestion_job to verify the retrieved job's user_id
matches user.user_id and return 404 if not, using the existing MOCK_JOBS,
list_ingestion_jobs, get_ingestion_job, and User.user_id symbols to locate the
changes.
- Around line 590-599: The get_ingestion_job endpoint currently returns a
JSONResponse directly and lacks a response_model; update the function signature
for get_ingestion_job to declare the proper response_model (the Pydantic model
used for ingestion jobs) and keep the Depends(User=get_api_key_user_async) auth,
then replace the JSONResponse-not-found branch with raise
HTTPException(status_code=404, detail="Ingestion job not found") and return the
job model instance (or job.model_dump() if your response_model expects dict) so
FastAPI/OpenAPI can validate and document the response; ensure HTTPException is
imported and reference MOCK_JOBS, get_ingestion_job, and get_api_key_user_async
when making the changes.
In `@src/app/routes/public_v1.py`:
- Around line 91-103: The two route registrations that call
v1_ingestion_jobs.list_ingestion_jobs and v1_ingestion_jobs.get_ingestion_job
currently use tags=["ingestion"]; update them to use tags=["public"] to match
the rest of this registration file (or, if a separate "ingestion" grouping is
intentional, add a short comment explaining the reason). Ensure you change the
tags argument on both add_api_route calls so OpenAPI grouping is consistent.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: d90d6550-202f-492a-a72f-d67cff507ea6
📒 Files selected for processing (2)
src/api/v1/ingestion_jobs.pysrc/app/routes/public_v1.py
Comment on lines
+546
to
+587
| async def list_ingestion_jobs( | ||
| status: Optional[str] = Query( | ||
| None, description="Filter by status: completed, failed, running, pending" | ||
| ), | ||
| component: Optional[str] = Query( | ||
| None, description="Filter by component: openrag, docling, langflow, opensearch" | ||
| ), | ||
| actionability: Optional[str] = Query( | ||
| None, | ||
| description="Filter by actionability: USER_ACTIONABLE, RETRYABLE, ADMIN_ACTIONABLE, DEVELOPER_ACTIONABLE", | ||
| ), | ||
| user: User = Depends(get_api_key_user_async), | ||
| ): | ||
| """List ingestion jobs with optional filtering.""" | ||
| jobs = list(MOCK_JOBS.values()) | ||
|
|
||
| if status: | ||
| jobs = [j for j in jobs if j.status == status] | ||
| if component: | ||
| jobs = [j for j in jobs if j.component == component] | ||
| if actionability: | ||
| jobs = [j for j in jobs if j.actionability == actionability] | ||
|
|
||
| summaries = [ | ||
| IngestionJobSummary( | ||
| ingestion_job_id=j.ingestion_job_id, | ||
| file_name=j.file_name, | ||
| source_type=j.source_type, | ||
| status=j.status, | ||
| component=j.component, | ||
| phase=j.phase, | ||
| error_code=j.error_code, | ||
| actionability=j.actionability, | ||
| retryable=j.retryable, | ||
| user_title=j.user_title, | ||
| created_at=j.created_at, | ||
| ) | ||
| for j in jobs | ||
| ] | ||
|
|
||
| response = IngestionJobListResponse(total=len(summaries), jobs=summaries) | ||
| return JSONResponse(response.model_dump()) |
Contributor
There was a problem hiding this comment.
🛠️ Refactor suggestion | 🟠 Major | 🏗️ Heavy lift
Consider adding pagination for scalability.
List endpoints in public APIs should typically support pagination to handle large datasets efficiently. Without pagination, this endpoint could return hundreds or thousands of jobs, leading to performance issues and poor user experience.
💡 Suggested pagination parameters
async def list_ingestion_jobs(
status: Optional[str] = Query(None, description="Filter by status: completed, failed, running, pending"),
component: Optional[str] = Query(None, description="Filter by component: openrag, docling, langflow, opensearch"),
actionability: Optional[str] = Query(None, description="Filter by actionability: USER_ACTIONABLE, RETRYABLE, ADMIN_ACTIONABLE, DEVELOPER_ACTIONABLE"),
limit: int = Query(50, ge=1, le=100, description="Maximum number of jobs to return"),
offset: int = Query(0, ge=0, description="Number of jobs to skip"),
user: User = Depends(get_api_key_user_async),
) -> IngestionJobListResponse:
"""List ingestion jobs with optional filtering and pagination."""
jobs = list(MOCK_JOBS.values())
# Apply filters
if status:
jobs = [j for j in jobs if j.status == status]
if component:
jobs = [j for j in jobs if j.component == component]
if actionability:
jobs = [j for j in jobs if j.actionability == actionability]
# Apply pagination
total = len(jobs)
jobs = jobs[offset:offset + limit]
summaries = [
IngestionJobSummary(
ingestion_job_id=j.ingestion_job_id,
file_name=j.file_name,
source_type=j.source_type,
status=j.status,
component=j.component,
phase=j.phase,
error_code=j.error_code,
actionability=j.actionability,
retryable=j.retryable,
user_title=j.user_title,
created_at=j.created_at,
)
for j in jobs
]
return IngestionJobListResponse(total=total, jobs=summaries)Note: The IngestionJobListResponse model may need additional fields like limit, offset, or has_more to support pagination metadata.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/api/v1/ingestion_jobs.py` around lines 546 - 587, Add pagination to the
list_ingestion_jobs endpoint: modify list_ingestion_jobs to accept limit and
offset Query params (e.g., limit with default 50 and bounds, offset default 0),
compute total = len(filtered_jobs) after applying filters on MOCK_JOBS, then
slice jobs = jobs[offset: offset + limit], build summaries from the sliced jobs
and return an IngestionJobListResponse containing total and the paginated jobs
(and optionally include limit/offset/has_more fields in the response model if
needed); update references in the function (list_ingestion_jobs, MOCK_JOBS,
IngestionJobSummary, IngestionJobListResponse) and keep get_api_key_user_async
dependency unchanged.
🛠️ Refactor suggestion | 🟠 Major | ⚡ Quick win
Declare response model and return Pydantic model directly.
FastAPI endpoints should declare the response model for automatic OpenAPI documentation generation and response validation. Additionally, return the Pydantic model directly instead of manually converting to JSONResponse.
♻️ Proposed refactor
async def list_ingestion_jobs(
status: Optional[str] = Query(
None, description="Filter by status: completed, failed, running, pending"
),
component: Optional[str] = Query(
None, description="Filter by component: openrag, docling, langflow, opensearch"
),
actionability: Optional[str] = Query(
None,
description="Filter by actionability: USER_ACTIONABLE, RETRYABLE, ADMIN_ACTIONABLE, DEVELOPER_ACTIONABLE",
),
user: User = Depends(get_api_key_user_async),
-):
+) -> IngestionJobListResponse:
"""List ingestion jobs with optional filtering."""
jobs = list(MOCK_JOBS.values())
if status:
jobs = [j for j in jobs if j.status == status]
if component:
jobs = [j for j in jobs if j.component == component]
if actionability:
jobs = [j for j in jobs if j.actionability == actionability]
summaries = [
IngestionJobSummary(
ingestion_job_id=j.ingestion_job_id,
file_name=j.file_name,
source_type=j.source_type,
status=j.status,
component=j.component,
phase=j.phase,
error_code=j.error_code,
actionability=j.actionability,
retryable=j.retryable,
user_title=j.user_title,
created_at=j.created_at,
)
for j in jobs
]
- response = IngestionJobListResponse(total=len(summaries), jobs=summaries)
- return JSONResponse(response.model_dump())
+ return IngestionJobListResponse(total=len(summaries), jobs=summaries)As per coding guidelines, FastAPI routes should use proper response model typing.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/api/v1/ingestion_jobs.py` around lines 546 - 587, The endpoint
list_ingestion_jobs currently returns a raw JSONResponse and lacks a declared
response model; update the FastAPI route signature to include
response_model=IngestionJobListResponse, and return the Pydantic model instance
(IngestionJobListResponse(total=..., jobs=...)) directly instead of
JSONResponse(response.model_dump()); keep the filtering logic over MOCK_JOBS and
construction of IngestionJobSummary unchanged, but ensure the function signature
still depends on get_api_key_user_async so OpenAPI and response validation use
IngestionJobListResponse and IngestionJobSummary types.
Comment on lines
+546
to
+599
| async def list_ingestion_jobs( | ||
| status: Optional[str] = Query( | ||
| None, description="Filter by status: completed, failed, running, pending" | ||
| ), | ||
| component: Optional[str] = Query( | ||
| None, description="Filter by component: openrag, docling, langflow, opensearch" | ||
| ), | ||
| actionability: Optional[str] = Query( | ||
| None, | ||
| description="Filter by actionability: USER_ACTIONABLE, RETRYABLE, ADMIN_ACTIONABLE, DEVELOPER_ACTIONABLE", | ||
| ), | ||
| user: User = Depends(get_api_key_user_async), | ||
| ): | ||
| """List ingestion jobs with optional filtering.""" | ||
| jobs = list(MOCK_JOBS.values()) | ||
|
|
||
| if status: | ||
| jobs = [j for j in jobs if j.status == status] | ||
| if component: | ||
| jobs = [j for j in jobs if j.component == component] | ||
| if actionability: | ||
| jobs = [j for j in jobs if j.actionability == actionability] | ||
|
|
||
| summaries = [ | ||
| IngestionJobSummary( | ||
| ingestion_job_id=j.ingestion_job_id, | ||
| file_name=j.file_name, | ||
| source_type=j.source_type, | ||
| status=j.status, | ||
| component=j.component, | ||
| phase=j.phase, | ||
| error_code=j.error_code, | ||
| actionability=j.actionability, | ||
| retryable=j.retryable, | ||
| user_title=j.user_title, | ||
| created_at=j.created_at, | ||
| ) | ||
| for j in jobs | ||
| ] | ||
|
|
||
| response = IngestionJobListResponse(total=len(summaries), jobs=summaries) | ||
| return JSONResponse(response.model_dump()) | ||
|
|
||
|
|
||
| async def get_ingestion_job( | ||
| job_id: str, | ||
| user: User = Depends(get_api_key_user_async), | ||
| ): | ||
| """Get a specific ingestion job by ID.""" | ||
| job = MOCK_JOBS.get(job_id) | ||
| if not job: | ||
| return JSONResponse({"error": "Ingestion job not found"}, status_code=404) | ||
|
|
||
| return JSONResponse(job.model_dump()) |
Contributor
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Description: Check for user-based filtering patterns in ingestion-related code
# Search for user filtering patterns in API endpoints
rg -n -C3 'user\.id|user_id.*filter|filter.*user' --type=py src/api/
# Search for TODO/FIXME comments about user filtering or multi-tenancy
rg -n 'TODO|FIXME|HACK' --type=py src/api/v1/ingestion_jobs.py
# Check if other v1 endpoints filter by user
rg -n -B5 'Depends\(get_api_key_user_async\)' --type=py src/api/v1/ | rg -A5 'user\.'Repository: langflow-ai/openrag
Length of output: 11668
Filter ingestion jobs by authenticated user to match codebase patterns.
Both endpoints inject user: User = Depends(get_api_key_user_async) but never use it. Other API endpoints (knowledge filters, settings) filter results by user.user_id to ensure users see only their own data. Apply the same pattern here to prevent information disclosure and align with existing security practices.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/api/v1/ingestion_jobs.py` around lines 546 - 599, Both endpoints accept
user: User = Depends(get_api_key_user_async) but never enforce ownership; update
list_ingestion_jobs to filter MOCK_JOBS by job.user_id == user.user_id before
building IngestionJobSummary (so only the authenticated user's jobs are
returned) and update get_ingestion_job to verify the retrieved job's user_id
matches user.user_id and return 404 if not, using the existing MOCK_JOBS,
list_ingestion_jobs, get_ingestion_job, and User.user_id symbols to locate the
changes.
Comment on lines
+590
to
+599
| async def get_ingestion_job( | ||
| job_id: str, | ||
| user: User = Depends(get_api_key_user_async), | ||
| ): | ||
| """Get a specific ingestion job by ID.""" | ||
| job = MOCK_JOBS.get(job_id) | ||
| if not job: | ||
| return JSONResponse({"error": "Ingestion job not found"}, status_code=404) | ||
|
|
||
| return JSONResponse(job.model_dump()) |
Contributor
There was a problem hiding this comment.
🛠️ Refactor suggestion | 🟠 Major | ⚡ Quick win
Use response_model annotation and raise HTTPException for errors.
FastAPI endpoints should declare response models for OpenAPI documentation and use HTTPException for error responses rather than returning JSONResponse directly.
♻️ Proposed refactor
Add this import at the top:
from fastapi import Depends, HTTPException, QueryThen refactor the endpoint:
+from fastapi import Depends, HTTPException, Query
-from fastapi import Depends, Query
async def get_ingestion_job(
job_id: str,
user: User = Depends(get_api_key_user_async),
-):
+) -> IngestionJobDetail:
"""Get a specific ingestion job by ID."""
job = MOCK_JOBS.get(job_id)
if not job:
- return JSONResponse({"error": "Ingestion job not found"}, status_code=404)
+ raise HTTPException(status_code=404, detail="Ingestion job not found")
- return JSONResponse(job.model_dump())
+ return jobAs per coding guidelines, FastAPI routes should use proper response model typing and correct HTTP status codes.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/api/v1/ingestion_jobs.py` around lines 590 - 599, The get_ingestion_job
endpoint currently returns a JSONResponse directly and lacks a response_model;
update the function signature for get_ingestion_job to declare the proper
response_model (the Pydantic model used for ingestion jobs) and keep the
Depends(User=get_api_key_user_async) auth, then replace the
JSONResponse-not-found branch with raise HTTPException(status_code=404,
detail="Ingestion job not found") and return the job model instance (or
job.model_dump() if your response_model expects dict) so FastAPI/OpenAPI can
validate and document the response; ensure HTTPException is imported and
reference MOCK_JOBS, get_ingestion_job, and get_api_key_user_async when making
the changes.
Comment on lines
+91
to
+103
| # Ingestion jobs endpoints | ||
| app.add_api_route( | ||
| "/v1/ingestion/jobs", | ||
| v1_ingestion_jobs.list_ingestion_jobs, | ||
| methods=["GET"], | ||
| tags=["ingestion"], | ||
| ) | ||
| app.add_api_route( | ||
| "/v1/ingestion/jobs/{job_id}", | ||
| v1_ingestion_jobs.get_ingestion_job, | ||
| methods=["GET"], | ||
| tags=["ingestion"], | ||
| ) |
Contributor
There was a problem hiding this comment.
Inconsistent OpenAPI tags may affect documentation and SDK generation.
The new ingestion job routes use tags=["ingestion"] while all other routes in this file use tags=["public"]. This inconsistency could:
- Create unexpected groupings in the OpenAPI/Swagger documentation
- Affect SDK generation if it's tag-based
- Confuse developers expecting uniform tagging in
public_v1.py
Consider using tags=["public"] for consistency with other routes in this registration file, or if separate tagging is intentional, document the reasoning.
🏷️ Proposed fix for consistency
# Ingestion jobs endpoints
app.add_api_route(
"/v1/ingestion/jobs",
v1_ingestion_jobs.list_ingestion_jobs,
methods=["GET"],
- tags=["ingestion"],
+ tags=["public"],
)
app.add_api_route(
"/v1/ingestion/jobs/{job_id}",
v1_ingestion_jobs.get_ingestion_job,
methods=["GET"],
- tags=["ingestion"],
+ tags=["public"],
)📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| # Ingestion jobs endpoints | |
| app.add_api_route( | |
| "/v1/ingestion/jobs", | |
| v1_ingestion_jobs.list_ingestion_jobs, | |
| methods=["GET"], | |
| tags=["ingestion"], | |
| ) | |
| app.add_api_route( | |
| "/v1/ingestion/jobs/{job_id}", | |
| v1_ingestion_jobs.get_ingestion_job, | |
| methods=["GET"], | |
| tags=["ingestion"], | |
| ) | |
| # Ingestion jobs endpoints | |
| app.add_api_route( | |
| "/v1/ingestion/jobs", | |
| v1_ingestion_jobs.list_ingestion_jobs, | |
| methods=["GET"], | |
| tags=["public"], | |
| ) | |
| app.add_api_route( | |
| "/v1/ingestion/jobs/{job_id}", | |
| v1_ingestion_jobs.get_ingestion_job, | |
| methods=["GET"], | |
| tags=["public"], | |
| ) |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/app/routes/public_v1.py` around lines 91 - 103, The two route
registrations that call v1_ingestion_jobs.list_ingestion_jobs and
v1_ingestion_jobs.get_ingestion_job currently use tags=["ingestion"]; update
them to use tags=["public"] to match the rest of this registration file (or, if
a separate "ingestion" grouping is intentional, add a short comment explaining
the reason). Ensure you change the tags argument on both add_api_route calls so
OpenAPI grouping is consistent.
* add feedback tracking for assistant messages * style: apply biome auto-fixes [skip ci] * don't show during onboarding * coderabbit suggestions --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* add more finalizer removal logic * crd pvreclaimpolicy * review comments
* feat: Implement two-phase ingestion flow with Docling polling - Added configuration settings for Docling polling intervals and retries. - Refactored LangflowFileProcessor to support two-phase ingestion with Docling. - Introduced DoclingPollingService to handle backend polling for Docling task completion. - Updated LangflowFileService to manage the new two-phase ingestion process. - Enhanced FileTask model to track Docling task ID and status. - Created unit tests for DoclingPollingService and LangflowFileService to ensure correct behavior during two-phase ingestion. - Updated task service to include new fields for tracking ingestion phases and statuses. Co-authored-by: Copilot <copilot@github.com> * fix: Fix summary: Added ENABLE_BACKEND_DOCLING_POLLING env flag in src/config/settings.py (defaults to true). Guarded the auto-construction in src/models/processors.py:841-857 so the DoclingPollingService is only built when the flag is on; otherwise self.docling_polling_service stays None and upload_and_ingest_file() falls through to the legacy single-call path. Added two tests asserting both directions of the flag in tests/unit/test_langflow_file_service_two_phase.py. * Architectural improvement: The polling service is now a true singleton, wired in the same place and same way as every other service. The processor is back to a passive consumer with no knowledge of feature flags or service construction. * fix: Ensure docling_status is set to SUCCESS in legacy path for coherent status fields * style: ruff format (auto) --------- Co-authored-by: Copilot <copilot@github.com> Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
* sharepoint folder handling * code rabbit fix * remove download url * style: ruff format (auto) * logging issue * linting --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
* remove four flow ids * fix fmt
* fix imports * ruff fix * Use PEP 604 types and modernize imports Replace typing.Optional/Dict with PEP 604 union syntax and built-in dict annotations to modernize type hints (requires Python 3.10+). Update annotations in DoclingStatusSnapshot (detail/raw), DoclingService.__init__ and fetch_task_result. Clean up and reorder imports (platform moved up, removed unused typing names) for clarity. * Use StrEnum for DoclingTaskState; chain exception Switch DoclingTaskState to StrEnum (imported from enum) so enum members are proper string enums. Also preserve the original error context in LangflowFileService by raising the upload exception with "from e" to keep exception chaining and traceback information. * style: ruff format (auto) * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * style: ruff format (auto) --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> Co-authored-by: Rico Furtado <hfurtado@gmail.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* Add default LangFlow flow IDs Provide fallback UUIDs for LangFlow-related environment variables in src/config/settings.py. LANGFLOW_CHAT_FLOW_ID, LANGFLOW_INGEST_FLOW_ID, LANGFLOW_URL_INGEST_FLOW_ID, and NUDGES_FLOW_ID now default to specific UUIDs when the corresponding environment variables are not set, preventing None values and ensuring the app has usable flow IDs in development or when env vars are missing. * style: ruff format (auto) * Use 'or' fallback for Langflow flow IDs Switch LANGFLOW_* and NUDGES_FLOW_ID settings to use os.getenv(...) or <default> instead of os.getenv(..., <default>). This makes the code fall back to the hardcoded defaults when an environment variable is set to an empty string as well as when it's unset. Updated variables: LANGFLOW_CHAT_FLOW_ID, LANGFLOW_INGEST_FLOW_ID, LANGFLOW_URL_INGEST_FLOW_ID, NUDGES_FLOW_ID. * style: ruff format (auto) --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Add a new `typecheck` job to .github/workflows/lint-frontend.yml that sets up Node 20, installs frontend deps, and runs `npm run typecheck` to perform a TypeScript check in CI. Also add a `typecheck` script (`tsc --noEmit`) to frontend/package.json so the pipeline can run `tsc` without emitting files. This ensures TypeScript errors are caught during CI.
Import Trash2 from lucide-react in knowledge-batch-actions-bar to support delete UI actions. Correct the Tabs import in settings-tabs to use the proper @radix-ui/react-tabs namespace import (was importing from 'radix-ui').
* create and use onboarding state hook * small coderabbit check
* fix next vulerability * ts config * style: apply biome auto-fixes [skip ci] --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* Add flag to skip OpenSearch security setup Introduce OPENRAG_SKIP_OS_SECURITY_SETUP (env var, default false) in settings to allow deployments to opt out of OpenSearch security configuration when the platform manages roles/role-mappings externally. Use this flag in startup_orchestrator.startup_tasks and utils.opensearch_init.init_index to skip calling setup_opensearch_security and emit informative log lines; index creation still runs in init_index. Add unit tests to verify both the skipped and non-skipped behaviors and to ensure indices are still created when security setup is skipped. * style: ruff format (auto) --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
* Handle user insert races and add test timeout Increase TypeScript integration test timeout to 120s to reduce flakiness during slow CI runs. Enhance user_service.ensure_user_row IntegrityError handling to explicitly handle concurrent-insert races: detect a (oauth_provider, oauth_subject) race and return the existing row, detect an email_lookup_hash race by looking up the email and returning the concurrent identity when it matches, and handle PK collisions by retrying the insert with a new UUID. Add explanatory comments about which collisions are recoverable and when errors should propagate to the caller. * style: ruff format (auto) * Use PEP 604 union for agent config return type Replace typing.Optional[dict] with PEP 604 union syntax (dict | None) for get_effective_agent_config and remove the now-unused Optional import. This is a pure type-annotation cleanup (no runtime behavior changes); note it requires Python 3.10+ for the `|` union syntax. --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
* fixed onedrive not working * style: ruff format (auto) * removed sites.read.all from onedrive * style: ruff format (auto) * fixed allowed users and groups * fix lint error * fixed lint * fixed mypy lint * style: ruff format (auto) --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
* Skip deleted connector files; return orphan IDs Handle files deleted at source and improve orphan reconciliation. - Add TaskStatus.SKIPPED and update ConnectorFileProcessor to catch FileNotFoundError/ValueError from connector.get_file_content. When a file is missing, mark the file task as SKIPPED, set a skipped result, update timestamps and counters, and avoid raising so the upload can continue. - Change reconcile_orphans_for_connector_type to return a list of orphan file IDs ([]) instead of an int, update early returns and docstring to reflect returning the deleted/removed document IDs so callers can exclude them from subsequent sync passes. - Minor whitespace cleanup in sync_all_connectors. These changes avoid aborting syncs on missing source files and make orphan deletions explicit for downstream processing. * Skip deleted connector files; return orphan IDs Handle files deleted at source and improve orphan reconciliation. - Add TaskStatus.SKIPPED and update ConnectorFileProcessor to catch FileNotFoundError/ValueError from connector.get_file_content. When a file is missing, mark the file task as SKIPPED, set a skipped result, update timestamps and counters, and avoid raising so the upload can continue. - Change reconcile_orphans_for_connector_type to return a list of orphan file IDs ([]) instead of an int, update early returns and docstring to reflect returning the deleted/removed document IDs so callers can exclude them from subsequent sync passes. - Minor whitespace cleanup in sync_all_connectors. These changes avoid aborting syncs on missing source files and make orphan deletions explicit for downstream processing. * Add *.db to .gitignore Add a '*.db' pattern to .gitignore to prevent local database files (e.g., SQLite) from being committed to the repository. * Add orphan reconcile and bulk-delete helper Introduce a reconcile pass and supporting bulk-delete utility to remove OpenSearch chunks for files that no longer exist remotely. - Add reconcile_orphans_for_connector_type (src/api/connectors.py): lists all active connections for a connector type, strictly gates on unauthenticated connections or listing errors (abort with 0 deletes), aggregates paginated remote file IDs, computes orphans (indexed IDs not present remotely) and invokes delete_chunks_by_document_ids. Integrated into connector_sync and sync_all_connectors (skips reconcile when sync is capped). - Add delete_chunks_by_document_ids (src/api/documents.py): issues a single delete_by_query with terms(document_id, ...) and conflicts="proceed", returns deleted count, and short-circuits on empty input. - Update connector metadata painless params (src/connectors/service.py): include filename param and assign ctx._source.filename = params.filename in the update script so renamed files update indexed chunks. - Add unit tests: cover delete_chunks_by_document_ids, reconcile_orphans_for_connector_type (gating, pagination, multi-connection union, error handling), and connector metadata filename behavior. Behavior notes: reconcile is conservative to avoid false-positive deletions; bulk delete is defensive and returns 0 on unexpected responses or failures. * fix lint * style: ruff format (auto) * fix lint * Use max_files param and add typing Add an explicit type annotation for files_to_process and update the connector.list_files call to use the max_files parameter instead of limit to match the connector API. Also remove redundant `# type: ignore` comments on connector.cfg assignments as they are no longer necessary. Minor cleanup to improve type clarity and API consistency. * Add connector sync preview UI and API Introduce sync-preview functionality for connectors and sync-all flows. Backend: add ID→filename aggregation, compute_orphans (non-destructive orphan detection), orphan deletion helper, and preview endpoints (connector_sync_preview, connectors_sync_all_preview) plus wiring in internal routes. Frontend: add useSyncConnectorPreview/useSyncAllConnectorsPreview mutations, wire preview flows into Knowledge pages/components, and add SyncConfirmDialog component to show orphan lists, per-connector synced counts, availability flags, and confirm deletion+sync interactions. Also update UI buttons to open the preview dialog and handle loading/syncing states. * style: ruff format (auto) * Hoist and consolidate imports in processors Move many inline imports to module scope and consolidate repeated imports across TaskProcessor and its subclasses. Adds top-level imports (asyncio, datetime, mimetypes, os, time), config/settings and utility functions (clients, get_embedding_model, get_index_name, get_openrag_config, extract_relevant, process_text_file, resplit_chunks_character_windows, ensure_embedding_field_exists, auto_cleanup_tempfile, hash_id, build_filename_delete_body, build_filename_search_body) and imports TaskStatus. Removes redundant per-function imports to improve readability and reduce repeated import overhead; no functional changes intended. * Type UseQueryOptions with SearchResult Specify UseQueryOptions<SearchResult> for the useGetSearchQuery hook's options parameter to improve TypeScript type inference and ensure the query options expect SearchResult data. This change tightens typings without altering runtime behavior. --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
* fix factory reset not deleting data * make onboarding errors pop up on e2e tests * style: ruff format (auto) * added check for error when uploading * added check if its on first step to rollback, not only if its complete * reset correctly * update timeout for uploading document * remove misclick * fixed lint * fix mypy errors * style: ruff format (auto) --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
* file upload for context * coderabbit suggestions
* Add kind build/load targets and kind sample Add Makefile targets to build and load app images into a kind cluster (KIND_CLUSTER_NAME default: openrag): kind-load-app-images and kind-build-load-apps, and expose them in the help output. Update kubernetes/operator/README.md with instructions to run the operator locally, apply a kind-local sample, build/load images into kind, and restart deployments after rebuilding. Add a new config sample openrag_v1alpha1_openrag-kind-local.yaml tuned for 2-CPU kind/Colima clusters (lower resource requests, imagePullPolicy: Never). Also annotate the default sample to point users to the kind-local file for local clusters. * Add Makefile help for operator & kind Add a new help_operator target and OPERATOR_DIR variable to the Makefile, and include it in the .PHONY list. The new help output documents Kubernetes operator and kind-related commands (build/load app images into kind, operator deps/install/run/build/test/lint/manifests/generate, deploy/undeploy, docker-build, helm install) and provides a typical kind + local images workflow and sample CR usage. This makes it easier for developers to discover and run local operator/cluster tasks from the repo root.
Add a COPY instruction to include the local flows/ directory at /app/flows in the image so bundled flows are available at runtime. Placed before the entrypoint/ownership handling to ensure the files are copied with the intended UID/permissions.
* remove logic for previous tab selected, need to be discussed with Ana * reduce repeated code and use darkmodelight class * reduce repeated code and use darkmodelight class * remove unecessary !important * remove unecessary !important * remove duplicate css blocks * remove !important and merge active and hover state * put back TabsContent anf fix hover issue --------- Co-authored-by: Olfa Maslah <olfamaslah@Olfas-MacBook-Pro.local>
* Add ruff autofix step to CI workflows Add an in-place Ruff fix step to .github/workflows/autofix.ci.yml that runs uv run ruff check --fix on changed files, rename the job and autofix commit message to reflect "ruff autofix", and update comments to clarify that autofix.ci applies safe fixes and formatting. Also update .github/workflows/lint-backend.yml comments to state that safe lint fixes and formatting are auto-applied by autofix.ci while keeping strict lint (--no-fix) and mypy in the lint workflow. * Update autofix.ci.yml
* Enum IDs and delete OpenSearch docs by _id Add DLS-safe OpenSearch delete helpers and use them to avoid silent no-ops from delete_by_query. Introduces utils/opensearch_delete.py with collect_visible_document_ids and delete_document_ids (enumerate visible _ids via search/scroll, then delete by primary _id). Update delete_chunks_by_document_ids to enumerate chunk IDs and delete each by _id. Ensure langflow_connector_service and TaskProcessor clear stale chunks (by document_id) before re-ingest/re-index to prevent duplicate or trailing chunks after renames. Improve connector listing by scoping cfg.file_ids/folder_ids when available to avoid false orphan detection. Add and update unit tests to assert the new enumeration-and-delete behavior and ordering. * style: ruff format (auto) * ruff fix * ruff fix --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Remove google_drive from the cloud-brand exclusion so Google Drive connectors are visible when isCloudBrand is true. Applied the change in connector-cards.tsx and knowledge-dropdown.tsx (OneDrive remains excluded).
* fix cr deletion * refactoring * fix unit test * fix lint * fix cross namespace CR deletion
…on (#1625) * feat: Implement in-process JWT claims caching with TTL and LRU eviction * style: ruff format (auto) * refactor: Update cache initialization to use settings for max size and TTL * feat: code cleanup * fix: Improve public key file reading and enhance token scheme handling * style: ruff autofix (auto) * fix: Handle user insert races and add test timeout (#1618) * Handle user insert races and add test timeout Increase TypeScript integration test timeout to 120s to reduce flakiness during slow CI runs. Enhance user_service.ensure_user_row IntegrityError handling to explicitly handle concurrent-insert races: detect a (oauth_provider, oauth_subject) race and return the existing row, detect an email_lookup_hash race by looking up the email and returning the concurrent identity when it matches, and handle PK collisions by retrying the insert with a new UUID. Add explanatory comments about which collisions are recoverable and when errors should propagate to the caller. * style: ruff format (auto) * Use PEP 604 union for agent config return type Replace typing.Optional[dict] with PEP 604 union syntax (dict | None) for get_effective_agent_config and remove the now-unused Optional import. This is a pure type-annotation cleanup (no runtime behavior changes); note it requires Python 3.10+ for the `|` union syntax. --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> * fix: display file names and fix ingestion for onedrive (#1609) * fixed onedrive not working * style: ruff format (auto) * removed sites.read.all from onedrive * style: ruff format (auto) * fixed allowed users and groups * fix lint error * fixed lint * fixed mypy lint * style: ruff format (auto) --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> * fix: Skip deleted connector files; return orphan IDs (#1592) * Skip deleted connector files; return orphan IDs Handle files deleted at source and improve orphan reconciliation. - Add TaskStatus.SKIPPED and update ConnectorFileProcessor to catch FileNotFoundError/ValueError from connector.get_file_content. When a file is missing, mark the file task as SKIPPED, set a skipped result, update timestamps and counters, and avoid raising so the upload can continue. - Change reconcile_orphans_for_connector_type to return a list of orphan file IDs ([]) instead of an int, update early returns and docstring to reflect returning the deleted/removed document IDs so callers can exclude them from subsequent sync passes. - Minor whitespace cleanup in sync_all_connectors. These changes avoid aborting syncs on missing source files and make orphan deletions explicit for downstream processing. * Skip deleted connector files; return orphan IDs Handle files deleted at source and improve orphan reconciliation. - Add TaskStatus.SKIPPED and update ConnectorFileProcessor to catch FileNotFoundError/ValueError from connector.get_file_content. When a file is missing, mark the file task as SKIPPED, set a skipped result, update timestamps and counters, and avoid raising so the upload can continue. - Change reconcile_orphans_for_connector_type to return a list of orphan file IDs ([]) instead of an int, update early returns and docstring to reflect returning the deleted/removed document IDs so callers can exclude them from subsequent sync passes. - Minor whitespace cleanup in sync_all_connectors. These changes avoid aborting syncs on missing source files and make orphan deletions explicit for downstream processing. * Add *.db to .gitignore Add a '*.db' pattern to .gitignore to prevent local database files (e.g., SQLite) from being committed to the repository. * Add orphan reconcile and bulk-delete helper Introduce a reconcile pass and supporting bulk-delete utility to remove OpenSearch chunks for files that no longer exist remotely. - Add reconcile_orphans_for_connector_type (src/api/connectors.py): lists all active connections for a connector type, strictly gates on unauthenticated connections or listing errors (abort with 0 deletes), aggregates paginated remote file IDs, computes orphans (indexed IDs not present remotely) and invokes delete_chunks_by_document_ids. Integrated into connector_sync and sync_all_connectors (skips reconcile when sync is capped). - Add delete_chunks_by_document_ids (src/api/documents.py): issues a single delete_by_query with terms(document_id, ...) and conflicts="proceed", returns deleted count, and short-circuits on empty input. - Update connector metadata painless params (src/connectors/service.py): include filename param and assign ctx._source.filename = params.filename in the update script so renamed files update indexed chunks. - Add unit tests: cover delete_chunks_by_document_ids, reconcile_orphans_for_connector_type (gating, pagination, multi-connection union, error handling), and connector metadata filename behavior. Behavior notes: reconcile is conservative to avoid false-positive deletions; bulk delete is defensive and returns 0 on unexpected responses or failures. * fix lint * style: ruff format (auto) * fix lint * Use max_files param and add typing Add an explicit type annotation for files_to_process and update the connector.list_files call to use the max_files parameter instead of limit to match the connector API. Also remove redundant `# type: ignore` comments on connector.cfg assignments as they are no longer necessary. Minor cleanup to improve type clarity and API consistency. * Add connector sync preview UI and API Introduce sync-preview functionality for connectors and sync-all flows. Backend: add ID→filename aggregation, compute_orphans (non-destructive orphan detection), orphan deletion helper, and preview endpoints (connector_sync_preview, connectors_sync_all_preview) plus wiring in internal routes. Frontend: add useSyncConnectorPreview/useSyncAllConnectorsPreview mutations, wire preview flows into Knowledge pages/components, and add SyncConfirmDialog component to show orphan lists, per-connector synced counts, availability flags, and confirm deletion+sync interactions. Also update UI buttons to open the preview dialog and handle loading/syncing states. * style: ruff format (auto) * Hoist and consolidate imports in processors Move many inline imports to module scope and consolidate repeated imports across TaskProcessor and its subclasses. Adds top-level imports (asyncio, datetime, mimetypes, os, time), config/settings and utility functions (clients, get_embedding_model, get_index_name, get_openrag_config, extract_relevant, process_text_file, resplit_chunks_character_windows, ensure_embedding_field_exists, auto_cleanup_tempfile, hash_id, build_filename_delete_body, build_filename_search_body) and imports TaskStatus. Removes redundant per-function imports to improve readability and reduce repeated import overhead; no functional changes intended. * Type UseQueryOptions with SearchResult Specify UseQueryOptions<SearchResult> for the useGetSearchQuery hook's options parameter to improve TypeScript type inference and ensure the query options expect SearchResult data. This change tightens typings without altering runtime behavior. --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> * fix: add reset db on command, expose error on e2e tests (#1627) * fix factory reset not deleting data * make onboarding errors pop up on e2e tests * style: ruff format (auto) * added check for error when uploading * added check if its on first step to rollback, not only if its complete * reset correctly * update timeout for uploading document * remove misclick * fixed lint * fix mypy errors * style: ruff format (auto) --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> * refactor: handler for file upload for context (#1624) * file upload for context * coderabbit suggestions * chore: add operator make commands (#1628) * Add kind build/load targets and kind sample Add Makefile targets to build and load app images into a kind cluster (KIND_CLUSTER_NAME default: openrag): kind-load-app-images and kind-build-load-apps, and expose them in the help output. Update kubernetes/operator/README.md with instructions to run the operator locally, apply a kind-local sample, build/load images into kind, and restart deployments after rebuilding. Add a new config sample openrag_v1alpha1_openrag-kind-local.yaml tuned for 2-CPU kind/Colima clusters (lower resource requests, imagePullPolicy: Never). Also annotate the default sample to point users to the kind-local file for local clusters. * Add Makefile help for operator & kind Add a new help_operator target and OPERATOR_DIR variable to the Makefile, and include it in the .PHONY list. The new help output documents Kubernetes operator and kind-related commands (build/load app images into kind, operator deps/install/run/build/test/lint/manifests/generate, deploy/undeploy, docker-build, helm install) and provides a typical kind + local images workflow and sample CR usage. This makes it easier for developers to discover and run local operator/cluster tasks from the repo root. * redirect on logout (#1440) (#1636) * fix: Copy flows directory into Docker image (#1632) Add a COPY instruction to include the local flows/ directory at /app/flows in the image so bundled flows are available at runtime. Placed before the entrypoint/ownership handling to ensure the files are copied with the intended UID/permissions. * feat: settings saas tabs menu (#1637) * remove logic for previous tab selected, need to be discussed with Ana * reduce repeated code and use darkmodelight class * reduce repeated code and use darkmodelight class * remove unecessary !important * remove unecessary !important * remove duplicate css blocks * remove !important and merge active and hover state * put back TabsContent anf fix hover issue --------- Co-authored-by: Olfa Maslah <olfamaslah@Olfas-MacBook-Pro.local> * chore: Add ruff autofix step to CI workflows (#1640) * Add ruff autofix step to CI workflows Add an in-place Ruff fix step to .github/workflows/autofix.ci.yml that runs uv run ruff check --fix on changed files, rename the job and autofix commit message to reflect "ruff autofix", and update comments to clarify that autofix.ci applies safe fixes and formatting. Also update .github/workflows/lint-backend.yml comments to state that safe lint fixes and formatting are auto-applied by autofix.ci while keeping strict lint (--no-fix) and mypy in the lint workflow. * Update autofix.ci.yml * fix: Enum IDs and delete OpenSearch docs by _id (#1638) * Enum IDs and delete OpenSearch docs by _id Add DLS-safe OpenSearch delete helpers and use them to avoid silent no-ops from delete_by_query. Introduces utils/opensearch_delete.py with collect_visible_document_ids and delete_document_ids (enumerate visible _ids via search/scroll, then delete by primary _id). Update delete_chunks_by_document_ids to enumerate chunk IDs and delete each by _id. Ensure langflow_connector_service and TaskProcessor clear stale chunks (by document_id) before re-ingest/re-index to prevent duplicate or trailing chunks after renames. Improve connector listing by scoping cfg.file_ids/folder_ids when available to avoid false orphan detection. Add and update unit tests to assert the new enumeration-and-delete behavior and ordering. * style: ruff format (auto) * ruff fix * ruff fix --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> * make probes more robust (#1642) * feat: Show google_drive connector for cloud brand (#1650) Remove google_drive from the cloud-brand exclusion so Google Drive connectors are visible when isCloudBrand is true. Applied the change in connector-cards.tsx and knowledge-dropdown.tsx (OneDrive remains excluded). * operator ubi base build (#1652) * remove flow download initContainer when flowRef is removed (#1653) * fix: cr deletion (#1656) * fix cr deletion * refactoring * fix unit test * fix lint * fix cross namespace CR deletion * Potential fix for pull request finding 'CodeQL / Information exposure through an exception' Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> Co-authored-by: Edwin Jose <edwin.jose@datastax.com> Co-authored-by: Lucas Oliveira <62335616+lucaseduoli@users.noreply.github.com> Co-authored-by: Mike Fortman <mfortman11@gmail.com> Co-authored-by: Wallgau <46035189+Wallgau@users.noreply.github.com> Co-authored-by: Olfa Maslah <olfamaslah@Olfas-MacBook-Pro.local> Co-authored-by: ming <itestmycode@gmail.com> Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
* update style for oss of the failed task in the task panel * keep logic on click, remove unecessary useeffect * fix padding * wip implementing Saas style * utils to reshape error until backend provide info we need * utils to reshape error until backend provide info we need * utils to reshape error until backend provide info we need and fixinf fallbacks of isTotalFailure * utils to reshape error until backend provide into * have Saas style for failed and complete labelstatus and width and border * few style adjustment to follow codebase pattern * adjust succeed and partially succeed case * adding comment for TODO implementation or more clarity * remove carbon icon package and replace carbon icon * add incident-reporter-icon --------- Co-authored-by: Olfa Maslah <olfamaslah@Olfas-MacBook-Pro.local>
* Encode IBM API key as Basic auth header Add base64 encoding for the IBM auth path: import base64, construct a Basic auth token from X-Username and X-Api-Key (username:apikey), and store it in user.jwt_token and user.opensearch_credentials. Also set request.state.user before attaching the DB user ID so downstream code can access the created user object. * style: ruff autofix (auto) --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
* restart deployment if env changes * unit test * lint
…S_TO_GET_FROM_ENVIRONMENT (#1667) * Ensure we dynamically update the list of Langflow .env environment variables with default values when the comma separated list defined in LANGFLOW_VARIABLES_TO_GET_FROM_ENVIRONMENT changes * fix tests * fix additional linting errors --------- Co-authored-by: rodageve <rodrigo.geve@datastax.com>
* Retire openrag-mcp; switch docs to streamable HTTP Remove the stdio-based MCP server and all in-repo MCP tooling, and update README to mark the package as retired. Deleted module files include the MCP entrypoint, server, config, registry and individual tools (chat, search, documents, settings). The README was rewritten to announce that openrag-mcp is retired, explain migration to the built-in streamable-HTTP /mcp endpoint, update Cursor/Claude examples to use URL+headers auth, list the new v1 API tools, and note that the last PyPI release is final. This change consolidates MCP functionality into the OpenRAG core and removes the subprocess/stdio implementation and its source code. * Mark MCP SDK retired and clean package metadata Update package metadata to reflect retirement and integration into the OpenRAG backend. Bump version to 0.3.0 and replace the project description with a retirement/migration note. Set Development Status to Inactive, remove explicit Python version classifiers, and clear runtime dependencies and the CLI script entrypoint. Also remove the hatch env pip-args setting; build-system and wheel package target remain unchanged. * chore: update uv.lock files after version bump * Update uv.lock --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* Add filename-based duplicate handling for connectors Add end-to-end support for filename-based duplicate handling on connector ingests. Frontend: send a new replace_duplicates flag with connector sync requests, perform a pre-sync duplicate check, and show a DuplicateHandlingDialog that lets users overwrite or skip duplicates when uploading from provider UI. Backend: propagate replace_duplicates through connector_router, request models, and connector services into the file processors. ConnectorFileProcessor and LangflowConnectorFileProcessor now check whether a filename already exists in the index and either fail the file task or delete the existing document before ingesting when replace_duplicates is true. Utilities/tests: clean_connector_filename now preserves original spacing/slashes and only enforces MIME-mapped extensions; get_filename_aliases adds underscore/sanitized variants so lookups match connector-indexed names. Add unit tests covering filename dedupe logic and filename alias behavior. * Use duplicateNames list and display names Replace numeric duplicateCount with a duplicateNames string[] across upload and dropdown flows so the UI can show the actual file names that would be overwritten. The duplicate-handling dialog now accepts duplicateNames, derives an effective count, and lists up to 5 duplicate filenames with an "… and N more" indicator; message labels and button text use the effective count. Toast messages and pending state in upload/[provider]/page.tsx and knowledge-dropdown.tsx were updated to pass and consume duplicateNames and to use duplicateNames.length for counts. * Update page.tsx * style: ruff autofix (auto) --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
github-actions
Bot
added
frontend
github-actions
Bot
added
enhancement
Contributor
There was a problem hiding this comment.
Actionable comments posted: 19
Note
Due to the large number of review comments, Critical, Major severity comments were prioritized as inline comments.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (5)
frontend/components/cloud-picker/provider-handlers.ts (1)
188-213:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winEmit picker open state for OneDrive before invoking
window.OneDrive.open.At Line 188,
onPickerStateChangeis only transitioned tofalsein outcome paths, but never set totruewhen opening. That makes picker-state tracking asymmetric.💡 Suggested fix
openPicker(onFileSelected: (files: CloudFile[]) => void): void { if (!window.OneDrive) { + this.onPickerStateChange?.(false); return; } + this.onPickerStateChange?.(true); // For SharePoint, use the SharePoint site URL as endpoint hint🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@frontend/components/cloud-picker/provider-handlers.ts` around lines 188 - 213, In openPicker, emit the picker-open state before calling window.OneDrive.open by invoking this.onPickerStateChange?.(true) just prior to the OneDrive.open call so state transitions are symmetric with the existing this.onPickerStateChange?.(false) exit paths; ensure you use optional chaining to avoid errors when onPickerStateChange is undefined and keep the call inside the early return guard that checks window.OneDrive.src/services/task_service.py (1)
245-275:⚠️ Potential issue | 🟠 Major | 🏗️ Heavy liftTrack appended background work on the task itself.
When
existing_task_idis reused, this path starts anotherbackground_taskbut only stores a cancel handle for freshly created tasks.cancel_task()still cancels justupload_task.background_task, so appended batches can keep ingesting after the user has cancelled the task.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/services/task_service.py` around lines 245 - 275, The issue is that when reusing an existing UploadTask (existing_task_id path) you start a new background_task but only assign upload_task.background_task for newly created tasks, so appended batches remain uncancelable; update the code after creating background_task (the asyncio.create_task call that invokes background_custom_processor) to always set upload_task.background_task = background_task for both branches (when upload_task comes from self.task_store[...] and when it's newly created) so cancel_task() (which expects upload_task.background_task) will cancel the most recent background job.src/connectors/onedrive/oauth.py (1)
1-8:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winSwitch
src/connectors/onedrive/oauth.pytoutils.logging_config.get_logger
This module still initializes the logger via stdliblogging.getLogger(__name__), bypassing the shared repo logging configuration.🧾 Suggested fix
-import logging import os from typing import Any import msal +from utils.logging_config import get_logger -logger = logging.getLogger(__name__) +logger = get_logger(__name__)🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/connectors/onedrive/oauth.py` around lines 1 - 8, Replace the module's local stdlib logger initialization with the shared logger factory: import get_logger from utils.logging_config and replace the line that sets logger = logging.getLogger(__name__) with logger = get_logger(__name__) so this module uses the repo-wide logging configuration (referencing the symbols get_logger and the module-level logger variable).src/connectors/onedrive/connector.py (2)
27-43:⚠️ Potential issue | 🟠 Major | ⚡ Quick winInitialize
BaseConnectoronly once, after normalizingconfig.
super().__init__(config)runs before theNoneguard and then runs again inside thetry. IfconfigisNone, Line 28 can already fail, and the double initialization can reset base-class state unexpectedly.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/connectors/onedrive/connector.py` around lines 27 - 43, The constructor currently calls super().__init__(config) twice and before normalizing config; change OneDrive connector __init__ so you first normalize the incoming config (if config is None set config = {} and log as needed) and then call super().__init__(config) exactly once inside the try/except block; remove the earlier unconditional super().__init__(config) call and keep the existing logger.debug/ logger.error messages around the single initialization to preserve diagnostics (refer to the __init__ method in the OneDrive connector for the symbols to update).
1-12: 🛠️ Refactor suggestion | 🟠 Major | ⚡ Quick winUse
get_loggerin this module.This file is still wiring logging through the stdlib logger instead of the repository logger. As per coding guidelines,
src/**/*.py: “Use get_logger from utils.logging_config — never import stdlib logging directly.”🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/connectors/onedrive/connector.py` around lines 1 - 12, The module currently imports the stdlib logging and defines logger = logging.getLogger(__name__); replace that with the repository logger by importing get_logger from utils.logging_config and create logger = get_logger(__name__) instead. Locate the top-level logger variable in this file (symbol: logger) and swap the import of logging for from utils.logging_config import get_logger, leaving all uses of logger (e.g., inside methods of BaseConnector-derived classes or functions interacting with OneDriveOAuth, ConnectorDocument, DocumentACL) unchanged so the module uses the centralized logging configuration.
🟡 Minor comments (8)
frontend/app/chat/_types/types.ts-24-28 (1)
24-28:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winCreate a fresh initial message per use instead of exporting a singleton object.
This constant freezes
timestampat module-load time and can leak shared object state if reused directly in message arrays. Prefer a factory that returns a newMessageeach time.💡 Proposed fix
-export const INITIAL_ASSISTANT_MESSAGE: Message = { - role: "assistant", - content: "How can I assist?", - timestamp: new Date(), -}; +export const createInitialAssistantMessage = (): Message => ({ + role: "assistant", + content: "How can I assist?", + timestamp: new Date(), +});🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@frontend/app/chat/_types/types.ts` around lines 24 - 28, INITIAL_ASSISTANT_MESSAGE is a singleton object that captures timestamp at module load and can leak shared mutable state; replace it with a factory function (e.g., createInitialAssistantMessage or getInitialAssistantMessage) that returns a new Message object with role "assistant", content "How can I assist?" and timestamp: new Date() each time, export that factory instead of the constant, and update callers to invoke the factory so each use gets a fresh Message instance.kubernetes/operator/config/crd/bases/openr.ag_openrags.yaml-12680-12686 (1)
12680-12686:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winRemove duplicated
pvcReclaimPolicyenum generation in the CRD.
openr.ag_openrags.yamlgenerates two identicalpvcReclaimPolicy.allOf[].enumblocks (Retain,Delete). The duplication comes from two// +kubebuilder:validation:Enum=Retain;Deletemarkers inkubernetes/operator/api/v1alpha1/openrag_types.go(one ontype PVCReclaimPolicy string, and one on thepvcReclaimPolicyfield alongside// +kubebuilder:default=Retain). Keep only one Enum marker (type or field) and regenerate the CRD so the schema has a single enum definition.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@kubernetes/operator/config/crd/bases/openr.ag_openrags.yaml` around lines 12680 - 12686, The CRD contains duplicated enum entries because both the PVCReclaimPolicy type and the pvcReclaimPolicy field have kubebuilder Enum markers; remove one of the duplicate markers in kubernetes/operator/api/v1alpha1/openrag_types.go — either delete the `// +kubebuilder:validation:Enum=Retain;Delete` above the `type PVCReclaimPolicy string` declaration or remove the Enum marker on the `pvcReclaimPolicy` field (keep the `// +kubebuilder:default=Retain` if desired), then regenerate the CRD so the final openr.ag_openrags.yaml has a single `pvcReclaimPolicy.allOf[].enum` entry.frontend/components/duplicate-handling-dialog.tsx-82-85 (1)
82-85:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winUse a unique React key for duplicate-name list items.
visibleNamesis derived fromduplicateNames(viaslice), and each<li>useskey={name}.duplicateNamesis built fromfile.name, and folder uploads strip directory paths (originalFile.name.split("/").pop()), so multiple duplicate entries can share the same base filename—making the key non-unique and risking reconciliation glitches.Proposed fix
- {visibleNames.map((name) => ( - <li key={name} className="break-all"> + {visibleNames.map((name, index) => ( + <li key={`${name}-${index}`} className="break-all"> {name} </li> ))}🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@frontend/components/duplicate-handling-dialog.tsx` around lines 82 - 85, The list items use key={name}, but names may repeat (duplicateNames derived from originalFile.name.split("/").pop()), causing non-unique React keys; update the render for visibleNames so each <li> uses a unique key (e.g., combine the filename with its index or a derived unique id) by referencing the map callback index or the original duplicateNames index when mapping over visibleNames to produce keys for the <li> elements.frontend/components/knowledge-search-bar.tsx-81-90 (1)
81-90:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winHandle partial sync results explicitly.
When both
synced_connectorsanderrorsare present, the current branch order reports success and hides failures.🔧 Proposed fix
- } else if ( - result.synced_connectors && - result.synced_connectors.length > 0 - ) { - toast.success( - `Sync started for ${result.synced_connectors.join(", ")}. Check task notifications for progress.`, - ); - } else if (result.errors && result.errors.length > 0) { - toast.error("Some connectors failed to sync"); + } else if (result.errors && result.errors.length > 0) { + if (result.synced_connectors && result.synced_connectors.length > 0) { + toast.error( + `Sync started for ${result.synced_connectors.join(", ")}, but some connectors failed to sync.`, + ); + } else { + toast.error("Some connectors failed to sync"); + } + } else if ( + result.synced_connectors && + result.synced_connectors.length > 0 + ) { + toast.success( + `Sync started for ${result.synced_connectors.join(", ")}. Check task notifications for progress.`, + ); }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@frontend/components/knowledge-search-bar.tsx` around lines 81 - 90, The current conditional in the knowledge-search-bar.tsx branch hides failures when both result.synced_connectors and result.errors exist; change the logic so you explicitly handle partial results by independently checking result.synced_connectors and result.errors (rather than using else-if), calling toast.success with the joined result.synced_connectors when present and toast.error with a clear message (including error details or count) when result.errors is present; update the block around the result handling where toast.success and toast.error are used to allow both to run so partial syncs show both success and failure notifications.frontend/app/api/mutations/useSyncConnector.ts-123-123 (1)
123-123:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winEncode
connectorTypein the preview URL path.Use
encodeURIComponent(connectorType)when building the endpoint path to avoid malformed requests for values containing reserved characters.Suggested fix
- `/api/connectors/${connectorType}/sync-preview`, + `/api/connectors/${encodeURIComponent(connectorType)}/sync-preview`,🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@frontend/app/api/mutations/useSyncConnector.ts` at line 123, The preview endpoint currently interpolates connectorType directly into the path (`/api/connectors/${connectorType}/sync-preview`), which can break for reserved characters; update the request URL construction in useSyncConnector (where connectorType is used) to call encodeURIComponent(connectorType) so the path becomes `/api/connectors/${encodeURIComponent(connectorType)}/sync-preview`, ensuring the connectorType is safely encoded before sending the request.tests/unit/test_processors_clear_stale_chunks.py-32-34 (1)
32-34:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winFix misleading inline comment about stale-chunk cleanup path.
At Line 33, the comment says the re-index path is where
delete_by_query“must fire,” which contradicts the test’s DLS-safe behavior and can mislead future maintenance.📝 Suggested fix
- # exists() is checked at the top of process_document_standard; returning - # False forces the re-index path (the path where delete_by_query must fire). + # exists() is checked at the top of process_document_standard; returning + # False forces the re-index path (where stale chunks are enumerated and + # deleted one-by-one by primary ID).🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tests/unit/test_processors_clear_stale_chunks.py` around lines 32 - 34, The inline comment above opensearch_client.exists = AsyncMock(return_value=False) is misleading about the "re-index path" and delete_by_query; update the comment near that AsyncMock to state that returning False forces the re-index code path in process_document_standard but, due to DLS safety in tests, delete_by_query should not actually run (or is prevented), so clarify that this test asserts DLS-safe behavior rather than expecting delete_by_query to fire. Mention process_document_standard, opensearch_client.exists, and delete_by_query to make the intent explicit.src/api/connectors.py-1077-1081 (1)
1077-1081:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winReturn
Nonewhen orphan detection is unavailable.This branch says filename-only fallback can't determine orphans safely, but it returns
[]. The callers useorphans is not Noneto populateorphans_available, so the preview currently reports “available with zero orphans” instead of “unavailable”.💡 Suggested fix
- return [], synced_count + return None, synced_count🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/api/connectors.py` around lines 1077 - 1081, The current branch in connectors.py returns an empty list for orphans when filename-only fallback can't detect orphans (existing_file_ids is falsy), causing callers that check `orphans is not None` to treat orphans as "available"; change the return to return None for the orphans value (i.e., return None, synced_count) so callers can differentiate "unavailable" from "zero orphans" — update the branch where `existing_file_ids` is checked and the `synced_count` calculation around those variables (`synced_count`, `existing_file_ids`, `existing_filenames`) to return None for orphans instead of an empty list.src/connectors/service.py-407-419 (1)
407-419:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winDon't fall back to raw folder IDs after expansion errors.
If
file_infosonly contains folders, this branch restoresfile_idsand the processor later tries to fetch folders as downloadable files. Keep the filtered list empty here and raise the same “No files to sync after expanding folders” validation error instead.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/connectors/service.py` around lines 407 - 419, The current except block falls back to raw file_ids when expansion fails, which can reintroduce folder IDs; in the except handler for list_files() keep using the filtered non_folder_ids (possibly empty) instead of restoring file_ids: compute non_folder_ids = [f["id"] for f in file_infos if f.get("id") and not f.get("isFolder")], set expanded_file_ids = non_folder_ids (do not use file_ids), and if expanded_file_ids is empty raise the same validation error (e.g., raise ValueError("No files to sync after expanding folders")) so the higher-level validation still blocks folders-only cases; continue to re-raise ValueError early as currently done.
🧹 Nitpick comments (2)
kubernetes/operator/Dockerfile (1)
14-14: ⚡ Quick winConsider making the
TARGETARCHfallback optional inkubernetes/operator/DockerfileThe repo’s operator image build paths already pass
TARGETARCH(e.g.,.github/workflows/operator-release.ymlsetsTARGETARCH=${{ matrix.arch }},.github/workflows/operator-ci.ymlsetsTARGETARCH=amd64, andkubernetes/operator/Makefilepasses--build-arg TARGETARCH=$$(go env GOARCH)for the localdocker-buildtarget). For the multi-archbuildxtarget,--platform linux/amd64,linux/arm64should supplyTARGETARCHautomatically, so the build-break risk is limited to ad-hocdocker buildusage.If you still want extra resilience, keep the small fallback:
Proposed fix
-RUN CGO_ENABLED=0 GOOS=${TARGETOS:-linux} GOARCH=${TARGETARCH} go build -a -o manager cmd/main.go +RUN CGO_ENABLED=0 GOOS=${TARGETOS:-linux} GOARCH=${TARGETARCH:-amd64} go build -a -o manager cmd/main.go🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@kubernetes/operator/Dockerfile` at line 14, The RUN build line currently hardcodes a fallback for TARGETARCH which can mask buildx/CI-provided args; update the RUN invocation that builds the binary (the line using CGO_ENABLED=0 GOOS=${TARGETOS:-linux} GOARCH=${TARGETARCH} go build -a -o manager cmd/main.go) so GOARCH relies solely on the build-arg TARGETARCH (remove any ":-" default expansion if present) or explicitly document/keep a minimal fallback only for ad-hoc local docker build; ensure the change targets that RUN command so multi-arch buildx and CI-provided TARGETARCH values are not overridden.tests/unit/test_skip_os_security_default.py (1)
22-40: ⚡ Quick winAdd coverage for explicit
OPENRAG_SKIP_OS_SECURITY_SETUPoverride.The docstring promises explicit override precedence, but this test only checks run-mode defaults.
🧪 Proposed test extension
`@pytest.mark.parametrize`( - "run_mode, expected", + "run_mode, explicit_skip, expected", [ - ("", "false"), # unset -> default oss path - ("oss", "false"), - ("OSS", "false"), - ("saas", "true"), - ("SaaS", "true"), - ("on_prem", "true"), - ("ON_PREM", "true"), - ("unknown-mode", "false"), # unrecognised falls back to false + ("", None, "false"), # unset -> default oss path + ("oss", None, "false"), + ("OSS", None, "false"), + ("saas", None, "true"), + ("SaaS", None, "true"), + ("on_prem", None, "true"), + ("ON_PREM", None, "true"), + ("unknown-mode", None, "false"), + ("saas", "false", "false"), # explicit override wins + ("oss", "true", "true"), # explicit override wins ], ) -def test_default_resolves_from_run_mode(monkeypatch, run_mode, expected): +def test_default_resolves_from_run_mode(monkeypatch, run_mode, explicit_skip, expected): if run_mode: monkeypatch.setenv("OPENRAG_RUN_MODE", run_mode) else: monkeypatch.delenv("OPENRAG_RUN_MODE", raising=False) + if explicit_skip is not None: + monkeypatch.setenv("OPENRAG_SKIP_OS_SECURITY_SETUP", explicit_skip) + else: + monkeypatch.delenv("OPENRAG_SKIP_OS_SECURITY_SETUP", raising=False) assert _resolve_skip_os_security_default() == expected🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tests/unit/test_skip_os_security_default.py` around lines 22 - 40, Extend the test_default_resolves_from_run_mode test (or add a new test in tests/unit/test_skip_os_security_default.py) to cover explicit OPENRAG_SKIP_OS_SECURITY_SETUP override: for several run_mode values (e.g., "oss", "saas") set monkeypatch.setenv("OPENRAG_SKIP_OS_SECURITY_SETUP", "true") and assert _resolve_skip_os_security_default() returns "true", then set it to "false" and assert it returns "false"; also include a case where OPENRAG_SKIP_OS_SECURITY_SETUP is unset to preserve the existing run_mode-based behavior so the explicit env var takes precedence over _resolve_skip_os_security_default()'s run-mode logic.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/autofix.ci.yml:
- Around line 66-73: The workflow currently injects PR-derived filenames
directly into the `run:` lines (`uv run ruff check --fix --extend-ignore F401
${{ steps.changed.outputs.files }}` and `uv run ruff format ${{
steps.changed.outputs.files }}`) which is unsafe; change the producer step that
sets `steps.changed.outputs.files` to serialize the filenames as a structured
value (e.g., JSON array or a newline-delimited block) and then in the `run:`
script reconstruct a safe argv array (e.g., read/parse the output into a shell
array and invoke `uv run ruff check --fix --extend-ignore F401 -- "${files[@]}"`
and similarly for `uv run ruff format -- "${files[@]}"`) so filenames are
properly quoted and cannot break argument boundaries when using
`steps.changed.outputs.files`.
In @.github/workflows/lint-frontend.yml:
- Around line 31-50: The workflow uses floating action refs and leaves checkout
credentials persisted: replace floating refs actions/checkout@v4 and
actions/setup-node@v4 with pinned references (e.g., full commit SHAs) for both
jobs (biome and typecheck) and add persist-credentials: false to each checkout
step; update the checkout step(s) under the "typecheck" job (and the matching
"biome" job) so the actions/checkout invocation includes persist-credentials:
false and swap actions/setup-node to a pinned SHA to prevent unexpected updates.
In `@frontend/app/knowledge/page.tsx`:
- Around line 151-165: The current else-if chain hides connector errors when
result.synced_connectors exists; update the logic around result (the object
returned), removing the mutually exclusive else-if so that after showing
toast.success for result.synced_connectors (when result.synced_connectors &&
result.synced_connectors.length > 0) you still check result.errors
(result.errors && result.errors.length > 0) and call toast.error to surface
partial-sync failures; keep the existing messages but ensure both success and
error toasts can appear for the same response.
In `@frontend/components/sync-confirm-dialog.tsx`:
- Around line 69-72: The handleConfirm handler can be invoked multiple times
before parent isSyncing updates; add a local in-flight guard (e.g., a
component-scoped boolean or useRef like isSubmitting) to return early if a
submission is already in progress, set the guard true immediately before
awaiting onConfirm(), and reset it in a finally block (so onOpenChange(false)
still runs appropriately). Apply the same pattern to the other submit handler
referenced at lines 214-215 to ensure duplicate sync jobs can't be queued;
reference the handlers by name (handleConfirm and the other submit handler) and
use onConfirm and onOpenChange as before.
In `@frontend/components/task-error-content.tsx`:
- Around line 81-92: The accordion trigger currently contains a native <button>
(the IncidentReporterIcon button) which creates nested interactive controls
because AccordionTrigger wraps AccordionPrimitive.Trigger directly; move the
incident-report button out of the trigger so the trigger's children are
non-interactive (or replace the nested <button> with a non-interactive element
like a <span> or <div> with appropriate role if you must keep it inside).
Concretely: in the component that builds the accordionTrigger prop (the code
rendering IncidentReporterIcon and the surrounding <button>), remove that
<button> from the accordionTrigger and render it as a sibling element alongside
<AccordionTrigger> in TaskErrorContent, preserving any event handlers (e.g.,
stopPropagation/preventDefault) on the relocated control; alternatively, if you
prefer nested action, change AccordionTrigger to accept asChild and forward to
AccordionPrimitive.Trigger so nested buttons are allowed, but the simpler fix is
to extract the IncidentReporterIcon button out of the accordionTrigger.
In `@kubernetes/operator/internal/controller/deletion_handler.go`:
- Around line 175-445: deleteResources currently swallows all Get/Update/Delete
errors and always returns nil so the OpenRAG finalizer can be removed even if
cleanup failed; update deleteResources (and ensure handleDeletion checks its
error) to propagate failures: any r.Update or r.Delete (and any non-NotFound
r.Get error) should cause deleteResources to return a non-nil error (either by
returning immediately on first failure or by aggregating errors and returning a
combined error) so reconciliation will retry and the CR finalizer is only
removed when cleanup actually succeeds; reference the deleteResources function
and all uses of r.Get, r.Update, r.Delete and controllerutil.RemoveFinalizer in
the function to locate places to return/aggregate errors.
In `@kubernetes/operator/internal/controller/openrag_controller.go`:
- Around line 511-519: reconcileNetworkPolicy() currently only allows egress to
external Docling when o.Spec.Docling is set, but the controller sets
DOCLING_SERVE_URL to the operator-managed docling-serve when
o.Spec.DoclingComponents is enabled; update reconcileNetworkPolicy() to also
permit egress to the operator-managed Docling when o.Spec.DoclingComponents !=
nil && o.Spec.DoclingComponents.Enabled && o.Spec.DoclingComponents.Serve != nil
(mirror the logic used when constructing DOCLING_SERVE_URL), using the same
service identity produced by getServiceName(o, "ds") and port selection logic so
Langflow→Docling traffic isn't blocked when networkPolicy.enabled=true and
doclingComponents.enabled=true.
In `@src/api/connectors.py`:
- Around line 80-285: The API module contains business logic for orphan
reconciliation; move compute_orphans_for_connector_type,
delete_orphan_documents, reconcile_orphans_for_connector_type, and
get_synced_id_to_filename_map out of src/api/connectors.py into a new or
existing connector service module (e.g., ConnectorOrphanService) so the API
layer only calls service methods; update the API code to import and call
ConnectorOrphanService.compute_orphans_for_connector_type,
.delete_orphan_documents, .reconcile_orphans_for_connector_type and
.get_synced_id_to_filename_map, preserving signatures (connector_type, user_id,
connector_service, session_manager, jwt_token, existing_file_ids,
id_to_filename) and ensuring logging/error handling is kept in the service; add
unit tests for the new service functions and adjust any imports/exports so
callers compile.
In `@src/auth/ibm_auth.py`:
- Around line 40-49: On cache misses the code decodes the JWT with
jwt.decode(..., options={"verify_signature": False}) but never enforces the exp
claim; update the logic after jwt.decode(token, ...) to check claims.get("exp")
(or 0) against time.time(), and only cache and return the claims when exp >
time.time(); if expired, do not store in _IBM_JWT_CLAIMS_CACHE and instead treat
it as an invalid/expired token (evict/return None or raise as the surrounding
logic expects).
In `@src/connectors/langflow_connector_service.py`:
- Around line 82-124: The current code calls collect_visible_document_ids and
delete_document_ids before re-ingest which can remove the only searchable copy
if the new ingest fails; change the flow in the block that calls
collect_visible_document_ids/delete_document_ids (the code under
self.session_manager using get_user_opensearch_client) so deletion happens only
after a successful ingest: either (A) write the new chunks first under a
temp/staging document id or separate index (or add new document_id suffix),
verify the new chunks were indexed successfully, then call delete_document_ids
to remove the old chunk ids and finally rename/swap document ids or alias if
needed; or (B) perform the deletion in a post-success cleanup path and implement
a rollback that re-inserts the old chunks if downstream steps fail. Keep the
enumerate-then-delete approach (collect_visible_document_ids ->
delete_document_ids) and ensure errors during deletion are still logged, but do
not remove the existing chunks until the ingest completion path (success/commit)
is confirmed.
In `@src/connectors/service.py`:
- Around line 241-243: The pagination key used in the sync loop is inconsistent
with the connector contract: change the code that advances page_token after
calling connector.list_files to read file_list.get("next_page_token") (not
"nextPageToken"), and ensure page_token is set to that value (or None/empty)
before the next iteration; update the other similar loop that calls
connector.list_files (the one using page_token, page_size and logger.debug) to
use the same "next_page_token" key so pagination continues across pages.
In `@src/connectors/sharepoint/connector.py`:
- Around line 28-45: The constructor calls super().__init__(config) twice and
does that before normalizing None, so remove the initial early call and only
invoke super().__init__(config) once after the None guard; update the SharePoint
connector __init__ so it checks "if config is None: config = {}", logs as
needed, then calls super().__init__(config) inside the try/except (referencing
the __init__ method and the super().__init__ call) to avoid double
initialization and ensure a non-None config is passed to the base.
In `@src/models/processors.py`:
- Around line 553-563: The current replace-duplicates path calls
delete_document_by_filename(), which uses delete_by_query and can leave
DLS-protected chunks behind; update the replacement flow to use the DLS-safe
enumerate-then-delete helpers instead of delete_document_by_filename.
Specifically, in the branch that obtains opensearch_client via
self.document_service.session_manager.get_user_opensearch_client(self.user_id,
self.jwt_token) and when replace_duplicates is true, replace the call to
delete_document_by_filename(document.filename, opensearch_client) with the
helper(s) that enumerate matching documents (by filename) and delete them by id
in a DLS-safe way (reuse the same enumerate-then-delete helper used elsewhere in
the file), and make the identical change in the second occurrence that parallels
this logic (the other branch around replace_duplicates).
- Around line 306-335: Right now stale chunks are deleted before writing new
chunks, risking data loss on write failure; change the flow in the block that
calls collect_visible_document_ids and delete_document_ids so you first call
collect_visible_document_ids (using opensearch_client, index=get_index_name(),
query={"term":{"document_id": file_hash}}) to capture stale_chunk_ids, then
perform the replacement upsert/write of new chunks, then compute leftover_ids =
stale_chunk_ids - set(new_chunk_ids) and only call
delete_document_ids(opensearch_client, index=get_index_name(),
document_ids=leftover_ids, refresh=True); keep error handling around deletion
but do not delete prior to successful write.
In `@src/services/docling_polling_service.py`:
- Around line 57-59: The poll loop currently calls check_task_status without
handling transient exceptions and only checks max_seconds after the call, so
network/5xx/NOT_FOUND errors escape and a hanging status call can bypass the
deadline; update the polling logic (the loop that calls check_task_status) to:
wrap each check_task_status invocation in a bounded timeout (e.g.,
asyncio.wait_for or equivalent) so a hung call cannot overrun max_seconds, catch
transient exceptions (network errors, 5xx, temporary NOT_FOUND) and
decrement/track transient_retry_budget to retry with backoff instead of failing
immediately, and ensure the overall deadline (max_seconds) is checked before
each attempt and after a timeout/exception so retries stop when the budget or
deadline is exhausted. Ensure these changes reference the existing
variables/functions transient_retry_budget, max_seconds, and check_task_status
so behavior is consistent with the docstring.
In `@src/services/docling_service.py`:
- Around line 213-304: The check_task_status and fetch_task_result methods call
client.get without IBM auth headers, causing IBM-auth mode to misclassify valid
tasks; update both calls (in check_task_status and fetch_task_result) to pass
the authentication headers produced by the existing helper (e.g.
headers=self._get_auth_headers(...)) to client.get so the requests include IBM
auth; ensure you use the same _get_auth_headers invocation/params as the other
polling path to preserve behavior.
In `@src/services/langflow_file_service.py`:
- Around line 596-605: Wrap the call to submit_to_docling(...) in a try/except
so that if it raises you update the in-memory/tracked file_task (when file_task
is not None) to a terminal failure state: set file_task.phase =
IngestionPhase.DOCLING (if not already), file_task.docling_status =
DoclingPhaseStatus.FAILED and optionally clear/leave docling_task_id unset;
persist/update the file_task record (use the existing save/update mechanism used
elsewhere) before re-raising or returning the exception so the task does not
remain DOCLING/PENDING. Ensure this handling surrounds the submit_to_docling
call and still sets PROCESSING/docling_task_id on success as currently
implemented.
In `@src/utils/file_utils.py`:
- Around line 129-132: The current blanket conversion of spaces/slashes to
underscores (aliases.extend(...)) can create ambiguous collisions (e.g., "Q1
Report.pdf" vs "Q1_Report.pdf"); change the logic to deduplicate and only add an
underscore/slash-normalized alias when it would not collide with an existing
name: build a set of existing alias/original names (from aliases), for each name
produce the candidate name.replace(" ", "_").replace("/", "_") and only add that
candidate if candidate != name and candidate not in the existing set; update
aliases via the set/extend accordingly so clean_connector_filename behavior is
mirrored but collisions are avoided.
In `@src/utils/opensearch_init.py`:
- Around line 108-112: The startup log in opensearch_init.py currently includes
the credential identifier admin_username as an extra field on the logger.info
call; remove the admin_username argument from the logger.info invocation (and
any similar logging around init_index or in the function/method that calls
logger.info) so the message remains but no credential-related identifiers are
emitted, and run a quick grep for admin_username usage in this module to ensure
no other startup logs include sensitive identifiers.
---
Outside diff comments:
In `@frontend/components/cloud-picker/provider-handlers.ts`:
- Around line 188-213: In openPicker, emit the picker-open state before calling
window.OneDrive.open by invoking this.onPickerStateChange?.(true) just prior to
the OneDrive.open call so state transitions are symmetric with the existing
this.onPickerStateChange?.(false) exit paths; ensure you use optional chaining
to avoid errors when onPickerStateChange is undefined and keep the call inside
the early return guard that checks window.OneDrive.
In `@src/connectors/onedrive/connector.py`:
- Around line 27-43: The constructor currently calls super().__init__(config)
twice and before normalizing config; change OneDrive connector __init__ so you
first normalize the incoming config (if config is None set config = {} and log
as needed) and then call super().__init__(config) exactly once inside the
try/except block; remove the earlier unconditional super().__init__(config) call
and keep the existing logger.debug/ logger.error messages around the single
initialization to preserve diagnostics (refer to the __init__ method in the
OneDrive connector for the symbols to update).
- Around line 1-12: The module currently imports the stdlib logging and defines
logger = logging.getLogger(__name__); replace that with the repository logger by
importing get_logger from utils.logging_config and create logger =
get_logger(__name__) instead. Locate the top-level logger variable in this file
(symbol: logger) and swap the import of logging for from utils.logging_config
import get_logger, leaving all uses of logger (e.g., inside methods of
BaseConnector-derived classes or functions interacting with OneDriveOAuth,
ConnectorDocument, DocumentACL) unchanged so the module uses the centralized
logging configuration.
In `@src/connectors/onedrive/oauth.py`:
- Around line 1-8: Replace the module's local stdlib logger initialization with
the shared logger factory: import get_logger from utils.logging_config and
replace the line that sets logger = logging.getLogger(__name__) with logger =
get_logger(__name__) so this module uses the repo-wide logging configuration
(referencing the symbols get_logger and the module-level logger variable).
In `@src/services/task_service.py`:
- Around line 245-275: The issue is that when reusing an existing UploadTask
(existing_task_id path) you start a new background_task but only assign
upload_task.background_task for newly created tasks, so appended batches remain
uncancelable; update the code after creating background_task (the
asyncio.create_task call that invokes background_custom_processor) to always set
upload_task.background_task = background_task for both branches (when
upload_task comes from self.task_store[...] and when it's newly created) so
cancel_task() (which expects upload_task.background_task) will cancel the most
recent background job.
---
Minor comments:
In `@frontend/app/api/mutations/useSyncConnector.ts`:
- Line 123: The preview endpoint currently interpolates connectorType directly
into the path (`/api/connectors/${connectorType}/sync-preview`), which can break
for reserved characters; update the request URL construction in useSyncConnector
(where connectorType is used) to call encodeURIComponent(connectorType) so the
path becomes
`/api/connectors/${encodeURIComponent(connectorType)}/sync-preview`, ensuring
the connectorType is safely encoded before sending the request.
In `@frontend/app/chat/_types/types.ts`:
- Around line 24-28: INITIAL_ASSISTANT_MESSAGE is a singleton object that
captures timestamp at module load and can leak shared mutable state; replace it
with a factory function (e.g., createInitialAssistantMessage or
getInitialAssistantMessage) that returns a new Message object with role
"assistant", content "How can I assist?" and timestamp: new Date() each time,
export that factory instead of the constant, and update callers to invoke the
factory so each use gets a fresh Message instance.
In `@frontend/components/duplicate-handling-dialog.tsx`:
- Around line 82-85: The list items use key={name}, but names may repeat
(duplicateNames derived from originalFile.name.split("/").pop()), causing
non-unique React keys; update the render for visibleNames so each <li> uses a
unique key (e.g., combine the filename with its index or a derived unique id) by
referencing the map callback index or the original duplicateNames index when
mapping over visibleNames to produce keys for the <li> elements.
In `@frontend/components/knowledge-search-bar.tsx`:
- Around line 81-90: The current conditional in the knowledge-search-bar.tsx
branch hides failures when both result.synced_connectors and result.errors
exist; change the logic so you explicitly handle partial results by
independently checking result.synced_connectors and result.errors (rather than
using else-if), calling toast.success with the joined result.synced_connectors
when present and toast.error with a clear message (including error details or
count) when result.errors is present; update the block around the result
handling where toast.success and toast.error are used to allow both to run so
partial syncs show both success and failure notifications.
In `@kubernetes/operator/config/crd/bases/openr.ag_openrags.yaml`:
- Around line 12680-12686: The CRD contains duplicated enum entries because both
the PVCReclaimPolicy type and the pvcReclaimPolicy field have kubebuilder Enum
markers; remove one of the duplicate markers in
kubernetes/operator/api/v1alpha1/openrag_types.go — either delete the `//
+kubebuilder:validation:Enum=Retain;Delete` above the `type PVCReclaimPolicy
string` declaration or remove the Enum marker on the `pvcReclaimPolicy` field
(keep the `// +kubebuilder:default=Retain` if desired), then regenerate the CRD
so the final openr.ag_openrags.yaml has a single `pvcReclaimPolicy.allOf[].enum`
entry.
In `@src/api/connectors.py`:
- Around line 1077-1081: The current branch in connectors.py returns an empty
list for orphans when filename-only fallback can't detect orphans
(existing_file_ids is falsy), causing callers that check `orphans is not None`
to treat orphans as "available"; change the return to return None for the
orphans value (i.e., return None, synced_count) so callers can differentiate
"unavailable" from "zero orphans" — update the branch where `existing_file_ids`
is checked and the `synced_count` calculation around those variables
(`synced_count`, `existing_file_ids`, `existing_filenames`) to return None for
orphans instead of an empty list.
In `@src/connectors/service.py`:
- Around line 407-419: The current except block falls back to raw file_ids when
expansion fails, which can reintroduce folder IDs; in the except handler for
list_files() keep using the filtered non_folder_ids (possibly empty) instead of
restoring file_ids: compute non_folder_ids = [f["id"] for f in file_infos if
f.get("id") and not f.get("isFolder")], set expanded_file_ids = non_folder_ids
(do not use file_ids), and if expanded_file_ids is empty raise the same
validation error (e.g., raise ValueError("No files to sync after expanding
folders")) so the higher-level validation still blocks folders-only cases;
continue to re-raise ValueError early as currently done.
In `@tests/unit/test_processors_clear_stale_chunks.py`:
- Around line 32-34: The inline comment above opensearch_client.exists =
AsyncMock(return_value=False) is misleading about the "re-index path" and
delete_by_query; update the comment near that AsyncMock to state that returning
False forces the re-index code path in process_document_standard but, due to DLS
safety in tests, delete_by_query should not actually run (or is prevented), so
clarify that this test asserts DLS-safe behavior rather than expecting
delete_by_query to fire. Mention process_document_standard,
opensearch_client.exists, and delete_by_query to make the intent explicit.
---
Nitpick comments:
In `@kubernetes/operator/Dockerfile`:
- Line 14: The RUN build line currently hardcodes a fallback for TARGETARCH
which can mask buildx/CI-provided args; update the RUN invocation that builds
the binary (the line using CGO_ENABLED=0 GOOS=${TARGETOS:-linux}
GOARCH=${TARGETARCH} go build -a -o manager cmd/main.go) so GOARCH relies solely
on the build-arg TARGETARCH (remove any ":-" default expansion if present) or
explicitly document/keep a minimal fallback only for ad-hoc local docker build;
ensure the change targets that RUN command so multi-arch buildx and CI-provided
TARGETARCH values are not overridden.
In `@tests/unit/test_skip_os_security_default.py`:
- Around line 22-40: Extend the test_default_resolves_from_run_mode test (or add
a new test in tests/unit/test_skip_os_security_default.py) to cover explicit
OPENRAG_SKIP_OS_SECURITY_SETUP override: for several run_mode values (e.g.,
"oss", "saas") set monkeypatch.setenv("OPENRAG_SKIP_OS_SECURITY_SETUP", "true")
and assert _resolve_skip_os_security_default() returns "true", then set it to
"false" and assert it returns "false"; also include a case where
OPENRAG_SKIP_OS_SECURITY_SETUP is unset to preserve the existing run_mode-based
behavior so the explicit env var takes precedence over
_resolve_skip_os_security_default()'s run-mode logic.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 5edf9776-7c14-4141-822a-04a745af9b46
⛔ Files ignored due to path filters (2)
frontend/package-lock.jsonis excluded by!**/package-lock.jsonsdks/mcp/uv.lockis excluded by!**/*.lock
📒 Files selected for processing (125)
.github/workflows/autofix.ci.yml.github/workflows/lint-backend.yml.github/workflows/lint-frontend.yml.github/workflows/operator-ci.yml.github/workflows/operator-release.yml.gitignoreDockerfile.langflowMakefilefrontend/app/api/mutations/useConnectConnectorMutation.tsfrontend/app/api/mutations/useSyncConnector.tsfrontend/app/api/queries/useGetSearchQuery.tsfrontend/app/chat/_components/assistant-message.tsxfrontend/app/chat/_components/message-actions.tsxfrontend/app/chat/_types/types.tsfrontend/app/chat/page.tsxfrontend/app/globals.cssfrontend/app/knowledge/page.tsxfrontend/app/onboarding/_components/onboarding-card.tsxfrontend/app/onboarding/_components/onboarding-content.tsxfrontend/app/onboarding/_components/onboarding-upload.tsxfrontend/app/settings/_components/connector-cards.tsxfrontend/app/settings/_components/settings-nav.tsxfrontend/app/settings/_components/settings-shell.tsxfrontend/app/upload/[provider]/page.tsxfrontend/components/chat-renderer.tsxfrontend/components/cloud-picker/file-item.tsxfrontend/components/cloud-picker/onedrive-v8-handler.tsfrontend/components/cloud-picker/provider-handlers.tsfrontend/components/cloud-picker/sharepoint-v8-handler.tsfrontend/components/duplicate-handling-dialog.tsxfrontend/components/icons/incident-reporter-icon.tsxfrontend/components/knowledge-actions-dropdown.tsxfrontend/components/knowledge-batch-actions-bar.tsxfrontend/components/knowledge-dropdown.tsxfrontend/components/knowledge-search-bar.tsxfrontend/components/navigation.tsxfrontend/components/sync-confirm-dialog.tsxfrontend/components/task-collapsible-section.tsxfrontend/components/task-error-content.tsxfrontend/components/task-notification-menu.tsxfrontend/components/tasks_details.tsxfrontend/components/ui/settings-tabs.tsxfrontend/components/user-nav.tsxfrontend/contexts/auth-context.tsxfrontend/contexts/chat-context.tsxfrontend/contexts/task-context.tsxfrontend/hooks/use-onboarding-state.tsfrontend/lib/task-error-display.tsfrontend/lib/task-utils.tsfrontend/lib/upload-utils.tsfrontend/package.jsonfrontend/stores/loadingStore.tsfrontend/tailwind.config.tsfrontend/tests/core/tasks-unified-panel.spec.tsfrontend/tests/utils/onboarding.tskubernetes/helm/operator/values.yamlkubernetes/operator/Dockerfilekubernetes/operator/Makefilekubernetes/operator/README.mdkubernetes/operator/api/v1alpha1/openrag_types.gokubernetes/operator/config/crd/bases/openr.ag_openrags.yamlkubernetes/operator/config/samples/openrag_v1alpha1_openrag-kind-local.yamlkubernetes/operator/config/samples/openrag_v1alpha1_openrag.yamlkubernetes/operator/internal/controller/deletion_handler.gokubernetes/operator/internal/controller/env.gokubernetes/operator/internal/controller/env_example_test.gokubernetes/operator/internal/controller/env_test.gokubernetes/operator/internal/controller/openrag_controller.gokubernetes/operator/internal/controller/openrag_controller_test.gokubernetes/operator/internal/controller/status.gosdks/mcp/README.mdsdks/mcp/pyproject.tomlsdks/mcp/src/openrag_mcp/__main__.pysdks/mcp/src/openrag_mcp/config.pysdks/mcp/src/openrag_mcp/server.pysdks/mcp/src/openrag_mcp/tools/__init__.pysdks/mcp/src/openrag_mcp/tools/chat.pysdks/mcp/src/openrag_mcp/tools/documents.pysdks/mcp/src/openrag_mcp/tools/registry.pysdks/mcp/src/openrag_mcp/tools/search.pysdks/mcp/src/openrag_mcp/tools/settings.pysdks/typescript/tests/integration.test.tssrc/api/connector_router.pysrc/api/connectors.pysrc/api/documents.pysrc/api/settings/endpoints.pysrc/api/v1/ingestion_jobs.pysrc/app/container.pysrc/app/lifespan.pysrc/app/routes/internal.pysrc/auth/ibm_auth.pysrc/config/settings.pysrc/connectors/langflow_connector_service.pysrc/connectors/onedrive/connector.pysrc/connectors/onedrive/oauth.pysrc/connectors/service.pysrc/connectors/sharepoint/connector.pysrc/dependencies.pysrc/models/processors.pysrc/models/tasks.pysrc/services/docling_polling_service.pysrc/services/docling_service.pysrc/services/langflow_file_service.pysrc/services/startup_orchestrator.pysrc/services/task_service.pysrc/services/user_service.pysrc/session_manager.pysrc/tui/screens/monitor.pysrc/utils/file_utils.pysrc/utils/opensearch_delete.pysrc/utils/opensearch_init.pytests/unit/api/test_delete_chunks_by_document_ids.pytests/unit/api/test_reconcile_orphans_for_connector_type.pytests/unit/connectors/__init__.pytests/unit/connectors/test_update_connector_metadata_filename.pytests/unit/services/test_skip_os_security_setup_startup.pytests/unit/test_connector_processor_filename_dedupe.pytests/unit/test_docling_polling_service.pytests/unit/test_docling_service_status_check.pytests/unit/test_file_utils_filename_aliases.pytests/unit/test_jwt_claims_cache.pytests/unit/test_langflow_file_service_two_phase.pytests/unit/test_processors_clear_stale_chunks.pytests/unit/test_skip_os_security_default.pytests/unit/test_skip_os_security_setup_init_index.py
💤 Files with no reviewable changes (12)
- frontend/components/cloud-picker/sharepoint-v8-handler.ts
- sdks/mcp/src/openrag_mcp/tools/init.py
- frontend/stores/loadingStore.ts
- sdks/mcp/src/openrag_mcp/main.py
- frontend/components/cloud-picker/onedrive-v8-handler.ts
- sdks/mcp/src/openrag_mcp/server.py
- sdks/mcp/src/openrag_mcp/tools/settings.py
- sdks/mcp/src/openrag_mcp/tools/documents.py
- sdks/mcp/src/openrag_mcp/tools/chat.py
- sdks/mcp/src/openrag_mcp/tools/registry.py
- sdks/mcp/src/openrag_mcp/config.py
- sdks/mcp/src/openrag_mcp/tools/search.py
✅ Files skipped from review due to trivial changes (8)
- kubernetes/helm/operator/values.yaml
- frontend/app/settings/_components/settings-shell.tsx
- frontend/app/onboarding/_components/onboarding-content.tsx
- frontend/package.json
- .github/workflows/lint-backend.yml
- kubernetes/operator/config/samples/openrag_v1alpha1_openrag.yaml
- kubernetes/operator/README.md
- frontend/app/onboarding/_components/onboarding-card.tsx
Comment on lines
+66
to
73
| - name: Ruff lint fix (in-place) | ||
| if: steps.changed.outputs.files != '' | ||
| continue-on-error: true | ||
| run: uv run ruff check --fix --extend-ignore F401 ${{ steps.changed.outputs.files }} | ||
|
|
||
| - name: Ruff format (in-place) | ||
| if: steps.changed.outputs.files != '' | ||
| run: uv run ruff format ${{ steps.changed.outputs.files }} |
Contributor
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Verify unsafe interpolation and serialization pattern.
rg -n '\$\{\{ steps\.changed\.outputs\.files \}\}' .github/workflows/autofix.ci.yml
rg -n 'files=\$\{files\[\*\]\}' .github/workflows/autofix.ci.ymlRepository: langflow-ai/openrag
Length of output: 287
🏁 Script executed:
#!/bin/bash
set -euo pipefail
FILE=".github/workflows/autofix.ci.yml"
# Show the full context around the reported lines
nl -ba "$FILE" | sed -n '40,95p'
# Find where `steps.changed.outputs.files` is set/echoed into $GITHUB_OUTPUT
rg -n "name:.*changed|id: changed|outputs:|GITHUB_OUTPUT|files=\$\{files\[\*\]\}" "$FILE"
# Show the exact block that writes `files=${files[*]}` to GITHUB_OUTPUT
nl -ba "$FILE" | sed -n '55,80p'Repository: langflow-ai/openrag
Length of output: 106
🏁 Script executed:
#!/bin/bash
set -euo pipefail
FILE=".github/workflows/autofix.ci.yml"
# Print context (with line numbers via awk)
awk 'NR>=40 && NR<=95 {printf "%d:%s\n", NR, $0}' "$FILE"
# Locate where outputs are defined and where `files=${files[*]}` is written
rg -n "GITHUB_OUTPUT|id:\s*changed|name:\s*.*changed|outputs:|files=\$\{files\[\*\]\}" "$FILE"
# Show the exact block around `echo "files=${files[*]}"`
rg -n "echo \"files=\$\{files\[\*\]\}\"" -n "$FILE"Repository: langflow-ai/openrag
Length of output: 2106
🏁 Script executed:
#!/bin/bash
set -euo pipefail
FILE=".github/workflows/autofix.ci.yml"
# Print top portion to capture runs-on / job config
awk 'NR<=120 {printf "%d:%s\n", NR, $0}' "$FILE" | head -n 120
# Check for jq usage/installation hints elsewhere in the workflow
rg -n "jq|runs-on|shell:" "$FILE" || trueRepository: langflow-ai/openrag
Length of output: 3154
🏁 Script executed:
#!/bin/bash
set -euo pipefail
FILE=".github/workflows/autofix.ci.yml"
rg -n "jq|shell:" "$FILE" || trueRepository: langflow-ai/openrag
Length of output: 45
Avoid interpolating PR-derived filenames into Ruff run: unquoted.
steps.changed.outputs.files is built from git diff --name-only, then serialized with echo "files=${files[*]}" and injected directly into uv run ruff ... ${{ steps.changed.outputs.files }} (lines 64, 69, 73). Crafted filenames can break argument boundaries and trigger shell parsing/injection. Serialize filenames as a structured list (e.g., JSON array or newline block) and reconstruct a safe argv array (-- "${files[@]}") when invoking ruff.
🧰 Tools
🪛 zizmor (1.25.2)
[info] 69-69: code injection via template expansion (template-injection): may expand into attacker-controllable code
(template-injection)
[info] 73-73: code injection via template expansion (template-injection): may expand into attacker-controllable code
(template-injection)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/autofix.ci.yml around lines 66 - 73, The workflow
currently injects PR-derived filenames directly into the `run:` lines (`uv run
ruff check --fix --extend-ignore F401 ${{ steps.changed.outputs.files }}` and
`uv run ruff format ${{ steps.changed.outputs.files }}`) which is unsafe; change
the producer step that sets `steps.changed.outputs.files` to serialize the
filenames as a structured value (e.g., JSON array or a newline-delimited block)
and then in the `run:` script reconstruct a safe argv array (e.g., read/parse
the output into a shell array and invoke `uv run ruff check --fix
--extend-ignore F401 -- "${files[@]}"` and similarly for `uv run ruff format --
"${files[@]}"`) so filenames are properly quoted and cannot break argument
boundaries when using `steps.changed.outputs.files`.
Comment on lines
+31
to
+50
| typecheck: | ||
| name: TypeScript type check | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
|
|
||
| - name: Set up Node.js | ||
| uses: actions/setup-node@v4 | ||
| with: | ||
| node-version: '20' | ||
| cache: 'npm' | ||
| cache-dependency-path: frontend/package-lock.json | ||
|
|
||
| - name: Install dependencies | ||
| working-directory: frontend | ||
| run: npm ci | ||
|
|
||
| - name: TypeScript type check | ||
| working-directory: frontend | ||
| run: npm run typecheck |
Contributor
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Verify floating refs and missing persist-credentials hardening in this workflow.
rg -n 'uses:\s*actions/(checkout|setup-node)`@v`' .github/workflows/lint-frontend.yml
rg -n -C2 'persist-credentials' .github/workflows/lint-frontend.ymlRepository: langflow-ai/openrag
Length of output: 215
🏁 Script executed:
#!/bin/bash
set -euo pipefail
FILE=".github/workflows/lint-frontend.yml"
wc -l "$FILE"
sed -n '1,120p' "$FILE" | nl -baRepository: langflow-ai/openrag
Length of output: 163
🏁 Script executed:
#!/bin/bash
set -euo pipefail
FILE=".github/workflows/lint-frontend.yml"
awk 'NR>=1 && NR<=120 {printf "%d:%s\n", NR, $0}' "$FILE"
echo "---- persist-credentials matches ----"
rg -n 'persist-credentials' "$FILE" || trueRepository: langflow-ai/openrag
Length of output: 1345
Pin GitHub Actions and disable persisted checkout credentials in CI.
.github/workflows/lint-frontend.yml uses floating action refs (actions/checkout@v4, actions/setup-node@v4) in both biome and typecheck, and the checkout steps do not set persist-credentials: false (avoidable credential/token persistence).
🧰 Tools
🪛 zizmor (1.25.2)
[warning] 35-35: credential persistence through GitHub Actions artifacts (artipacked): does not set persist-credentials: false
(artipacked)
[warning] 31-51: overly broad permissions (excessive-permissions): default permissions used due to no permissions: block
(excessive-permissions)
[error] 35-35: unpinned action reference (unpinned-uses): action is not pinned to a hash (required by blanket policy)
(unpinned-uses)
[error] 38-38: unpinned action reference (unpinned-uses): action is not pinned to a hash (required by blanket policy)
(unpinned-uses)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/lint-frontend.yml around lines 31 - 50, The workflow uses
floating action refs and leaves checkout credentials persisted: replace floating
refs actions/checkout@v4 and actions/setup-node@v4 with pinned references (e.g.,
full commit SHAs) for both jobs (biome and typecheck) and add
persist-credentials: false to each checkout step; update the checkout step(s)
under the "typecheck" job (and the matching "biome" job) so the actions/checkout
invocation includes persist-credentials: false and swap actions/setup-node to a
pinned SHA to prevent unexpected updates.
Comment on lines
+151
to
+165
| if (result.status === "no_files") { | ||
| toast.info( | ||
| result.message || | ||
| "No cloud files to sync. Add files from cloud connectors first.", | ||
| ); | ||
| } else if ( | ||
| result.synced_connectors && | ||
| result.synced_connectors.length > 0 | ||
| ) { | ||
| toast.success( | ||
| `Sync started for ${result.synced_connectors.join(", ")}. Check task notifications for progress.`, | ||
| ); | ||
| } else if (result.errors && result.errors.length > 0) { | ||
| toast.error("Some connectors failed to sync"); | ||
| } |
Contributor
There was a problem hiding this comment.
Handle partial-sync responses without hiding connector errors.
At Line 156, the else if chain suppresses result.errors when result.synced_connectors is also present, so partial failures can look like full success.
💡 Suggested fix
- if (result.status === "no_files") {
+ if (result.status === "no_files") {
toast.info(
result.message ||
"No cloud files to sync. Add files from cloud connectors first.",
);
- } else if (
- result.synced_connectors &&
- result.synced_connectors.length > 0
- ) {
+ }
+
+ if (result.synced_connectors && result.synced_connectors.length > 0) {
toast.success(
`Sync started for ${result.synced_connectors.join(", ")}. Check task notifications for progress.`,
);
- } else if (result.errors && result.errors.length > 0) {
+ }
+
+ if (result.errors && result.errors.length > 0) {
toast.error("Some connectors failed to sync");
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| if (result.status === "no_files") { | |
| toast.info( | |
| result.message || | |
| "No cloud files to sync. Add files from cloud connectors first.", | |
| ); | |
| } else if ( | |
| result.synced_connectors && | |
| result.synced_connectors.length > 0 | |
| ) { | |
| toast.success( | |
| `Sync started for ${result.synced_connectors.join(", ")}. Check task notifications for progress.`, | |
| ); | |
| } else if (result.errors && result.errors.length > 0) { | |
| toast.error("Some connectors failed to sync"); | |
| } | |
| if (result.status === "no_files") { | |
| toast.info( | |
| result.message || | |
| "No cloud files to sync. Add files from cloud connectors first.", | |
| ); | |
| } | |
| if (result.synced_connectors && result.synced_connectors.length > 0) { | |
| toast.success( | |
| `Sync started for ${result.synced_connectors.join(", ")}. Check task notifications for progress.`, | |
| ); | |
| } | |
| if (result.errors && result.errors.length > 0) { | |
| toast.error("Some connectors failed to sync"); | |
| } |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@frontend/app/knowledge/page.tsx` around lines 151 - 165, The current else-if
chain hides connector errors when result.synced_connectors exists; update the
logic around result (the object returned), removing the mutually exclusive
else-if so that after showing toast.success for result.synced_connectors (when
result.synced_connectors && result.synced_connectors.length > 0) you still check
result.errors (result.errors && result.errors.length > 0) and call toast.error
to surface partial-sync failures; keep the existing messages but ensure both
success and error toasts can appear for the same response.
Comment on lines
+69
to
+72
| const handleConfirm = async () => { | ||
| await onConfirm(); | ||
| onOpenChange(false); | ||
| }; |
Contributor
There was a problem hiding this comment.
Add a local in-flight guard to prevent duplicate sync submissions.
onConfirm can be triggered multiple times if users double-click before isSyncing propagates from parent state, which can enqueue duplicate sync jobs.
Suggested fix
-import type React from "react";
+import { useState, type React } from "react";
@@
}) => {
+ const [isSubmitting, setIsSubmitting] = useState(false);
+
const handleConfirm = async () => {
- await onConfirm();
- onOpenChange(false);
+ if (isSubmitting || isLoading || isSyncing) return;
+ setIsSubmitting(true);
+ try {
+ await onConfirm();
+ onOpenChange(false);
+ } finally {
+ setIsSubmitting(false);
+ }
};
@@
- const busy = isLoading || isSyncing;
+ const busy = isLoading || isSyncing || isSubmitting;Also applies to: 214-215
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@frontend/components/sync-confirm-dialog.tsx` around lines 69 - 72, The
handleConfirm handler can be invoked multiple times before parent isSyncing
updates; add a local in-flight guard (e.g., a component-scoped boolean or useRef
like isSubmitting) to return early if a submission is already in progress, set
the guard true immediately before awaiting onConfirm(), and reset it in a
finally block (so onOpenChange(false) still runs appropriately). Apply the same
pattern to the other submit handler referenced at lines 214-215 to ensure
duplicate sync jobs can't be queued; reference the handlers by name
(handleConfirm and the other submit handler) and use onConfirm and onOpenChange
as before.
Comment on lines
+81
to
+92
| <button | ||
| type="button" | ||
| aria-label="Report incident" | ||
| className="inline-flex shrink-0 items-center justify-center text-muted-foreground hover:text-foreground" | ||
| onClick={(event) => { | ||
| event.preventDefault(); | ||
| event.stopPropagation(); | ||
| }} | ||
| onPointerDown={(event) => event.stopPropagation()} | ||
| > | ||
| <IncidentReporterIcon className="size-4" /> | ||
| </button> |
Contributor
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "Inspect AccordionTrigger implementation(s):"
fd -i 'accordion.tsx' | while read -r file; do
echo "--- $file ---"
rg -n "AccordionTrigger|Primitive.Trigger|button" "$file" -C2
done
echo
echo "Inspect nested control usage in task-error-content:"
rg -n "AccordionTrigger|Report incident|<button" frontend/components/task-error-content.tsx -C2Repository: langflow-ai/openrag
Length of output: 2201
Avoid nested interactive controls in AccordionTrigger
In frontend/components/task-error-content.tsx, the accordionTrigger passed into <AccordionTrigger> includes a nested <button aria-label="Report incident" ...> (around 81–92) which is then rendered inside the trigger (around 153–163). frontend/components/ui/accordion.tsx’s AccordionTrigger wraps AccordionPrimitive.Trigger directly (no asChild), so the trigger itself is already an interactive control—this creates nested interactive elements (button-inside-trigger) and can break keyboard/click behavior. Move the incident action outside the trigger (separate surface) or ensure the trigger children are non-button elements.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@frontend/components/task-error-content.tsx` around lines 81 - 92, The
accordion trigger currently contains a native <button> (the IncidentReporterIcon
button) which creates nested interactive controls because AccordionTrigger wraps
AccordionPrimitive.Trigger directly; move the incident-report button out of the
trigger so the trigger's children are non-interactive (or replace the nested
<button> with a non-interactive element like a <span> or <div> with appropriate
role if you must keep it inside). Concretely: in the component that builds the
accordionTrigger prop (the code rendering IncidentReporterIcon and the
surrounding <button>), remove that <button> from the accordionTrigger and render
it as a sibling element alongside <AccordionTrigger> in TaskErrorContent,
preserving any event handlers (e.g., stopPropagation/preventDefault) on the
relocated control; alternatively, if you prefer nested action, change
AccordionTrigger to accept asChild and forward to AccordionPrimitive.Trigger so
nested buttons are allowed, but the simpler fix is to extract the
IncidentReporterIcon button out of the accordionTrigger.
Comment on lines
+57
to
+59
| Transient errors (network, 5xx, NOT_FOUND seen briefly before the task | ||
| is registered server-side) are absorbed up to ``transient_retry_budget`` | ||
| before being surfaced as failures. The interval grows by |
Contributor
There was a problem hiding this comment.
Transient failures are not retried as documented, and poll deadline can be bypassed by a hanging status call.
Lines 57-59 promise transient error absorption, but Line 78 has no exception handling, so a single network/5xx error exits immediately. Also, max_seconds is only checked after check_task_status returns, so a hung call can overrun the deadline.
💡 Suggested patch shape
class DoclingPollingService:
@@
async def poll_until_ready(...):
@@
- consecutive_not_found = 0
+ consecutive_not_found = 0
+ transient_errors = 0
@@
- snapshot = await self.docling_service.check_task_status(task_id)
+ remaining = deadline - time.monotonic()
+ if remaining <= 0:
+ return DoclingPollResult(
+ outcome=PollOutcome.TIMEOUT,
+ detail=f"Docling polling timed out after {max_seconds}s",
+ last_snapshot=last_snapshot,
+ elapsed_seconds=time.monotonic() - start,
+ )
+ try:
+ snapshot = await asyncio.wait_for(
+ self.docling_service.check_task_status(task_id),
+ timeout=remaining,
+ )
+ transient_errors = 0
+ except Exception as exc:
+ transient_errors += 1
+ if transient_errors > transient_retry_budget:
+ return DoclingPollResult(
+ outcome=PollOutcome.FAILED,
+ detail=f"Docling status check failed: {exc}",
+ last_snapshot=last_snapshot,
+ elapsed_seconds=time.monotonic() - start,
+ )
+ await asyncio.sleep(min(interval, remaining))
+ interval = min(interval * backoff_factor, max_interval)
+ continue
@@
- remaining = deadline - time.monotonic()
- if remaining <= 0:
- ...
await asyncio.sleep(min(interval, remaining))Also applies to: 78-82, 129-142
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/services/docling_polling_service.py` around lines 57 - 59, The poll loop
currently calls check_task_status without handling transient exceptions and only
checks max_seconds after the call, so network/5xx/NOT_FOUND errors escape and a
hanging status call can bypass the deadline; update the polling logic (the loop
that calls check_task_status) to: wrap each check_task_status invocation in a
bounded timeout (e.g., asyncio.wait_for or equivalent) so a hung call cannot
overrun max_seconds, catch transient exceptions (network errors, 5xx, temporary
NOT_FOUND) and decrement/track transient_retry_budget to retry with backoff
instead of failing immediately, and ensure the overall deadline (max_seconds) is
checked before each attempt and after a timeout/exception so retries stop when
the budget or deadline is exhausted. Ensure these changes reference the existing
variables/functions transient_retry_budget, max_seconds, and check_task_status
so behavior is consistent with the docstring.
Comment on lines
+213
to
+304
| async def check_task_status(self, task_id: str) -> DoclingStatusSnapshot: | ||
| """ | ||
| Single (non-blocking) status check against Docling Serve. | ||
|
|
||
| Used by the backend polling coordinator so that the polling loop lives | ||
| in OpenRAG and not inside Langflow. Maps the Docling Serve response | ||
| into a DoclingStatusSnapshot regardless of HTTP outcome. | ||
| """ | ||
| client = self._get_client() | ||
| url = f"{self.docling_url}/v1/status/poll/{task_id}" | ||
| try: | ||
| response = await client.get(url) | ||
| except httpx.RequestError as e: | ||
| # Transient network error — surface as PROCESSING so caller can | ||
| # retry without prematurely failing the file. | ||
| logger.debug("Transient error checking docling status", task_id=task_id, error=str(e)) | ||
| return DoclingStatusSnapshot(state=DoclingTaskState.PROCESSING, detail=str(e)) | ||
|
|
||
| if response.status_code == 404: | ||
| return DoclingStatusSnapshot(state=DoclingTaskState.NOT_FOUND, detail="Task not found") | ||
| if response.status_code >= 500: | ||
| logger.debug( | ||
| "Transient HTTP error from docling status endpoint", | ||
| task_id=task_id, | ||
| status_code=response.status_code, | ||
| ) | ||
| return DoclingStatusSnapshot( | ||
| state=DoclingTaskState.PROCESSING, | ||
| detail=f"HTTP {response.status_code}", | ||
| ) | ||
| if response.status_code >= 400: | ||
| return DoclingStatusSnapshot( | ||
| state=DoclingTaskState.FAILED, | ||
| detail=f"HTTP {response.status_code}: {response.text[:300]}", | ||
| ) | ||
|
|
||
| try: | ||
| payload = response.json() | ||
| except ValueError as e: | ||
| return DoclingStatusSnapshot( | ||
| state=DoclingTaskState.FAILED, | ||
| detail=f"Malformed status response: {str(e)}", | ||
| ) | ||
|
|
||
| status = payload.get("task_status") | ||
| if status == "success": | ||
| return DoclingStatusSnapshot(state=DoclingTaskState.SUCCESS, raw=payload) | ||
| if status == "failure": | ||
| return DoclingStatusSnapshot( | ||
| state=DoclingTaskState.FAILED, | ||
| detail=str(payload), | ||
| raw=payload, | ||
| ) | ||
| if status in ("started", "processing", "running"): | ||
| return DoclingStatusSnapshot(state=DoclingTaskState.PROCESSING, raw=payload) | ||
| return DoclingStatusSnapshot(state=DoclingTaskState.PENDING, raw=payload) | ||
|
|
||
| async def fetch_task_result(self, task_id: str) -> dict[str, Any]: | ||
| """ | ||
| Fetch the converted document for a Docling task that is already SUCCESS. | ||
|
|
||
| Raises: | ||
| DoclingServeError: if the result endpoint returns 404 (task expired | ||
| or unknown), an unexpected status code, or a payload missing | ||
| document.json_content. | ||
| """ | ||
| client = self._get_client() | ||
| url = f"{self.docling_url}/v1/result/{task_id}" | ||
| try: | ||
| response = await client.get(url) | ||
| except httpx.RequestError as e: | ||
| raise DoclingServeError(f"Network error fetching docling result: {str(e)}") from e | ||
|
|
||
| if response.status_code == 404: | ||
| raise DoclingServeError( | ||
| f"Docling result not found for task {task_id} (task expired or unknown)" | ||
| ) | ||
| if response.status_code >= 400: | ||
| raise DoclingServeError( | ||
| f"Docling result fetch failed with HTTP {response.status_code}: {response.text[:300]}" | ||
| ) | ||
|
|
||
| try: | ||
| payload = response.json() | ||
| except ValueError as e: | ||
| raise DoclingServeError(f"Malformed docling result payload: {str(e)}") from e | ||
|
|
||
| doc_content = payload.get("document", {}).get("json_content") | ||
| if doc_content is None: | ||
| raise DoclingServeError("docling-serve response missing document.json_content") | ||
| return doc_content | ||
|
|
Contributor
There was a problem hiding this comment.
Propagate IBM auth headers in new Docling polling methods.
Line 224 and Line 282 call Docling endpoints without headers, while the existing polling path uses _get_auth_headers(...). In IBM-auth mode this can turn valid tasks into false FAILED/NOT_FOUND outcomes.
Proposed fix
- async def check_task_status(self, task_id: str) -> DoclingStatusSnapshot:
+ async def check_task_status(
+ self,
+ task_id: str,
+ user_id: str | None = None,
+ auth_header: str | None = None,
+ ) -> DoclingStatusSnapshot:
@@
client = self._get_client()
url = f"{self.docling_url}/v1/status/poll/{task_id}"
+ headers = self._get_auth_headers(user_id, auth_header)
try:
- response = await client.get(url)
+ response = await client.get(url, headers=headers)
@@
- async def fetch_task_result(self, task_id: str) -> dict[str, Any]:
+ async def fetch_task_result(
+ self,
+ task_id: str,
+ user_id: str | None = None,
+ auth_header: str | None = None,
+ ) -> dict[str, Any]:
@@
client = self._get_client()
url = f"{self.docling_url}/v1/result/{task_id}"
+ headers = self._get_auth_headers(user_id, auth_header)
try:
- response = await client.get(url)
+ response = await client.get(url, headers=headers)🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/services/docling_service.py` around lines 213 - 304, The
check_task_status and fetch_task_result methods call client.get without IBM auth
headers, causing IBM-auth mode to misclassify valid tasks; update both calls (in
check_task_status and fetch_task_result) to pass the authentication headers
produced by the existing helper (e.g. headers=self._get_auth_headers(...)) to
client.get so the requests include IBM auth; ensure you use the same
_get_auth_headers invocation/params as the other polling path to preserve
behavior.
Comment on lines
+596
to
+605
| if file_task is not None: | ||
| file_task.phase = IngestionPhase.DOCLING | ||
| file_task.docling_status = DoclingPhaseStatus.PENDING | ||
|
|
||
| task_id = await self.submit_to_docling(filename, content, owner=owner, jwt_token=jwt_token) | ||
|
|
||
| if file_task is not None: | ||
| file_task.docling_task_id = task_id | ||
| file_task.docling_status = DoclingPhaseStatus.PROCESSING | ||
|
|
Contributor
There was a problem hiding this comment.
Mark Docling submission failures as failed in file_task.
If submit_to_docling(...) raises, the tracked task remains DOCLING/PENDING, which can leave status non-terminal after a hard failure.
💡 Suggested fix
if file_task is not None:
file_task.phase = IngestionPhase.DOCLING
file_task.docling_status = DoclingPhaseStatus.PENDING
- task_id = await self.submit_to_docling(filename, content, owner=owner, jwt_token=jwt_token)
+ try:
+ task_id = await self.submit_to_docling(
+ filename, content, owner=owner, jwt_token=jwt_token
+ )
+ except Exception:
+ if file_task is not None:
+ file_task.docling_status = DoclingPhaseStatus.FAILED
+ raise🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/services/langflow_file_service.py` around lines 596 - 605, Wrap the call
to submit_to_docling(...) in a try/except so that if it raises you update the
in-memory/tracked file_task (when file_task is not None) to a terminal failure
state: set file_task.phase = IngestionPhase.DOCLING (if not already),
file_task.docling_status = DoclingPhaseStatus.FAILED and optionally clear/leave
docling_task_id unset; persist/update the file_task record (use the existing
save/update mechanism used elsewhere) before re-raising or returning the
exception so the task does not remain DOCLING/PENDING. Ensure this handling
surrounds the submit_to_docling call and still sets PROCESSING/docling_task_id
on success as currently implemented.
Comment on lines
+129
to
+132
| # Mirror clean_connector_filename's space/slash -> underscore so lookups also | ||
| # match files indexed through a connector ingestion path. | ||
| aliases.extend(name.replace(" ", "_").replace("/", "_") for name in list(aliases)) | ||
|
|
Contributor
There was a problem hiding this comment.
Avoid generating ambiguous filename aliases that can collide across distinct documents.
At Line 131, converting spaces to underscores can make different filenames equivalent (e.g., Q1 Report.pdf and Q1_Report.pdf), which is risky for duplicate detection/deletion flows that rely on aliases.
💡 Suggested fix
- # Mirror clean_connector_filename's space/slash -> underscore so lookups also
- # match files indexed through a connector ingestion path.
- aliases.extend(name.replace(" ", "_").replace("/", "_") for name in list(aliases))
+ # Keep legacy path-separator compatibility without conflating
+ # natural-space filenames with underscore filenames.
+ aliases.extend(name.replace("/", "_") for name in list(aliases) if "/" in name)🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/utils/file_utils.py` around lines 129 - 132, The current blanket
conversion of spaces/slashes to underscores (aliases.extend(...)) can create
ambiguous collisions (e.g., "Q1 Report.pdf" vs "Q1_Report.pdf"); change the
logic to deduplicate and only add an underscore/slash-normalized alias when it
would not collide with an existing name: build a set of existing alias/original
names (from aliases), for each name produce the candidate name.replace(" ",
"_").replace("/", "_") and only add that candidate if candidate != name and
candidate not in the existing set; update aliases via the set/extend accordingly
so clean_connector_filename behavior is mirrored but collisions are avoided.
Comment on lines
+108
to
+112
| logger.info( | ||
| "Skipping OpenSearch security setup during init_index " | ||
| "(OPENRAG_SKIP_OS_SECURITY_SETUP=true)", | ||
| admin_username=admin_username, | ||
| ) |
Contributor
There was a problem hiding this comment.
Avoid logging admin_username in startup logs.
This can leak credential-related identifiers into shared logs; the message is sufficient without the field.
🔧 Proposed fix
if OPENRAG_SKIP_OS_SECURITY_SETUP:
logger.info(
"Skipping OpenSearch security setup during init_index "
- "(OPENRAG_SKIP_OS_SECURITY_SETUP=true)",
- admin_username=admin_username,
+ "(OPENRAG_SKIP_OS_SECURITY_SETUP=true)",
)📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| logger.info( | |
| "Skipping OpenSearch security setup during init_index " | |
| "(OPENRAG_SKIP_OS_SECURITY_SETUP=true)", | |
| admin_username=admin_username, | |
| ) | |
| logger.info( | |
| "Skipping OpenSearch security setup during init_index " | |
| "(OPENRAG_SKIP_OS_SECURITY_SETUP=true)", | |
| ) |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/utils/opensearch_init.py` around lines 108 - 112, The startup log in
opensearch_init.py currently includes the credential identifier admin_username
as an extra field on the logger.info call; remove the admin_username argument
from the logger.info invocation (and any similar logging around init_index or in
the function/method that calls logger.info) so the message remains but no
credential-related identifiers are emitted, and run a quick grep for
admin_username usage in this module to ensure no other startup logs include
sensitive identifiers.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
All jobs
curl -H "Authorization: Bearer " http://localhost:8000/v1/ingestion/jobs
Only failed Docling jobs
curl -H "Authorization: Bearer " "http://localhost:8000/v1/ingestion/jobs?status=failed&component=docling"
Full detail for the password-protected scenario
curl -H "Authorization: Bearer " http://localhost:8000/v1/ingestion/jobs/job-005
404 case
curl -H "Authorization: Bearer " http://localhost:8000/v1/ingestion/jobs/job-999
Summary by CodeRabbit
New Features
Bug Fixes / UX