Skip to content

Commit b0ead4a

Browse files
edwinjosechittilappillyautofix-ci[bot]mfortman11lucaseduolignPy
authored
feat: docling ingest vlms (#1841)
* docling ingest vlms * docling tab * resue proivders dropdown * style: ruff autofix (auto) * style: ruff autofix (auto) * fix: added docling settings under knowledge settings and added local model option (#2031) * fix: remediate image_scan CVEs in backend, frontend, and langflow images (#2000) * fix light color text catergory chips in light mode (#2013) Co-authored-by: Olfa Maslah <olfamaslah@Olfas-MacBook-Pro.local> * fix: upgraded version of docling (#1995) * changed version of docling * fix logs not being the correct ones * go back to sequential building to save space * add cache to builds * remove single use results from docling manager * revert single use results * change the ray tenant id header * added env * changed cache to hit if no changes were made * fix hash for cache * style: ruff autofix (auto) * fixed coderabbit * style: ruff autofix (auto) * added comment on docling manager * style: ruff autofix (auto) --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> * feat: add chunk level page source (#2015) * Changed export docling document to append page number * update result on frontend to append page number * document how to get page number in readmes * Update flows/components/export_docling_document.py Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * Update flows/components/export_docling_document.py Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * fix: apply CodeRabbit auto-fixes Fixed 1 file(s) based on 1 unresolved review comment. Co-authored-by: CodeRabbit <noreply@coderabbit.ai> * update component code --------- Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> Co-authored-by: CodeRabbit <noreply@coderabbit.ai> * fix: error in skip duplicates functionality for folder ingest for connectors (backport of #1941) (#2018) * fix: error in skip duplicates functionality for folder ingest for connectors 75616(#1842) (#1941) * fix: enhance duplicate handling and indexing for connector file uploads * style: ruff autofix (auto) * fix: improve duplicate handling in connector sync functionality * chore: trigger CodeRabbit --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> (cherry picked from commit effaf02) * style: ruff autofix (auto) * fix: remove duplicate test definitions from main merge The automated merge-from-main on this branch concatenated two tests (test_connector_processor_indexes_cleaned_filename, test_langflow_connector_processor_uses_cleaned_filename) that already existed under both names, tripping ruff F811. Keep one copy of each, preferring the version wired with connector_id_exists=True to match current check_document_exists behavior. --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> * fix: Ingestion of buckets not working right after connection is tested - CPD #75506 (#1942) (#2019) * fix: update bucket ingestion messages for clarity and consistency * fix: guard bucket pre-selection against connection ID mismatch The defaults query returns the first S3/COS connection regardless of active status. Only pre-select saved bucket_names when the defaults connection_id matches the current connector.connectionId to avoid seeding the wrong buckets if a stale connection exists alongside the active one. * fix: prevent defaults from overwriting user bucket selections Two issues in the initial-selection effect: 1. hasAppliedInitial was only set when valid buckets were found, leaving the effect live across bucket refreshes and allowing stale defaults to overwrite selections made after a refetch. 2. No guard against the async race where buckets loads before initialSelectedBuckets: user clicks buckets, defaults arrive later and silently overwrite them. Fix: mark hasAppliedInitial unconditionally on first evaluation, and only apply defaults when selectedBuckets is still empty (user hasn't acted yet). --------- (cherry picked from commit f3823b8) Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: handle orphan files and return appropriate responses when syncing connectors (#1785) (#2016) * feat: handle orphan files and return appropriate responses when syncing connectors * feat: refactor GoogleDriveOAuth to separate required scopes and add handling for missing optional group scopes in tests * style: ruff autofix (auto) * feat: add handling for skipped files and warnings in task processing and UI components --------- (cherry picked from commit aca865a) Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> * fix: enhance error handling for document processing and add OCR requirement check for image files (#1859) (#2017) * fix: enhance error handling for document processing and add OCR requirement check for image files * style: ruff autofix (auto) * remove clickhouse connect from langflow image (#1801) * chore: fix ci (#1889) * Update processors.py * test fix * fix ci --------- (cherry picked from commit a286581) Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> Co-authored-by: Lucas Oliveira <62335616+lucaseduoli@users.noreply.github.com> Co-authored-by: Edwin Jose <edwin.jose@datastax.com> * fix: bump golang.org/x/net to v0.55.0 to remediate CVE-2026-25680 and CVE-2026-39821 (#2014) * feat: add new design for messages and add inline references (#2020) * update opensearch multimodal and flows to support parser, chunk size and chunk overlap * update backend to support citations and chunk metadata * update frontend to support citations and metadata, implement new design for assistant message * changed the prompt for the agent * changed the prompt for the agent * remove hallucinated chunk references * adjust design of citations and where popover appears * adjusted design for message, popover and cards * fix React Doctor picks * fix: apply CodeRabbit auto-fixes Fixed 5 file(s) based on 4 unresolved review comments. Co-authored-by: CodeRabbit <noreply@coderabbit.ai> * style: apply biome auto-fixes [skip ci] * updated colors in ibm mode * fixed react doctor picks * added correct type and validation for function calls * remove style for message when on onboarding * fix message not being w full * remove search query in end of the message * change popup to use shadcn popover * fix coderabbit's suggestion * increase prompt limit --------- Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> Co-authored-by: CodeRabbit <noreply@coderabbit.ai> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * apply new settings on retry (#2010) * fix: preserve HTTPException status in update_docling_preset (#1586) (#2004) * fix: report actual synced connection count in connector_sync (#1547) (#2006) * fix: restore LLM model values in reapply_all_settings fallback (#1587) (#2008) * fix: prevent partial matches for exact token searches (#2003) * fix: prevent partial matches for exact token searches (#1937) * style: ruff autofix (auto) * chore: remove unused json import --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> * fix: target agent component when updating chat flow system prompt (#2001) * fix: target langflow agent component for system prompt update instead of provider LLM component Signed-off-by: vchen7629 <chenvincent7629@gmail.com> * fix: removed unused llm_provider arg from _update_langflow_system_prompt Signed-off-by: vchen7629 <chenvincent7629@gmail.com> * test: add unit tests for update_chat_flow_system_prompt Signed-off-by: vchen7629 <chenvincent7629@gmail.com> * style: ruff autofix (auto) * fix(test): resolve flow path relative to repo root in test_langflow_agent_system_prompt Signed-off-by: vchen7629 <chenvincent7629@gmail.com> * style: ruff autofix (auto) --------- Signed-off-by: vchen7629 <chenvincent7629@gmail.com> Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> * fix: replaced hardcoded docker.io prefix with a env variable prefix for private repo compatibility (#2002) Signed-off-by: vchen7629 <chenvincent7629@gmail.com> * fix: resolve Pydantic delta serialization warnings in agent streaming (#2005) * fix: updated agent model dump to exclude delta field Signed-off-by: vchen7629 <chenvincent7629@gmail.com> * test: added regression unit test to verify response field + no warnings * style: ruff autofix (auto) * test: added additional unit test to verify that model_dump without exclusion raises the warning Signed-off-by: vchen7629 <chenvincent7629@gmail.com> * style: ruff autofix (auto) * test: tighten delta warning assertion to match specific Pydantic error Signed-off-by: vchen7629 <chenvincent7629@gmail.com> * style: ruff autofix (auto) --------- Signed-off-by: vchen7629 <chenvincent7629@gmail.com> Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> * fix: prevent exception details from leaking in API error responses (#2007) * fix: removed str(e) in exceptions to prevent internals from leaking Signed-off-by: vchen7629 <chenvincent7629@gmail.com> * style: ruff autofix (auto) * fix: added missing status_code Signed-off-by: vchen7629 <chenvincent7629@gmail.com> * style: ruff autofix (auto) --------- Signed-off-by: vchen7629 <chenvincent7629@gmail.com> Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> * Changed docling options to include local support and be shown in the Langflow tab * style: ruff autofix (auto) * enable ollama and delete unused file * fix next lint --------- Signed-off-by: vchen7629 <chenvincent7629@gmail.com> Co-authored-by: Gautham N Pai <119022878+gnPy@users.noreply.github.com> Co-authored-by: Wallgau <46035189+Wallgau@users.noreply.github.com> Co-authored-by: Olfa Maslah <olfamaslah@Olfas-MacBook-Pro.local> Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> Co-authored-by: CodeRabbit <noreply@coderabbit.ai> Co-authored-by: Rico Furtado <hfurtado@gmail.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: Edwin Jose <edwin.jose@datastax.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Mike Fortman <mfortman11@gmail.com> Co-authored-by: hunterxtang <38674961+hunterxtang@users.noreply.github.com> Co-authored-by: NishadA05 <168224983+NishadA05@users.noreply.github.com> Co-authored-by: Zephyrus <114184635+Vchen7629@users.noreply.github.com> * Close accordian when pic desc is off * coderabbit comments * react-doctor * react-dr state fix * coderabbit fixes * Add vlm feature flag * test fix --------- Signed-off-by: vchen7629 <chenvincent7629@gmail.com> Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> Co-authored-by: Mike Fortman <mfortman11@gmail.com> Co-authored-by: Lucas Oliveira <62335616+lucaseduoli@users.noreply.github.com> Co-authored-by: Gautham N Pai <119022878+gnPy@users.noreply.github.com> Co-authored-by: Wallgau <46035189+Wallgau@users.noreply.github.com> Co-authored-by: Olfa Maslah <olfamaslah@Olfas-MacBook-Pro.local> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> Co-authored-by: CodeRabbit <noreply@coderabbit.ai> Co-authored-by: Rico Furtado <hfurtado@gmail.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: hunterxtang <38674961+hunterxtang@users.noreply.github.com> Co-authored-by: NishadA05 <168224983+NishadA05@users.noreply.github.com> Co-authored-by: Zephyrus <114184635+Vchen7629@users.noreply.github.com>
1 parent ec857c2 commit b0ead4a

25 files changed

Lines changed: 1386 additions & 237 deletions

flows/components/export_docling_document.py

Lines changed: 12 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -157,11 +157,18 @@ def export_document(self) -> list[Data]:
157157
continue
158158

159159
# Fall back to whole-document export (no page granularity)
160-
content = doc.export_to_markdown(
161-
image_mode=image_mode,
162-
image_placeholder=self.md_image_placeholder,
163-
page_break_placeholder=self.md_page_break_placeholder,
164-
)
160+
try:
161+
content = doc.export_to_markdown(
162+
image_mode=image_mode,
163+
image_placeholder=self.md_image_placeholder,
164+
page_break_placeholder=self.md_page_break_placeholder,
165+
)
166+
except TypeError:
167+
# Older docling-core versions lack page_break_placeholder
168+
content = doc.export_to_markdown(
169+
image_mode=image_mode,
170+
image_placeholder=self.md_image_placeholder,
171+
)
165172
elif self.export_format == "HTML":
166173
content = doc.export_to_html(image_mode=image_mode)
167174
elif self.export_format == "Plaintext":

frontend/app/api/mutations/useUpdateSettingsMutation.ts

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,17 @@ export interface UpdateSettingsRequest {
2222
embedding_model?: string;
2323
embedding_provider?: string;
2424

25+
// Docling VLM pipeline settings
26+
vlm_enabled?: boolean;
27+
vlm_provider?: string;
28+
vlm_model?: string;
29+
vlm_prompt?: string;
30+
vlm_response_format?: string;
31+
vlm_max_tokens?: number;
32+
vlm_concurrency?: number;
33+
vlm_timeout?: number;
34+
vlm_watsonx_api_version?: string;
35+
2536
// Provider-specific settings (for dialogs)
2637
model_provider?: string; // Deprecated, kept for backward compatibility
2738
api_key?: string;

frontend/app/api/queries/useGetModelsQuery.ts

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@ export interface ModelOption {
99
value: string;
1010
label: string;
1111
default?: boolean;
12+
supports_images?: boolean;
1213
}
1314

1415
export interface ModelsResponse {

frontend/app/api/queries/useGetSearchQuery.ts

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,8 @@ export interface ChunkResult {
3636
parser?: string;
3737
chunk_size?: number;
3838
chunk_overlap?: number;
39+
chunk_id?: string;
40+
id?: string;
3941
index?: number;
4042
allowed_users?: string[];
4143
allowed_groups?: string[];

frontend/app/api/queries/useGetSettingsQuery.ts

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,15 @@ export interface KnowledgeSettings {
2020
ocr?: boolean;
2121
picture_descriptions?: boolean;
2222
disable_ingest_with_langflow?: boolean;
23+
vlm_enabled?: boolean;
24+
vlm_provider?: string;
25+
vlm_model?: string;
26+
vlm_prompt?: string;
27+
vlm_response_format?: string;
28+
vlm_max_tokens?: number;
29+
vlm_concurrency?: number;
30+
vlm_timeout?: number;
31+
vlm_watsonx_api_version?: string;
2332
}
2433

2534
export interface ProviderSettings {
@@ -41,6 +50,9 @@ export interface ProviderSettings {
4150
endpoint?: string;
4251
configured?: boolean;
4352
};
53+
local?: {
54+
configured?: boolean;
55+
};
4456
}
4557

4658
export interface OnboardingState {
@@ -79,6 +91,8 @@ export interface Settings {
7991
localhost_url?: string;
8092
ingest_via_chat?: boolean;
8193
show_provider_ingest_settings?: boolean;
94+
show_vlm_settings?: boolean;
95+
local_vlm_models?: string[];
8296
show_shared_upload_toggle?: boolean;
8397
show_workspace_oauth_overrides?: boolean;
8498
segment_write_key?: string;

0 commit comments

Comments
 (0)