fix: Add OpenSearch basic-auth for on-prem deployments, improve onboarding flow, and support asymmetric JWT signing

[edwinjosechittilappilly]

Summary by CodeRabbit

• Improvements
  • Enhanced OpenSearch authentication with multiple credential paths (platform service token, basic auth, user-JWT fallback), runtime credential resolution, stricter startup validation, and backward-compatibility warnings.
• New Features
  • Configurable signing and verification for ingest-callback JWTs supporting symmetric and various asymmetric key types/algorithms.
• Tests
  • Added a unit test to ensure the configured JWT signing key is preferred over the session secret.

[coderabbitai]

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

• ▶️ Resume reviews
• 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: af7c172a-e020-49eb-b4e5-5f6de92160a0

📥 Commits

Reviewing files that changed from the base of the PR and between e4c2911 and 7118af0.

📒 Files selected for processing (1)

• src/app/lifespan.py

💤 Files with no reviewable changes (1)

• src/app/lifespan.py

---

Walkthrough

Adds runtime OpenSearch credential getters and run-mode selection to choose basic-auth, service-JWT, or unauthenticated clients; applies this in AppClients.initialize(), startup bootstrap, and onboarding. Separately, makes Langflow ingest token signing/verification configurable (PEM private-key support) and adds a unit test.

Changes

OpenSearch Dual Authentication Path

Layer / File(s)	Summary
OpenSearch auth credential accessors and basic-auth selection src/config/settings.py	Runtime getters for OPENSEARCH_USERNAME/OPENSEARCH_PASSWORD and AppClients.initialize() updated to choose basic-auth, JWT-based, or unauthenticated OpenSearch client based on run mode and service token presence.
Startup OpenSearch security bootstrap with conditional auth src/app/lifespan.py	Startup bootstrap now chooses basic-auth (on_prem/oss) or service-JWT (saas), validating service token presence and extracting admin_username for JWT path.
Onboarding index initialization with auth branching src/api/settings/endpoints.py	Onboarding OpenSearch index setup now prefers OPENRAG_SERVICE_TOKEN-derived admin username and JWT client in saas, falls back to onboarding user or unauthenticated client, and uses basic-auth in on_prem/oss.

Langflow Ingest Token Signing/Verification

Layer / File(s)	Summary
Default signing config resolver src/services/langflow_ingest_token_service.py	Add _resolve_default_signing_config() to derive signing and verification keys plus JWT alg from JWT_SIGNING_KEY, falling back to SESSION_SECRET as HS256.
LangflowIngestTokenService wiring src/services/langflow_ingest_token_service.py	Per-instance _signing_key, _verification_key, and algorithm fields; create_token signs with _signing_key, validate_token and revoke_token verify with _verification_key.
Unit test for signing key preference tests/unit/test_langflow_ingest_callback.py	New test ensures tokens signed with JWT_SIGNING_KEY validate and that the same token is rejected when only SESSION_SECRET is used for verification.

Sequence Diagram(s)

sequenceDiagram
  participant Client
  participant OnboardingEndpoint
  participant AppClients
  participant Lifespan
  participant OpenSearch
  participant ServiceToken
  Client->>OnboardingEndpoint: start onboarding
  OnboardingEndpoint->>ServiceToken: check OPENRAG_SERVICE_TOKEN (saas)
  OnboardingEndpoint->>AppClients: request opensearch client for onboarding
  Lifespan->>ServiceToken: get_openrag_service_token() (bootstrap)
  AppClients->>OpenSearch: initialize writer client (basic-auth | JWT | unauthenticated)
  OnboardingEndpoint->>OpenSearch: create index / set admin_username

Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

• langflow-ai/openrag#1626: Both PRs modify src/app/lifespan.py to add/extend a flag-gated OpenSearch security bootstrap that derives admin_username from a platform/service JWT and uses a JWT-auth OpenSearch client.
• langflow-ai/openrag#1664: Touches the user.jwt_token-driven fallback for constructing OpenSearch credentials; changes there affect onboarding/auth initialization fallbacks used by this PR.

Suggested reviewers

• phact
• zzzming

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 38.46% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and specifically summarizes the three main changes: OpenSearch basic-auth for on-prem, onboarding flow improvements, and asymmetric JWT signing support.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix-onboarding-setup

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[Copilot]

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

[edwinjosechittilappilly]

@coderabbitai suggest a title and summary for the PR update the title and summary if possible

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/services/langflow_ingest_token_service.py`:
- Around line 71-73: mypy can't infer the third tuple element's type when
unpacking into self.algorithm; call _resolve_default_signing_config() into a
temporary variable and then assign its elements in separate statements so types
are clear — e.g. tmp = _resolve_default_signing_config(); self._signing_key,
self._verification_key = tmp[0], tmp[1]; self.algorithm = typing.cast(str,
tmp[2]) (or assign tmp[2] on its own if return type is already correct); update
references to self._signing_key, self._verification_key, and self.algorithm in
the same scope.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 0332f3a5-c2ac-4e86-afe9-86df89d2fb27

📥 Commits

Reviewing files that changed from the base of the PR and between 8199f76 and bd9c610.

📒 Files selected for processing (2)

• src/services/langflow_ingest_token_service.py
• tests/unit/test_langflow_ingest_callback.py

[coderabbitai]

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Fetch current PR description for reference
gh pr view 1713 --repo langflow-ai/openrag --json title,body

Length of output: 253

---

🏁 Script executed:

#!/bin/bash
gh pr edit 1713 \
  --repo langflow-ai/openrag \
  --title "fix: Add OpenSearch basic-auth for on-prem deployments, improve onboarding flow, and support asymmetric JWT signing" \
  --body "## Summary

### OpenSearch Basic-Auth Support (On-Prem / IBM-Auth)
- Introduced \`get_opensearch_username()\`, \`get_opensearch_password()\`, and \`use_opensearch_basic_auth()\` in \`src/config/settings.py\` to read credentials from the environment at call time (allows overrides without restart).
- \`use_opensearch_basic_auth()\` enables basic-auth mode when IBM auth is active, the deployment is on-prem, and OpenSearch credentials are present.
- \`AppClients.initialize()\` now selects basic-auth when IBM auth is disabled **or** when \`use_opensearch_basic_auth()\` returns true; otherwise passes \`http_auth=None\` to rely on platform JWT.

### Onboarding & Startup Improvements
- Updated \`src/api/settings/endpoints.py\` onboarding to conditionally create an OpenSearch client and determine the admin username:
  - Uses basic-auth credentials when \`use_opensearch_basic_auth()\` is enabled.
  - Falls back to the platform service token (\`OPENRAG_SERVICE_TOKEN\`) for JWT-based auth when IBM auth is enabled.
  - If neither is available, falls back to the onboarding user's JWT with a backward-compatibility warning.
- Updated \`src/app/lifespan.py\` startup bootstrap (\`OPENRAG_BOOTSTRAP_OS_SECURITY_ON_STARTUP\`): uses on-prem basic-auth credentials when available; otherwise validates and uses the platform service JWT, raising \`RuntimeError\` if either is missing.

### Asymmetric JWT Signing for Langflow Ingest Tokens
- \`src/services/langflow_ingest_token_service.py\` now supports configurable signing/verification key material:
  - \`JWT_SIGNING_KEY\` is preferred over \`SESSION_SECRET\`; PEM private keys are auto-detected with algorithm selection (RSA → RS256, EC → ES256/ES384/ES512, EdDSA).
  - Separate \`_signing_key\` and \`_verification_key\` are stored for \`jwt.encode\` / \`jwt.decode\` respectively.
- Added unit test \`test_ingest_token_prefers_jwt_signing_key_over_session_secret\` confirming that a token signed with \`JWT_SIGNING_KEY\` cannot be validated using only \`SESSION_SECRET\`.

## Testing
- Unit tests added in \`tests/unit/test_langflow_ingest_callback.py\`.
- Manual testing recommended for on-prem OpenSearch basic-auth onboarding and startup bootstrap flows."

Length of output: 2952

---

---

@edwinjosechittilappilly, unfortunately the gh pr edit command is blocked by the workflow restriction settings in this environment, so I'm unable to update the PR directly. You can apply the following title and summary manually via the GitHub UI (click the ✏️ edit pencil next to the PR title and the description box):

---

Suggested Title:

fix: Add OpenSearch basic-auth for on-prem deployments, improve onboarding flow, and support asymmetric JWT signing

---

Suggested Description:

## Summary

### OpenSearch Basic-Auth Support (On-Prem / IBM-Auth)
- Introduced `get_opensearch_username()`, `get_opensearch_password()`, and `use_opensearch_basic_auth()` in `src/config/settings.py` to read credentials from the environment at call time (allows overrides without restart).
- `use_opensearch_basic_auth()` enables basic-auth mode when IBM auth is active, the deployment is on-prem, and OpenSearch credentials are present.
- `AppClients.initialize()` now selects basic-auth when IBM auth is disabled **or** when `use_opensearch_basic_auth()` returns true; otherwise passes `http_auth=None` to rely on platform JWT.

### Onboarding & Startup Improvements
- Updated `src/api/settings/endpoints.py` onboarding to conditionally create an OpenSearch client and determine the admin username:
  - Uses basic-auth credentials when `use_opensearch_basic_auth()` is enabled.
  - Falls back to the platform service token (`OPENRAG_SERVICE_TOKEN`) for JWT-based auth when IBM auth is enabled.
  - If neither is available, falls back to the onboarding user's JWT with a backward-compatibility warning.
- Updated `src/app/lifespan.py` startup bootstrap (`OPENRAG_BOOTSTRAP_OS_SECURITY_ON_STARTUP`): uses on-prem basic-auth credentials when available; otherwise validates and uses the platform service JWT, raising `RuntimeError` if either is missing.

### Asymmetric JWT Signing for Langflow Ingest Tokens
- `src/services/langflow_ingest_token_service.py` now supports configurable signing/verification key material:
  - `JWT_SIGNING_KEY` is preferred over `SESSION_SECRET`; PEM private keys are auto-detected with algorithm selection (RSA → RS256, EC → ES256/ES384/ES512, EdDSA).
  - Separate `_signing_key` and `_verification_key` are stored for `jwt.encode` / `jwt.decode` respectively.
- Added unit test `test_ingest_token_prefers_jwt_signing_key_over_session_secret` confirming that a token signed with `JWT_SIGNING_KEY` cannot be validated using only `SESSION_SECRET`.

## Testing
- Unit tests added in `tests/unit/test_langflow_ingest_callback.py`.
- Manual testing recommended for on-prem OpenSearch basic-auth onboarding and startup bootstrap flows.

[coderabbitai]

Actionable comments posted: 1

🧹 Nitpick comments (2)

src/app/lifespan.py (1)

224-233: 💤 Low value

Consider validating OpenSearch password before bootstrap.

Same issue as in settings.py: get_opensearch_password() may return None for on_prem/oss modes. A missing password would cause an unclear failure during setup_opensearch_security().

🛡️ Suggested validation

         if is_run_mode_on_prem() or is_run_mode_oss():
             admin_username = get_opensearch_username()
+            os_password = get_opensearch_password()
+            if not os_password:
+                raise RuntimeError(
+                    "OPENSEARCH_PASSWORD must be set when "
+                    "OPENRAG_BOOTSTRAP_OS_SECURITY_ON_STARTUP is enabled for on_prem/oss"
+                )
             opensearch_client = clients.create_basic_opensearch_client(
-                admin_username, get_opensearch_password()
+                admin_username, os_password
             )

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/app/lifespan.py` around lines 224 - 233, The code calls
get_opensearch_password() without validating it in the on-prem/oss branch, which
can return None and cause unclear failures when create_basic_opensearch_client
or setup_opensearch_security run; update the block that uses
is_run_mode_on_prem()/is_run_mode_oss() to retrieve the password into a local
variable, check that password is not None/empty (and log+raise a clear error or
exit if it is), then pass the validated password to
clients.create_basic_opensearch_client and subsequent setup_opensearch_security
so failures are explicit; reference symbols: is_run_mode_on_prem,
is_run_mode_oss, get_opensearch_password, create_basic_opensearch_client,
setup_opensearch_security, admin_username.

src/config/settings.py (1)

618-628: 💤 Low value

Consider validating OpenSearch password before using in basic auth.

In on_prem/oss modes, get_opensearch_password() can return None. Passing (username, None) as http_auth may cause unclear auth failures later. The existing code in dls_principal_service.py (lines 251-252) checks if not OPENSEARCH_PASSWORD: return None before creating the basic client.

A defensive check here would provide a clearer fail-fast behavior:

🛡️ Suggested validation

             if is_run_mode_on_prem() or is_run_mode_oss():
+                os_password = get_opensearch_password()
+                if not os_password:
+                    raise RuntimeError(
+                        "OPENSEARCH_PASSWORD must be set for on_prem/oss run modes"
+                    )
-                os_auth = (get_opensearch_username(), get_opensearch_password())
+                os_auth = (get_opensearch_username(), os_password)
                 logger.info(

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/config/settings.py` around lines 618 - 628, When initializing os_auth in
the is_run_mode_on_prem()/is_run_mode_oss() branch, validate
get_opensearch_password() before using it in the tuple; if the password is
missing (None or empty) log a clear error and fail-fast (return None or raise a
RuntimeError) instead of setting os_auth=(username, None) so we don't create a
basic-auth client with a missing password—mirror the defensive behavior used in
dls_principal_service where it returns None when OPENSEARCH_PASSWORD is not set.
Use the existing symbols is_run_mode_on_prem, is_run_mode_oss,
get_opensearch_username, get_opensearch_password and os_auth to locate and
change the logic.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/app/lifespan.py`:
- Around line 214-218: The import list in lifespan.py includes an unused symbol
is_run_mode_saas which triggers ruff F401; remove is_run_mode_saas from the from
utils.run_mode_utils import (...) statement so only is_run_mode_on_prem and
is_run_mode_oss remain, and update the import formatting/commas accordingly to
keep the file syntactically correct (refer to the import line that currently
lists is_run_mode_on_prem, is_run_mode_oss, is_run_mode_saas).

---

Nitpick comments:
In `@src/app/lifespan.py`:
- Around line 224-233: The code calls get_opensearch_password() without
validating it in the on-prem/oss branch, which can return None and cause unclear
failures when create_basic_opensearch_client or setup_opensearch_security run;
update the block that uses is_run_mode_on_prem()/is_run_mode_oss() to retrieve
the password into a local variable, check that password is not None/empty (and
log+raise a clear error or exit if it is), then pass the validated password to
clients.create_basic_opensearch_client and subsequent setup_opensearch_security
so failures are explicit; reference symbols: is_run_mode_on_prem,
is_run_mode_oss, get_opensearch_password, create_basic_opensearch_client,
setup_opensearch_security, admin_username.

In `@src/config/settings.py`:
- Around line 618-628: When initializing os_auth in the
is_run_mode_on_prem()/is_run_mode_oss() branch, validate
get_opensearch_password() before using it in the tuple; if the password is
missing (None or empty) log a clear error and fail-fast (return None or raise a
RuntimeError) instead of setting os_auth=(username, None) so we don't create a
basic-auth client with a missing password—mirror the defensive behavior used in
dls_principal_service where it returns None when OPENSEARCH_PASSWORD is not set.
Use the existing symbols is_run_mode_on_prem, is_run_mode_oss,
get_opensearch_username, get_opensearch_password and os_auth to locate and
change the logic.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 9b0b2cfb-969d-4252-b1f0-7db7f04feda2

📥 Commits

Reviewing files that changed from the base of the PR and between ee3ef3b and e4c2911.

📒 Files selected for processing (3)

• src/api/settings/endpoints.py
• src/app/lifespan.py
• src/config/settings.py

[edwinjosechittilappilly]

@coderabbitai suggest a title and summary for the PR

[coderabbitai]

Tip

For best results, initiate chat on the files or code changes.

@edwinjosechittilappilly, here are the suggested title and description for this PR:

---

Suggested Title:

fix: Add OpenSearch basic-auth for on-prem deployments, improve onboarding flow, and support asymmetric JWT signing

---

Suggested Description:

## Summary

### OpenSearch Basic-Auth Support (On-Prem / OSS)
- Added `get_opensearch_username()`, `get_opensearch_password()` in `src/config/settings.py` to read OpenSearch credentials from the environment at call time (supports overrides without restart).
- `AppClients.initialize()` now selects the OpenSearch writer client by run mode: basic-auth in on-prem/OSS, JWT-based auth in SaaS (when a platform service token is present), or unauthenticated otherwise.

### Onboarding & Startup Improvements
- `src/api/settings/endpoints.py` onboarding now selects the OpenSearch client and `admin_username` by run mode:
  - **SaaS**: uses `OPENRAG_SERVICE_TOKEN` JWT; falls back to the onboarding user's JWT with a backward-compatibility warning if the service token is absent.
  - **On-prem / OSS**: uses OpenSearch basic-auth credentials.
- `src/app/lifespan.py` startup bootstrap (`OPENRAG_BOOTSTRAP_OS_SECURITY_ON_STARTUP`): on-prem/OSS uses OpenSearch basic-auth; SaaS validates and uses the platform service JWT, raising `RuntimeError` if either is missing.

### Asymmetric JWT Signing for Langflow Ingest Tokens
- `src/services/langflow_ingest_token_service.py` now supports configurable signing/verification key material:
  - `JWT_SIGNING_KEY` is preferred over `SESSION_SECRET`; PEM private keys are auto-detected with algorithm selection (RSA → RS256, EC → ES256/ES384/ES512, EdDSA → EdDSA).
  - Separate `_signing_key` and `_verification_key` are stored per instance for `jwt.encode` / `jwt.decode`.

## Testing
- Unit test added: `test_ingest_token_prefers_jwt_signing_key_over_session_secret` in `tests/unit/test_langflow_ingest_callback.py`.
- Manual testing recommended for on-prem OpenSearch basic-auth onboarding and startup bootstrap flows.

---

You can apply these via the GitHub UI by clicking the ✏️ pencil icon next to the PR title and editing the description box directly. Unfortunately I'm unable to update the PR programmatically due to workflow restrictions in this environment.