Skip to content

fix: cache images with architecture and check architecture in startup scripts#2087

Merged
lucaseduoli merged 11 commits into
mainfrom
fix/build_arch
Jul 14, 2026
Merged

fix: cache images with architecture and check architecture in startup scripts#2087
lucaseduoli merged 11 commits into
mainfrom
fix/build_arch

Conversation

@lucaseduoli

@lucaseduoli lucaseduoli commented Jul 14, 2026

Copy link
Copy Markdown
Collaborator

This pull request improves the handling of container image architecture compatibility in CI and E2E scripts, ensuring that cached Docker images match the host architecture. It adds architecture checks and warnings to help avoid performance issues due to emulation, and enhances debugging information for backend startup failures.

CI/CD workflow improvements:

  • Updated cache keys in .github/workflows/test-ci.yml to include ${{ runner.arch }}, ensuring that cached Docker images are architecture-specific and preventing cross-architecture cache issues.
  • Enhanced the cache loading logic in .github/workflows/test-ci.yml to inspect the architecture of cached images, only using them if they match the host, and discarding incompatible caches.

Architecture compatibility checks:

  • Added host and image architecture normalization and comparison in scripts/ci/run_integration_suite.sh and scripts/setup-e2e.sh, with warnings if a mismatch is detected (indicating the backend will run under QEMU emulation, which may cause slowness and timeouts). [1] [2]

Debugging enhancements:

  • Improved error reporting in scripts/ci/run_integration_suite.sh by printing the last 50 lines of backend logs if the backend fails to start, aiding in diagnosing startup issues.

Summary by CodeRabbit

  • Bug Fixes

    • Improved Docker image caching to reuse cached images only when host and image CPU architectures match.
    • Integration and E2E setup now warn on CPU architecture mismatches (possible QEMU emulation slowness/timeouts) and, on OIDC readiness failure, print the last backend container logs.
  • CI Improvements

    • CI now uses a workspace-local Hugging Face cache.
    • CI maximizes build space by restarting Docker and remapping Docker storage for key build/test jobs.
    • Updated image tarball caching to be architecture-aware across OpenSearch, backend, frontend, and Langflow.

@lucaseduoli
lucaseduoli requested a review from Wallgau July 14, 2026 14:01
@lucaseduoli lucaseduoli self-assigned this Jul 14, 2026
@github-actions github-actions Bot added ci ⬛ CI/CD, build, and infrastructure issues bug 🔴 Something isn't working. labels Jul 14, 2026
@coderabbitai

coderabbitai Bot commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

Walkthrough

CI jobs now provision workspace-backed Docker storage, use architecture-aware image caches, and validate cached image compatibility. Integration and E2E scripts compare host and backend image architectures, while integration failures include recent backend logs when OIDC startup times out.

Changes

Architecture-aware image handling

Layer / File(s) Summary
Architecture-aware CI image caching
.github/workflows/test-ci.yml
Build, test, and E2E jobs maximize available space; HuggingFace and Docker storage use workspace paths; cache keys include runner OS and architecture, and cached images are reused only when their architecture matches the host.
Integration runtime diagnostics
scripts/ci/run_integration_suite.sh
The integration suite warns about backend architecture mismatches and prints the last 50 backend log lines when the OIDC endpoint times out.
E2E architecture warning
scripts/setup-e2e.sh
E2E setup compares host and backend image architectures and warns when QEMU emulation may be used.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related PRs

  • langflow-ai/openrag#1962: Updates the base CI workflow that this change further modifies for architecture-aware caching and maximized Docker space.

Suggested reviewers: edwinjosechittilappilly, wallgau

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: architecture-aware image caching and startup script checks.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/build_arch

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added bug 🔴 Something isn't working. and removed bug 🔴 Something isn't working. labels Jul 14, 2026
@github-actions github-actions Bot added the lgtm label Jul 14, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
scripts/setup-e2e.sh (1)

72-81: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Extract duplicated host architecture normalization logic.

The logic to normalize the host_arch strings is duplicated across multiple scripts. Consider abstracting it into a shared helper script (e.g., scripts/utils.sh) to reduce code duplication and simplify future architecture edge-case handling.

  • scripts/setup-e2e.sh#L72-L81: Replace the inline logic by sourcing and invoking the extracted utility.
  • scripts/ci/run_integration_suite.sh#L152-L161: Replace the inline logic with the extracted utility.
  • .github/workflows/test-ci.yml#L154-L163: Refactor to use the shared utility, or retain the inline logic if importing and evaluating an external script inside the CI YAML context proves too cumbersome.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@scripts/setup-e2e.sh` around lines 72 - 81, Extract the duplicated host
architecture normalization into a shared helper, such as a utility function in
scripts/utils.sh, preserving the existing x86_64, aarch64/arm64, and fallback
behavior. Update scripts/setup-e2e.sh lines 72-81 and
scripts/ci/run_integration_suite.sh lines 152-161 to source and invoke the
helper; refactor .github/workflows/test-ci.yml lines 154-163 to use it if
practical, otherwise explicitly retain its inline logic.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@scripts/setup-e2e.sh`:
- Around line 83-91: Move the image architecture validation block in
scripts/setup-e2e.sh (lines 83-91) to immediately after the service startup
command so the image exists before docker inspect runs. In
scripts/ci/run_integration_suite.sh (lines 163-171), ensure the backend image is
pulled or built before the architecture check and reorder the check accordingly;
preserve the existing mismatch warning behavior.

---

Nitpick comments:
In `@scripts/setup-e2e.sh`:
- Around line 72-81: Extract the duplicated host architecture normalization into
a shared helper, such as a utility function in scripts/utils.sh, preserving the
existing x86_64, aarch64/arm64, and fallback behavior. Update
scripts/setup-e2e.sh lines 72-81 and scripts/ci/run_integration_suite.sh lines
152-161 to source and invoke the helper; refactor .github/workflows/test-ci.yml
lines 154-163 to use it if practical, otherwise explicitly retain its inline
logic.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 81bb4c6a-f02a-4d56-b4d1-abbf737ea5a2

📥 Commits

Reviewing files that changed from the base of the PR and between d28706a and 87a3781.

📒 Files selected for processing (3)
  • .github/workflows/test-ci.yml
  • scripts/ci/run_integration_suite.sh
  • scripts/setup-e2e.sh

Comment thread scripts/setup-e2e.sh
Comment on lines +83 to +91
# Get image architecture
image_name="langflowai/openrag-backend:${OPENRAG_VERSION:-latest}"
image_arch=$(${CONTAINER_RUNTIME} inspect --format '{{.Architecture}}' "$image_name" 2>/dev/null || echo "")

if [[ -n "$image_arch" && "$image_arch" != "$host_arch_norm" ]]; then
echo "WARNING: Architecture mismatch detected!"
echo "Host architecture is '${host_arch}' (${host_arch_norm}), but container image architecture is '${image_arch}'."
echo "The backend will run under QEMU emulation, which is extremely slow and may cause timeouts."
fi

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Ensure the image exists locally before inspecting its architecture.

If the target container image has not been pulled or built locally prior to running docker inspect, the command will fail and evaluate to an empty string. This causes the architecture mismatch check (if [[ -n "$image_arch" ... ]]) to silently bypass the warning exactly when it is most needed, such as on a developer's first-time setup.

  • scripts/setup-e2e.sh#L83-L91: Move this validation block immediately below the service startup command (e.g., after make dev-cpu on line 100) so the image is guaranteed to be present.
  • scripts/ci/run_integration_suite.sh#L163-L171: Verify that the backend image is pulled or built before this check is executed, and reorder it to occur after the pull/build step if necessary.
📍 Affects 2 files
  • scripts/setup-e2e.sh#L83-L91 (this comment)
  • scripts/ci/run_integration_suite.sh#L163-L171
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@scripts/setup-e2e.sh` around lines 83 - 91, Move the image architecture
validation block in scripts/setup-e2e.sh (lines 83-91) to immediately after the
service startup command so the image exists before docker inspect runs. In
scripts/ci/run_integration_suite.sh (lines 163-171), ensure the backend image is
pulled or built before the architecture check and reorder the check accordingly;
preserve the existing mismatch warning behavior.

@github-actions github-actions Bot added bug 🔴 Something isn't working. and removed bug 🔴 Something isn't working. labels Jul 14, 2026
@github-actions github-actions Bot added bug 🔴 Something isn't working. and removed bug 🔴 Something isn't working. labels Jul 14, 2026
@github-actions github-actions Bot added bug 🔴 Something isn't working. and removed bug 🔴 Something isn't working. labels Jul 14, 2026
@github-actions github-actions Bot added bug 🔴 Something isn't working. and removed bug 🔴 Something isn't working. labels Jul 14, 2026
@github-actions github-actions Bot added the bug 🔴 Something isn't working. label Jul 14, 2026
@github-actions github-actions Bot added bug 🔴 Something isn't working. and removed bug 🔴 Something isn't working. labels Jul 14, 2026
@github-actions github-actions Bot added bug 🔴 Something isn't working. and removed bug 🔴 Something isn't working. labels Jul 14, 2026
@github-actions github-actions Bot added bug 🔴 Something isn't working. and removed bug 🔴 Something isn't working. labels Jul 14, 2026
@github-actions github-actions Bot added bug 🔴 Something isn't working. and removed bug 🔴 Something isn't working. labels Jul 14, 2026
@github-actions github-actions Bot added bug 🔴 Something isn't working. and removed bug 🔴 Something isn't working. labels Jul 14, 2026
@github-actions github-actions Bot added bug 🔴 Something isn't working. and removed bug 🔴 Something isn't working. labels Jul 14, 2026
@lucaseduoli
lucaseduoli merged commit 7f3ee04 into main Jul 14, 2026
24 of 26 checks passed
@github-actions
github-actions Bot deleted the fix/build_arch branch July 14, 2026 17:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug 🔴 Something isn't working. ci ⬛ CI/CD, build, and infrastructure issues lgtm

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants