fix: add replicas to opensearch in cpd

[lucaseduoli]

This pull request introduces improved configuration and management of OpenSearch index shard and replica counts, especially for OpenRAG-managed indices. The main changes add new environment variables to control the number of shards and replicas, ensure that existing indices are automatically reconciled to these settings at startup, and refactor related code for clarity and testability. Documentation and tests are updated to reflect and verify the new behavior.

OpenSearch index configuration and management:

• Added new environment variables (OPENRAG_OPENSEARCH_NUMBER_OF_SHARDS, OPENRAG_OPENSEARCH_NUMBER_OF_REPLICAS, and OPENRAG_ENSURE_INDEX_REPLICAS_ON_STARTUP) to control the number of shards and replicas for OpenRAG indices, with appropriate defaults for local and production environments. (.env.example, [1]; docker-compose.yml, [2]; docs/docs/reference/configuration.mdx, [3]
• Updated OpenSearch index creation logic throughout the codebase to use the configured shard and replica counts, replacing hardcoded values. (src/config/settings.py, [1] [2]; src/utils/embeddings.py, [3]; src/utils/opensearch_init.py, [4] [5]

Automatic index replica reconciliation:

• Implemented startup logic to reconcile the replica count of all managed OpenRAG indices to match the configured value, including a helper function and integration into the application startup sequence. (src/app/lifespan.py, [1]; src/config/settings.py, [2]; src/utils/opensearch_init.py, [3] [4] [5] [6] [7]

Testing and documentation:

• Added and updated unit tests to verify that index creation and replica reconciliation use the configured values and behave as expected. (tests/unit/test_embedding_fields.py, [1]; tests/unit/test_ensure_index_replicas.py, [2]
• Updated documentation to describe the new configuration options and their effects. (docs/docs/reference/configuration.mdx, docs/docs/reference/configuration.mdxR155-R156)

These changes make OpenRAG's OpenSearch index management more flexible, robust, and production-ready.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: c42a5ec2-898f-4087-9625-d677c7f18f36

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/replicas_os

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

Build successful! ✅
Deploying docs draft.
Deploy successful! View draft