Skip to content

Commit 68363c2

Browse files
feat: RBAC + database migration layer (#1538)
Co-authored-by: Sebastián Estévez <estevezsebastian@gmail.com>
1 parent 8e87366 commit 68363c2

122 files changed

Lines changed: 10674 additions & 488 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.env.example

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -258,3 +258,26 @@ SESSION_SECRET=
258258
# - "saas": Public Cloud
259259
# - "on_prem": Private Cloud
260260
OPENRAG_RUN_MODE=oss
261+
262+
# Permission/identity cache backend. Only "memory" is currently wired.
263+
# When CACHE_BACKEND=memory the RBAC permission cache and OAuth-subject
264+
# cache are per-process, so OpenRAG must run with UVICORN_WORKERS=1 (and
265+
# replicaCount=1 in helm). Anything >1 hard-fails at startup until cache
266+
# is shared across processes (planned: Redis).
267+
CACHE_BACKEND=memory
268+
# UVICORN_WORKERS=1
269+
270+
# Storage mode. Default `db` — workspace config + chat history live in
271+
# the SQL DB only; no `config.yaml`, `conversations.json`, or
272+
# `session_ownership.json` are ever written. Pre-existing JSON files
273+
# are imported once on first boot, then ignored.
274+
# - db (default) DB-only, no JSON written
275+
# - hybrid DB + JSON dual-write (Phase B fallback)
276+
# - files legacy JSON-only path; DB untouched
277+
OPENRAG_STORAGE_MODE=db
278+
279+
# RBAC kill switch. Default OFF — the system behaves like the pre-RBAC
280+
# release: any authenticated user has full access; API-key role
281+
# overrides are also bypassed. Set to `true` to turn the permissions
282+
# system on for multi-user deployments. Available in all run modes.
283+
# OPENRAG_RBAC_ENFORCE=true

.gitignore

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -52,3 +52,7 @@ documents/ibm_anthropic.pdf
5252
documents/docling.pdf
5353
/opensearch-data-new-lf
5454
/opensearch-data2
55+
data/openrag.db
56+
data/openrag.db
57+
*.db
58+
/data

AGENTS.md

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,3 +37,19 @@ Read the `SKILL.md` files directly. The frontmatter `description` tells you when
3737
- Skill bodies are intentionally kept agent-neutral. Do not add references to tools or features that only exist in one runtime (for example, do not name specific slash commands, hook systems, or task-tracking tools).
3838
- Claude-Code-specific plumbing belongs in `plugin.json` or `.claude/`, not in `SKILL.md`.
3939
- See `plugins/README.md` for the full layout and distribution model.
40+
41+
## Operational constraints
42+
43+
**Single-worker only (until Redis cache lands).** The RBAC permission cache and OAuth-subject→DB-id cache are both per-process (`cachetools.TTLCache`). Running with multiple uvicorn workers or multiple helm replicas means a role grant or revoke takes effect in only one process; the others serve stale permissions for up to `OPENRAG_PERM_CACHE_TTL` seconds (default 60). The startup event in `src/main.py` enforces `UVICORN_WORKERS<=1` and `CACHE_BACKEND=memory` and hard-fails otherwise. To horizontally scale, swap the cache to Redis first.
44+
45+
**RBAC is opt-in.** `OPENRAG_RBAC_ENFORCE` defaults to `false`, which makes OpenRAG behave like the pre-RBAC release: every authenticated user has full access; API-key role overrides are also bypassed. To turn the permissions system on (admin/developer/user/viewer roles, `require_permission` gates, audit denials), set `OPENRAG_RBAC_ENFORCE=true`. Available in all `OPENRAG_RUN_MODE` values — operators own the trade-off. The startup event logs the enforcement state on every boot.
46+
47+
**Dev-local with backend on host.** When you run `make dev-local-cpu` (or `dev-local`) and then `make backend` on the host, OpenSearch needs to resolve `openrag-backend` to the host machine so it can fetch JWKS for OIDC validation. The base `docker-compose.yml` does NOT add this alias — it would break CI, where the backend is a docker-compose service. To enable the host-backend mode, layer the override file:
48+
49+
```bash
50+
docker compose -f docker-compose.yml -f docker-compose.host-backend.yml up -d opensearch dashboards langflow
51+
make backend # in another terminal
52+
make frontend # in another terminal
53+
```
54+
55+
The override only adds `extra_hosts: openrag-backend:host-gateway` to the OpenSearch service. Without it, OIDC works because docker DNS routes `openrag-backend` to the in-compose backend container.

Dockerfile.backend

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -41,6 +41,12 @@ COPY src/ ./src/
4141
COPY flows/ ./flows/
4242
COPY securityconfig/ ./securityconfig/
4343
COPY cloud_securityconfig/ ./cloud_securityconfig/
44+
# Alembic migrations — the runtime calls run_alembic_upgrade("head") on
45+
# startup. Without these the migration silently no-ops with
46+
# "alembic.ini not found; skipping schema upgrade", and the SQL DB
47+
# starts empty (no users / conversations / session_ownership tables).
48+
COPY alembic.ini ./
49+
COPY alembic/ ./alembic/
4450

4551
# -----------------------------------------------------------------------------
4652
# Stage: runtime (minimal image)

alembic.ini

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
[alembic]
2+
script_location = alembic
3+
prepend_sys_path = . src
4+
version_path_separator = os
5+
file_template = %%(rev)s_%%(slug)s
6+
7+
# DATABASE_URL is resolved at runtime in env.py — do not set sqlalchemy.url here
8+
sqlalchemy.url =
9+
10+
[loggers]
11+
keys = root,sqlalchemy,alembic
12+
13+
[handlers]
14+
keys = console
15+
16+
[formatters]
17+
keys = generic
18+
19+
[logger_root]
20+
level = WARN
21+
handlers = console
22+
qualname =
23+
24+
[logger_sqlalchemy]
25+
level = WARN
26+
handlers =
27+
qualname = sqlalchemy.engine
28+
29+
[logger_alembic]
30+
level = INFO
31+
handlers =
32+
qualname = alembic
33+
34+
[handler_console]
35+
class = StreamHandler
36+
args = (sys.stderr,)
37+
level = NOTSET
38+
formatter = generic
39+
40+
[formatter_generic]
41+
format = %(levelname)-5.5s [%(name)s] %(message)s
42+
datefmt = %H:%M:%S

alembic/env.py

Lines changed: 90 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,90 @@
1+
"""Alembic env — supports both sync (offline / CLI) and async runtimes.
2+
3+
DATABASE_URL is resolved via db.engine.get_database_url so the same code
4+
path applies whether you run alembic from the CLI or programmatically
5+
during app startup.
6+
"""
7+
8+
from __future__ import annotations
9+
10+
import asyncio
11+
import os
12+
import sys
13+
from logging.config import fileConfig
14+
from pathlib import Path
15+
16+
from alembic import context
17+
from sqlalchemy import engine_from_config, pool
18+
from sqlalchemy.ext.asyncio import AsyncEngine, create_async_engine
19+
20+
# Make `src/` importable
21+
ROOT = Path(__file__).resolve().parent.parent
22+
SRC = ROOT / "src"
23+
if str(SRC) not in sys.path:
24+
sys.path.insert(0, str(SRC))
25+
26+
from db.base import metadata as base_metadata # noqa: E402
27+
from db.engine import get_database_url # noqa: E402
28+
import db.models # noqa: E402,F401 (registers tables on metadata)
29+
from sqlmodel import SQLModel # noqa: E402
30+
31+
config = context.config
32+
33+
if config.config_file_name is not None:
34+
try:
35+
fileConfig(config.config_file_name)
36+
except Exception:
37+
pass
38+
39+
target_metadata = SQLModel.metadata
40+
41+
42+
def _sync_url() -> str:
43+
"""Alembic prefers a sync URL for offline mode. Convert async drivers."""
44+
url = get_database_url()
45+
return (
46+
url.replace("+aiosqlite", "")
47+
.replace("postgresql+asyncpg", "postgresql+psycopg")
48+
.replace("mysql+aiomysql", "mysql+pymysql")
49+
)
50+
51+
52+
def run_migrations_offline() -> None:
53+
context.configure(
54+
url=_sync_url(),
55+
target_metadata=target_metadata,
56+
literal_binds=True,
57+
dialect_opts={"paramstyle": "named"},
58+
render_as_batch=True, # SQLite-friendly
59+
)
60+
with context.begin_transaction():
61+
context.run_migrations()
62+
63+
64+
def do_run_migrations(connection) -> None:
65+
context.configure(
66+
connection=connection,
67+
target_metadata=target_metadata,
68+
render_as_batch=True,
69+
)
70+
with context.begin_transaction():
71+
context.run_migrations()
72+
73+
74+
async def run_async_migrations() -> None:
75+
connectable: AsyncEngine = create_async_engine(
76+
get_database_url(), poolclass=pool.NullPool, future=True
77+
)
78+
async with connectable.connect() as connection:
79+
await connection.run_sync(do_run_migrations)
80+
await connectable.dispose()
81+
82+
83+
def run_migrations_online() -> None:
84+
asyncio.run(run_async_migrations())
85+
86+
87+
if context.is_offline_mode():
88+
run_migrations_offline()
89+
else:
90+
run_migrations_online()

alembic/script.py.mako

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
"""${message}
2+
3+
Revision ID: ${up_revision}
4+
Revises: ${down_revision | comma,n}
5+
Create Date: ${create_date}
6+
7+
"""
8+
from typing import Sequence, Union
9+
10+
from alembic import op
11+
import sqlalchemy as sa
12+
${imports if imports else ""}
13+
14+
# revision identifiers, used by Alembic.
15+
revision: str = ${repr(up_revision)}
16+
down_revision: Union[str, Sequence[str], None] = ${repr(down_revision)}
17+
branch_labels: Union[str, Sequence[str], None] = ${repr(branch_labels)}
18+
depends_on: Union[str, Sequence[str], None] = ${repr(depends_on)}
19+
20+
21+
def upgrade() -> None:
22+
${upgrades if upgrades else "pass"}
23+
24+
25+
def downgrade() -> None:
26+
${downgrades if downgrades else "pass"}

0 commit comments

Comments
 (0)