Enhance Redis integration and update documentation

- Updated `.env.example` to clarify Redis usage for rate limiting and presence management, ensuring developers understand the fallback mechanisms.
- Enhanced `AGENTS.md` with updated instructions for local API URL configurations and Redis integration details.
- Added Redis service configuration to `docker-compose.gcp-ops.yml` for local development and updated CI workflows to include Redis health checks.
- Improved health check routes in `health.rs` to utilize Redis for connection management, ensuring accurate service status reporting.
- Updated metrics handling in `metrics.rs` to track rate limit rejections, enhancing observability.

These changes improve the application's Redis integration, enhance documentation clarity, and provide better monitoring capabilities.

Author: infinitedim

.cursor/hooks/state/continual-learning-index.json

@@ -7,7 +7,7 @@
       "mtimeMs": 1780043491255.2393
     },
     "/home/yourblooo/.cursor/projects/home-yourblooo-development-portfolio-backend/agent-transcripts/b9c426f8-a2a1-4df3-bc36-8e849d755662/b9c426f8-a2a1-4df3-bc36-8e849d755662.jsonl": {
-      "mtimeMs": 1780236743482.536
+      "mtimeMs": 1780246125446.7759
     }
   }
 }

.cursor/hooks/state/continual-learning.json

@@ -1,8 +1,8 @@
 {
   "version": 1,
-  "lastRunAtMs": 1780236735739,
-  "turnsSinceLastRun": 11,
-  "lastTranscriptMtimeMs": 1780236735655.7478,
-  "lastProcessedGenerationId": "0369d852-36d7-4633-a5c0-10ff6be84fa7",
+  "lastRunAtMs": 1780246116164,
+  "turnsSinceLastRun": 5,
+  "lastTranscriptMtimeMs": 1780246115874.715,
+  "lastProcessedGenerationId": "fc8e9382-f917-4ab6-8ee6-433e6d5407bf",
   "trialStartedAtMs": null
 }

.env.example

@@ -21,7 +21,7 @@ DB_POOL_MIN=2
 DB_POOL_MAX=10
 DB_CONNECT_RETRY_SECS=60
 
-# Optional Redis URL (health probe only; app code does not connect yet — presence/gate not persisted)
+# Optional Redis URL (rate limiting + presence; falls back to in-memory when unset)
 REDIS_URL=redis://localhost:6379
 
 # ----------------------------------------------------------------------------
@@ -119,7 +119,7 @@ GEMINI_API_KEY=
 
 # ----------------------------------------------------------------------------
 # Phase 4B — Visitor presence (WebSocket)
-# GET /ws/presence — in-memory room counts; REDIS_URL reserved for scaling.
+# GET /ws/presence — Redis-backed room counts when REDIS_URL is set; in-memory fallback otherwise.
 # ----------------------------------------------------------------------------
 
 # ----------------------------------------------------------------------------

.github/workflows/ci.yml

@@ -69,8 +69,18 @@ jobs:
           --health-interval 5s
           --health-timeout 5s
           --health-retries 12
+      redis:
+        image: redis:7-alpine
+        ports:
+          - 6379:6379
+        options: >-
+          --health-cmd "redis-cli ping"
+          --health-interval 5s
+          --health-timeout 5s
+          --health-retries 12
     env:
       TEST_DATABASE_URL: postgres://portfolio:portfolio@localhost:5432/portfolio_test
+      TEST_REDIS_URL: redis://localhost:6379
     steps:
       - name: Checkout code
         uses: actions/checkout@v4

AGENTS.md

@@ -6,7 +6,7 @@
 - Do not discard the terminal experience; prefer dual entry (standard landing + gated terminal) over a full UI rewrite.
 - Create git commits and pull requests only when explicitly asked.
 - Gate stays enabled in development; bypass only via `GATE_BYPASS_SECRET`, not an automatic dev-off flag.
-- User fills API keys and other random secrets in `.env.development` locally (copy from `.env.example`); production via `.env` on GCP or Vercel platform env.
+- User fills API keys and other random secrets in `.env.development` locally (copy from `.env.example`); production via `.env` on GCP or Vercel platform env. Local `.env.development` API URLs should target `localhost:8080`, not production Cloud Run.
 - Gate puzzles should be NATAS-style web challenges (login, hidden paths, Referer), not SSH/Bandit/Behemoth simulations.
 - First time deploying to GCP; prefers clear, beginner-friendly infrastructure documentation and bootstrap guides.
 - Portfolio-frontend chats must proactively use Cursor SDK + Team Kit skills/MCP (`.cursor/rules/cursor-sdk-team-kit.mdc`, alwaysApply).
@@ -18,12 +18,12 @@
 - Multi-repo portfolio: `portfolio-backend` (Rust/Axum, SQLx/PostgreSQL) and `portfolio-frontend` (Next.js 16, React 19, terminal-interactive portfolio).
 - Planned public routing: standard landing at `/`, terminal at `/terminal`, three-level gate at `/gate` (and `/gate/[level]`).
 - Blog, social share, RSS, and other content routes stay shared/public; do not duplicate them across standard and terminal UIs.
-- Gate puzzles (backend-validated, no answers in frontend bundle): L1 static login `yourblooo0`/`yourblooo0`; L2 requires L1 completion before `/s3cr3t/users.txt` reveals login `yourblooo1` + env password; L3 backend validates `Referer: {SITE_URL}/terminal` in `complete_level_3`; gate JWT pinned to HS256 with `iss`/`aud`. Terminal route `noindex`; terminal SSR verifies JWT via `/api/gate/status`. Gate/terminal is UX puzzle layer, not API auth perimeter (admin JWT separate). `GATE_BYPASS_SECRET` via Next.js `proxy.ts` (`X-Gate-Bypass`), not Rust handlers. Legacy env `GATE_L2_STUB_MD5`, `GATE_L3_SHELLCODE_HASH`, `GATE_L3_OFFSET` unused (archived shellcode design); only `GATE_L1_ANSWER`, `GATE_L2_ANSWER`, `GATE_TOKEN_SECRET` required.
-- Backend API default port 8080; frontend align `NEXT_PUBLIC_API_URL` / `BACKEND_URL` with 8080 locally. Production (Vercel): both must be the HTTPS Cloud Run service base URL (no trailing slash); `NEXT_PUBLIC_*` changes require a Vercel redeploy; a wrong `*.run.app` URL yields a Google-edge 404 with zero Cloud Run requests. Most prod API traffic is Vercel SSR server-side, with silent empty fallbacks if the backend is unreachable. Local dev: backend `load_env_file()` reads `.env.development` when `ENVIRONMENT != production`; frontend `bun run dev` uses `--env-file=.env.development`; Docker Compose via `scripts/compose-dev.sh` or `--env-file .env.development`. CORS auto-merges localhost origins (ports 3000–3002) when `ENVIRONMENT != production`; gate vars documented in `.env.example`.
+- Gate puzzles (backend-validated, no answers in frontend bundle): L1 static login `yourblooo0`/`yourblooo0`; L2 requires L1 completion before `/s3cr3t/users.txt` reveals login `yourblooo1` + env password; L3 backend validates `Referer: {SITE_URL}/terminal` in `complete_level_3`; gate JWT pinned to HS256 with `iss`/`aud`. Terminal route `noindex`; terminal SSR verifies JWT via `/api/gate/status`. Gate/terminal is UX puzzle layer, not API auth perimeter (admin JWT separate). `GATE_BYPASS_SECRET` via Next.js `proxy.ts` (`X-Gate-Bypass`), not Rust handlers. Legacy env `GATE_L2_STUB_MD5`, `GATE_L3_SHELLCODE_HASH`, `GATE_L3_OFFSET` unused (archived shellcode design); only `GATE_L1_ANSWER`, `GATE_L2_ANSWER`, `GATE_TOKEN_SECRET` required. Rebuild/restart local Docker backend after gate answer or gate code changes — stale image keeps old L1 credentials.
+- Backend API default port 8080; frontend align `NEXT_PUBLIC_API_URL` / `BACKEND_URL` with 8080 locally. Production (Vercel): both must be the HTTPS Cloud Run service base URL (no trailing slash); `NEXT_PUBLIC_*` changes require a Vercel redeploy; a wrong `*.run.app` URL yields a Google-edge 404 with zero Cloud Run requests. Most prod API traffic is Vercel SSR server-side, with silent empty fallbacks if the backend is unreachable. `ROADMAP_EMAIL`/`ROADMAP_PASSWORD` are Cloud Run/GCP secrets only — not Vercel env. Local dev: backend `load_env_file()` reads `.env.development` when `ENVIRONMENT != production`; frontend `bun run dev` uses `--env-file=.env.development`; Docker Compose via `scripts/compose-dev.sh` or `--env-file .env.development`. CORS auto-merges localhost origins (ports 3000–3002) when `ENVIRONMENT != production`; gate vars documented in `.env.example`.
 - Feature status SSOT: `portfolio-frontend/FEATURE_PLANNING.md` (+ `docs/dual-ui-gate.md` for gate ops); `ROADMAP.md` removed May 2026 after performance backlog moved to Feature #33.
 - PWA is site-wide (`public/manifest.json`, `public/sw.js`, scope `/`, offline page); install prompt only after terminal onboarding tour completes.
-- Observability stack (Loki, Grafana, Prometheus) runs on a GCE ops VM in production via `docker-compose.gcp-ops.yml`; same stack in local `docker-compose.yml` for dev. Redis health uses real PING via `REDIS_URL`; not yet used for app cache or gate sessions. Optional `METRICS_BEARER_TOKEN` protects `/metrics`; Prometheus prod scrape uses bearer auth.
-- GCP production: backend on Cloud Run (asia-southeast2/Jakarta) with `max_instances=1` for in-memory gate sessions (Redis not used for gate yet), public `allUsers` `run.invoker` (Vercel frontend + public API; admin auth app-level), frontend on Vercel; `DATABASE_URL` must be the ops VM internal IP over Serverless VPC Access (not a subnet-CIDR host, not `localhost`); ops VM may use default VPC addressing or Terraform ops ranges—firewall must allow TCP 5432 from the VPC connector CIDR to the VM; unreachable Postgres at startup panics before the revision listens on 8080. Serverless VPC Access connector (`10.10.1.0/28`, not Direct VPC egress) with private-ranges-only egress to ops subnet `10.10.0.0/24` unless Cloud NAT for all-traffic; container command/args empty (image CMD `/app/portfolio-backend`, port 8080); Terraform (network, iam, artifact_registry, secrets, compute_ops, cloud_run); ops VM (e2-medium, no public IP, IAP SSH) hosts Postgres 16 + observability at `/mnt/data`; `REDIS_URL` optional on Cloud Run (health probe only, not in prod compose stack). CI via `deploy-gcp.yml` Workload Identity Federation.
+- Observability stack (Loki, Grafana, Prometheus) runs on a GCE ops VM in production via `docker-compose.gcp-ops.yml`; same stack in local `docker-compose.yml` for dev. Redis on ops VM (`6379`) backs distributed rate limiting and WebSocket presence when `REDIS_URL` is set; falls back to in-memory `tower_governor` + local presence counts when unset or unreachable. Optional `METRICS_BEARER_TOKEN` protects `/metrics`; Prometheus prod scrape uses bearer auth.
+- GCP production: backend on Cloud Run (asia-southeast2/Jakarta) with `max_instances=1` for in-memory gate sessions (Redis not used for gate yet), public `allUsers` `run.invoker` (Vercel frontend + public API; admin auth app-level), frontend on Vercel; `DATABASE_URL` must be the ops VM internal IP over Serverless VPC Access (not a subnet-CIDR host, not `localhost`); `REDIS_URL=redis://<ops_vm_internal_ip>:6379` from Terraform; ops VM may use default VPC addressing or Terraform ops ranges—firewall must allow TCP 5432 and 6379 from the VPC connector CIDR to the VM; unreachable Postgres at startup panics before the revision listens on 8080. Serverless VPC Access connector (`10.10.1.0/28`, not Direct VPC egress); Terraform egress `PRIVATE_RANGES_ONLY` — `ALL_TRAFFIC` through VPC without Cloud NAT breaks outbound APIs (roadmap.sh, GitHub, Resend, Gemini); container command/args empty (image CMD `/app/portfolio-backend`, port 8080); Terraform (network, iam, artifact_registry, secrets, compute_ops, cloud_run); ops VM (e2-medium, no public IP, IAP SSH) hosts Postgres 16 + observability at `/mnt/data`; `REDIS_URL` optional on Cloud Run (health probe only, not in prod compose stack). CI via `deploy-gcp.yml` Workload Identity Federation.
 - Next.js 16 edge handler is `src/proxy.ts` (`export function proxy()`), not legacy `middleware.ts`.
-- Frontend performance: `cacheComponents: true` (PPR) with Feature #33 docs at `docs/features/FEATURE_33_PERFORMANCE.md`; dual RUM (pino/Loki + Vercel Speed Insights). Pre-completion verification: backend `cargo fmt/check/clippy/test`; frontend `bun run lint` + `type-check`. Known Vitest hang: `background-manager.test.tsx` (~16 min).
+- Frontend performance: `cacheComponents: true` (PPR) with Feature #33 docs at `docs/features/FEATURE_33_PERFORMANCE.md`; `/roadmap` uses request-time fetch via `await headers()` under PPR; dual RUM (pino/Loki + Vercel Speed Insights). Production CSP must not use `strict-dynamic`/nonce without wiring nonce into Next.js scripts (`'unsafe-inline'` used); Vercel builds disable file logging (`LOG_TO_FILE`, `NEXT_PHASE=phase-production-build` — no `logs/server` mkdir); client logs POST plain JSON to `/api/logs` (in-memory crypto session fails across Vercel serverless instances). Pre-completion verification: backend `cargo fmt/check/clippy/test`; frontend `bun run lint` + `type-check`. Known Vitest hang: `background-manager.test.tsx` (~16 min).
 - Backend performance & integrations: all API routes P95 <50ms SLO (`docs/performance/API_SLA.md`, `config/slo-rules.yml`, Grafana 50ms alerts, `scripts/latency-smoke.sh` in CI); roadmap/github in-memory cache with stale-while-revalidate; roadmap auth via `ROADMAP_EMAIL`/`ROADMAP_PASSWORD` → `POST …/v1-login` (Terraform secrets `roadmap-email`/`roadmap-password`; `ROADMAP_AUTH_TOKEN` removed); Swagger UI disabled in production.

Cargo.lock

@@ -95,8 +95,8 @@ totp-rs = { version = "5", features = ["gen_secret", "otpauth"] }
 metrics = "0.24"
 metrics-exporter-prometheus = "0.16"
 
-# Redis (health probe; presence scaling reserved)
-redis = { version = "0.27", features = ["tokio-comp"] }
+# Redis (health, rate limiting, presence)
+redis = { version = "0.27", features = ["tokio-comp", "connection-manager"] }
 
 # OpenAPI / Swagger documentation. `axum_extras` lets utoipa parse path /
 # query params automatically; `uuid` and `chrono` map our DB types to

Cargo.toml

@@ -26,6 +26,27 @@ services:
       retries: 5
       start_period: 30s
 
+  redis:
+    image: redis:7-alpine
+    container_name: portfolio-redis
+    restart: unless-stopped
+    command: >
+      redis-server
+      --appendonly yes
+      --appendfsync everysec
+    ports:
+      - "6379:6379"
+    volumes:
+      - /mnt/data/redis:/data
+    networks:
+      - ops
+    healthcheck:
+      test: ["CMD", "redis-cli", "ping"]
+      interval: 10s
+      timeout: 5s
+      retries: 5
+      start_period: 10s
+
   loki:
     image: grafana/loki:3.3.2
     container_name: portfolio-loki

docker-compose.gcp-ops.yml

@@ -63,6 +63,8 @@ Target: **P95 < 50ms** for all HTTP routes.
 Routes in this tier cannot meet 50ms synchronously on cache miss. Strategy: Redis
 cache with background refresh; HTTP returns cached data or 202 Accepted.
 
+**Rate limiting:** When `REDIS_URL` is configured, auth/gate/contact/logs/analytics/github/newsletter/ai routes use Redis fixed-window counters (`ratelimit:{bucket}:{ip}`) with fail-open on Redis errors. When unset, `tower_governor` in-memory limits apply unchanged.
+
 | Method | Path | Handler | Module | Bottleneck | Strategy |
 |--------|------|---------|--------|------------|----------|
 | GET | `/api/roadmap/streak` | `get_streak` | `roadmap.rs` | Upstream HTTP ~100-600ms | Redis cache + background refresh |
@@ -84,7 +86,7 @@ cache with background refresh; HTTP returns cached data or 202 Accepted.
 | POST | `/api/newsletter/unsubscribe` | `unsubscribe` | `newsletter.rs` | DB UPDATE | OK |
 | POST | `/api/admin/newsletter/broadcast` | `broadcast` | `newsletter.rs` | N x email | Return 202 + async |
 | POST | `/api/auth/2fa/*` | various | `twofa.rs` | DB + crypto | OK |
-| GET | `/ws/presence` | `ws_handler` | `presence.rs` | WebSocket (long-lived) | N/A |
+| GET | `/ws/presence` | `ws_handler` | `presence.rs` | WebSocket (long-lived) | Redis room/total counters when `REDIS_URL` set |
 
 ### Exceptions (cannot meet 50ms by nature)