Skip to content

patterns.md

Latest commit

 

History

History
295 lines (285 loc) · 20.9 KB

patterns.md

File metadata and controls

295 lines (285 loc) · 20.9 KB
Raw

Project Patterns

Current conventions for the Cymbal Coffee Oracle 26ai + Vertex AI demo. This file and .agents/knowledge/ are the living surfaces for durable agent guidance.

Architecture

  • Litestar app setup stays thin: src/app/server/ wires plugins, middleware, routes, and templates; domain packages own behavior.
  • Domain packages use controllers/, services/, and schemas/ subpackages. Cross-domain imports go through package exports, not deep private modules.
  • Dishka has exactly three user providers in src/app/ioc.py: LitestarPersistenceProvider, IntegrationsProvider, and DomainServiceProvider.
  • LitestarPersistenceProvider owns APP-scoped OracleAsyncConfig and REQUEST-scoped OracleAsyncDriver.
  • IntegrationsProvider owns APP-scoped external integrations: GenAI client, Oracle ADK store, SQLSpec ADK session service, intent classifier, persona manager, and ADK runner.
  • DomainServiceProvider owns REQUEST-scoped services: products, stores, cache, metrics, tools, Vertex AI, and vector-search orchestration.
  • SQL lives in src/app/db/sql/*.sql as named queries. Domain services call db_manager.get_sql("name"); inline service SQL is rejected by tests.
  • List endpoints use create_filter_dependencies in controllers and list_with_count(*filters) in services for pagination/filter wiring.
  • Product vectors and embedding-cache vectors are VECTOR(3072, FLOAT32) values generated by gemini-embedding-2.
  • Oracle feature-demo metadata belongs inline in the baseline DDL when the branch is still curating the reference schema. Keep COMMENT ON statements for compatibility and add 26ai ANNOTATIONS(...) on documented table, column, and normal-index surfaces; do not create follow-up migrations or separate ALTER ... ANNOTATIONS(...) statements just because local SQL parsers lag Oracle syntax.
  • ADK conversation state uses SQLSpec's Oracle ADK extension tables (adk_session, adk_event, optional memory). Litestar browser sessions use the separate SQLSpec Litestar session table (app_session).
  • Chat uses classifier-first ADK 2 orchestration. ADKRunner checks response cache, classifies intent, sends PRODUCT_RAG turns through direct menu grounding (storing recommended products in session state under last_products for subsequent pronoun resolution), and uses the ADK Workflow/BaseNode graph for non-RAG streamed model conversation.
  • ADK tools are closure-bound per request inside ADKRunner; they must use the active Dishka request services, not module globals.
  • /api/chat/stream is an SSE endpoint. Streaming runs ADK with RunConfig(streaming_mode=StreamingMode.SSE) for non-RAG turns and preserves the non-streaming response payload keys.
  • Product RAG turns must not stream speculative model deltas. The runner emits a single grounded final event. _compose_grounded_answer may make a bounded Gemini structured-output call to choose candidate product ids and detect an off-menu term, but the model must not write final product copy. Python validates selected ids against retrieved rows, renders names/prices/details from those rows, and falls back to _grounded_product_answer on timeout, malformed output, invalid ids, or non-credential model errors. Credential failures surface as AIServiceUnconfigured.
  • Chat page history is hydrated from the Litestar-session-bound ADK session. Display history lives in ADK session state under display_history, with ADK event replay as a fallback.
  • The chat sidebar clear button deletes the current ADK session/events and clears only the Litestar ADK bridge keys. It must not clear menu data, metrics, response cache, or embedding cache.
  • Product RAG tool calls record vector_query, embedding_ms, oracle_ms, tool_ms, result counts, embedding cache status, and grounded-answer mode (structured, template, timeout, rejected, or error); the chat UI shows the stable message-level telemetry along with intent and response-cache status.
  • Store/location/inventory work stays in the products domain: seed coordinates, Dallas, and curated inventory; route store answers from service facts.
  • Store selectors and inventory UI must be driven by StoreService and fixture data. Do not hardcode a few store IDs in Jinja as "presets"; that silently narrows the demo to those locations and drifts from the seeded store list.
  • Store-aware product RAG must vary or bypass response-cache reads whenever safe location/store context is present, so one store's inventory answer is not reused for another store.
  • Maps URLs are the no-key baseline. Browser coordinates are opt-in and request-scoped; embeds require a separate restricted Maps Embed key.
  • Frontend pages are HTMX + Jinja + Tailwind v4 + vanilla JavaScript/ApexCharts through litestar-vite template mode, not a standalone SPA.
  • src/resources/ is the Vite/npm project root. Keep package.json, package-lock.json, tsconfig.json, vite.config.ts, public/, generated bridge files, and node_modules/ there; Python ViteConfig.paths.root must point at the same directory.
  • Explore-only educational calculators stay client-only in src/resources/*.js and wire to Jinja markup through data-* attributes. Keep this as vanilla Vite-bundled JavaScript.
  • Vector footprint estimates use raw bytes by format (N * d * 8 for FLOAT64, N * d * 4 for FLOAT32, N * d for INT8, N * ceil(d / 8) for BINARY), and Oracle's rough HNSW Vector Pool estimate of 1.3 * N * d * element_size. Use DBMS_VECTOR.INDEX_VECTOR_MEMORY_ADVISOR for exact HNSW/IVF sizing; do not model IVF or HNSW neighbors with invented client-side overhead formulas.
  • EXPLAIN PLAN uses two named SQL calls: one EXPLAIN PLAN FOR ... and one DBMS_XPLAN.DISPLAY() read.
  • coffee is the app CLI. Keep bulk-embed, export-fixtures, load-fixtures, clear-cache, model-info, and run visible there.
  • Keep public CLI modules mostly declarative. Large workflows belong in focused private app.cli._helpers modules; small command-local helpers can live in commands.py. Do not add compatibility shim or facade modules.
  • sanitize_for_json camel-cases msgspec Struct keys for wire encoding. In the chat domain (like ADKRunner and grounding formatting helpers), check both snake_case and camelCase keys using _get_field(row, snake_name) to remain resilient to case conversion differences between unit test mocks and runtime database responses.
  • Product availability lookups (PRODUCT_AVAILABILITY) resolve product names via exact match first, falling back to pronoun resolution from last_products in session history, and finally using Vertex AI vector search to resolve partial/imprecise product names (e.g. 'Gemini' -> 'Gemini Rush') before checking store inventory.
  • ORDS Stateless Deployment on Cloud Run: Deploy ORDS on Cloud Run stateless, peered via Direct VPC, utilizing Secret Manager for the database system password (DATABASE_SYSTEM_PASSWORD) to enable automatic schema registration/validation on container boot. (derived from cloudrun-ords-deploy)
  • Oracle CDN for APEX Assets: Set IMAGE_PREFIX to Oracle's public APEX CDN (https://apex.oracle.com/i/ or version-specific URL) via database script. This keeps the ORDS container stateless and avoids the need for mounting static asset directories or using Cloud Storage FUSE. (derived from cloudrun-ords-deploy)
  • Image Re-hosting for Private Deployments: When deploying to Cloud Run in restricted environments where external registry authentication is unavailable or unreliable at deployment time, re-host official images (like container-registry.oracle.com/database/ords:<version>) in GCP Artifact Registry. (derived from cloudrun-ords-deploy)
  • AutoREST Table Enablement: Utilize Oracle ORDS AutoREST capability to expose database tables (e.g., PRODUCTS) as REST endpoints directly via SQLSpec migrations or startup scripts using simple PL/SQL calls (ORDS.enable_object), providing a zero-code private data access path. (derived from cloudrun-ords-deploy)
  • Naming a query parameter argument query in a Litestar controller action (e.g., def handler(query: FromQuery[str])) causes Litestar to pass the entire query parameters MultiDict to it. Use a different Python argument name (e.g., search_query) and map it explicitly using QueryParameter(name="query") to avoid this. (derived from fix-explore-search-ux)

Code Style

  • Use SPDX headers on source files: SPDX-FileCopyrightText: 2026 Google LLC plus SPDX-License-Identifier: Apache-2.0. Ruff's CPY001 rule enforces presence; the license-headers prek hook (running tools/license_headers.py) inserts them on commit. See .agents/code-styleguides/python.md for the per-language comment forms.
  • Use PEP 604 unions (str | None) and typed public boundaries.
  • Keep demo-facing public Python modules public-first: exported controllers, services, settings, commands, and lifecycle entry points should appear before private mechanics. Move substantial helper runs into focused private sibling modules or below the public API; avoid compatibility shim or facade modules.
  • src/app/ioc.py must not use from __future__ import annotations; Dishka introspects provider annotations at runtime.
  • Prefer handler-argument Inject[T] over route-level @inject decorators.
  • Use @async_inject for async Click commands; command modules should not call sqlspec.utils.sync_tools.run_ directly or define nested async runners.
  • SQLSpec reads should pass schema_type= whenever a typed row/schema exists.
  • Use SQLSpecAsyncService from app.lib.service as the domain service base.
  • Use SQLSpec builders for dynamic writes (sql.update, sql.insert_into) and named SQL for static reads.
  • Use from_json / to_json from app.utils.serialization, not ad hoc json.loads / json.dumps, in app code.
  • Use schema_dump(..., wire_format=False) from app.utils.serialization when dumping msgspec schemas into database writes that require Python/Oracle column names instead of camelized wire names.
  • Push aggregate null handling into SQL with COALESCE, not Python value or 0.
  • For gemini-embedding-2, encode query/document purpose in the embedding input text. Do not send the old embedding task_type API parameter.
  • Keep placeholder Vertex project checks on the typed AIServiceUnconfigured path so local misconfiguration returns a clean 503.
  • Settings stay dataclass/cached, quiet, and effectively immutable; remove future knobs until source behavior wires them.

Testing

  • Prefer @pytest.mark.anyio for async tests.
  • Test files live under strict behavior and module-path buckets: src/tests/unit/<module path>/test_*.py or src/tests/integration/<module path>/test_*.py. Do not add new top-level issue buckets such as src/tests/api, and do not add direct src/tests/unit/test_*.py or src/tests/integration/test_*.py files.
  • Before creating a new test file, find the existing module-path test file and add the behavior there. Prefer parameterized cases, shared fixtures, and existing functional sections over one-file-per-issue coverage.
  • Nested test directories are real packages with SPDX-bearing __init__.py files so repo ruff checks do not treat moved tests as implicit namespaces.
  • Tests should prove application behavior: service logic, controller responses, typed SQL/data contracts, settings parsing, DI resolution, chat payloads, vector telemetry, and HTMX partial behavior.
  • Do not add tests whose only subject is repo structure, source ordering, docs text, pyproject.toml, workflow YAML, tool scripts, or third-party import surfaces. Use lint, build, smoke checks, or focused tool validation for those concerns instead of growing src/tests.
  • coffee upgrade is the packaged/end-user install command and loads committed fixtures after migrations. Raw SQLSpec developer commands such as downgrade and current stay under python manage.py database ...; do not expose coffee downgrade.
  • Put reusable test constants and repo paths in src/tests/support/ instead of hardcoding relative parent depths in moved tests.
  • Handler tests that hit Inject[T] must mock the full DI chain when the test is not meant to construct real services.
  • API tests for HTMX partial branches should send HX-Request: true or use the repo's HTMX test fixture.
  • Real Oracle integration tests use the repo-managed Oracle lifecycle (make start-infra, then uv run python manage.py database upgrade --no-prompt) and share DDL/fixture loading through integration fixtures. Keep SQLSpec pool resets only where needed for pytest event-loop binding.
  • Vector-search tests should assert typed contextual fields and result counts, not only final answer text.
  • Keep cache and Oracle mutation tests deterministic under parallel workers by using unique keys/SKUs plus targeted cleanup instead of truncating shared fixture tables per test.
  • Use focused tests while editing, then run make lint and make test before claiming branch-level completion.

Operational Gotchas

  • HNSW ORGANIZATION INMEMORY NEIGHBOR GRAPH indexes require non-zero vector_memory_size before index creation. The local Oracle startup scripts configure this before migrations create vector indexes.
  • Oracle Free Edition has a hard SGA/PGA cap; the project default vector pool is sized to fit the demo dataset instead of trying to raise SGA targets.
  • Local APEX/ORDS stays optional: make start-infra is DB-only, while APEX-enabled flows start ORDS through the shared OrdsSidecar used by manage.py infra ords. ORDS startup must probe the runtime version and fail clearly below 26.1.1; inconclusive version probes are retryable until timeout. /ords/ readiness may accept routed non-5xx responses, but /i/ static media readiness must reject 403/404 because the smoke check uses curl -fsS.
  • V$SGAINFO reports the vector pool as Vector Memory Area; verification SQL should accept that label.
  • VECTOR(N, FLOAT32) DDL, EMBEDDING_DIMENSIONS, and committed fixture vector lengths are co-versioned. Do not mix older lower-dimensional examples into this app.
  • ORACLE_ADK_IN_MEMORY and ORACLE_LITESTAR_SESSION_IN_MEMORY default to true so SQLSpec extension tables use Oracle INMEMORY where the database supports it.
  • OracleAsyncConfig uses connection_config=; older pool_config= wiring can silently drop credentials and surface as DPY-4001.
  • .agents/archive/ is ignored disposable history; synthesize durable knowledge into .agents/knowledge/ or this file before archive cleanup.
  • Flow sync/status is a Flow skill workflow backed by Beads state and .agents/ docs; do not assume a flow sync shell command exists, and do not run bd sync because Beads does not expose that command.
  • APEXlang lifecycle commands require SQLcl 26.1.2+ and live under manage.py infra apex generate|export|validate|import. Keep source-controlled APEX assets under src/apex/<alias>/. SQLcl 26.1.2 generated this repo's APEXlang project with hyphenated directories such as shared-components/ and supporting-objects/; treat generated SQLcl output as authoritative when docs and older notes disagree.
  • APEX 26.1 can validate app-level restDataSource APEXlang, but this local runtime currently rolls back import with WWV_WEBSRC_MODULE_RSERVER_FK when REST Data Sources reference a restDataSourceServer. Use SQL-backed APEX reports for importable demo pages and keep REST Source Catalog wiring open until a generated App Builder round-trip or Oracle patch proves the server/data-source import path.
  • Logging must use app.lib.log processors, set Litestar/Granian env defaults, exclude static-asset logs, and filter only known ADK/Authlib warning noise.
  • PyApp onefile releases use the custom Bundle-Patch-Compile path, not [tool.pyapp] or uvx pyapp build: build assets and wheel, bundle Python 3.13 dependencies with tools/bundler.py, patch PyApp to install under the XDG config path (~/.config/cymbal-coffee by default), and compile Linux launchers with cargo zigbuild --target <arch>-unknown-linux-gnu.2.17.
  • make build-onefile must force UV_PYTHON/PYAPP_BUILD_PYTHON to the explicit CPython 3.13 build interpreter because the repo-local .python-version remains 3.12 for normal development.
  • Release containers wrap the onefile binary, not a separate Python install path. Build from the single distroless Dockerfile at tools/deploy/docker/Dockerfile, allow Docker context access to dist/coffee-*-linux-gnu, and expose /app/wallet with TNS_ADMIN/WALLET_LOCATION pointed there for read-only Oracle wallet mounts.
  • Release CI builds onefiles and containers on native architecture runners: ubuntu-24.04 for x86_64/amd64 and ubuntu-24.04-arm for aarch64/arm64. Cross-compilation via emulation is rejected — both arches build natively in the matrix and the final GitHub Release job downloads dist/coffee-linux-x86_64
    • dist/coffee-linux-aarch64 plus checksums and attaches them with fail_on_unmatched_files: true. Onefile and container smoke checks invoke coffee upgrade --help (the packaged end-user command) rather than coffee run.
  • Docs publish via .github/workflows/docs.yml on push to main: builds with sphinx-build -W --keep-going, uploads actions/upload-pages-artifact@v3, deploys via actions/deploy-pages@v4. Do not add oracledb, vertexai, google.adk, google.cloud, or google.genai to autodoc_mock_imports in docs/conf.py — they are hard deps and mocking breaks sqlspec's import-time oracledb.__version__ lookup.
  • Docs information architecture is three tiers and locked: index.md hero + tour.md walkthrough at the front, concepts/{vector-search,rag,agent-flow}.md for deep dives, and reference/{quickstart,cli,api,internals,developers}.md for the reference appendix. Concept page count is exactly 3 — not 2, not 4. Use the sphinx-immaterial theme (not shibuya) with custom admonitions tour-stop, oracle-internals, and agent-detail defined in conf.py.
  • Code embeds in docs use literalinclude with # docs:start-<name> / # docs:end-<name> anchor pairs in the source file — never copy-paste a snippet inline. Each embed has a one-or-two-sentence framing paragraph above it that names the file/class and its role so readers know what larger thing the snippet is part of.
  • Autodoc scope is intentionally narrow: ADKRunner, CacheService, MetricsService, ProductService, and the three domain/*/schemas packages. Do NOT add CLI commands, settings dataclasses, plugin wiring, or IoC providers — their shapes are stable and reading the source is faster than reading generated docs.
  • AsyncTestClient MUST be entered with async with; a bare instance never runs the app lifespan, so Dishka containers (and the Oracle pools they hold) leak across tests in the same process. Fixtures look like async def client(app): async with AsyncTestClient(app) as c: yield c.
  • HTMX endpoints that also need to accept JSON (e.g. /api/vector-demo) parse the request body directly (await request.json() or await request.form()) — do NOT declare them as a typed data: body parameter. Litestar's body-DTO path rejects HTMX form posts when the route declares JSON body data.
  • hx-ext="litestar" belongs only on JSON-templating panels. Scope it out with ignore:litestar on partial-HTML-swap surfaces, otherwise the extension intercepts HTML responses and breaks the swap.
  • The chat form must capture FormData before applying any busy/disabled state to inputs — disabling a control removes it from new FormData(form), which causes the backend to receive an empty message and stream Message cannot be empty. Regression-check this ordering.
  • Cached chat responses store only the product-lookup sql_phases. At read time the cache hit is wrapped in a fresh get-cached-response phase so the UI shows both the cache hit and the original product-lookup context without retaining the original cache-miss phase.
  • Streaming /api/chat/stream exception handling lives inside the async generator: catch broadly, log with logger.aexception(), and yield a sanitized SSE error event. Do not rely on Litestar exception middleware for failures that occur after SSE headers have been sent — it cannot intercept post-start stream errors.
  • If make lint fails with frontend-typecheck due to missing tsc (TypeScript compiler), ensure frontend assets are installed in src/resources/ by running uv run python manage.py assets install first. (derived from fix-explore-search-ux)
  • When committing uv.lock, ensure it has been resolved using the public PyPI index (e.g., by running with UV_CONFIG_FILE=/dev/null and --default-index https://pypi.org/simple if your workstation configures a Google-internal repository like us-python.pkg.dev), otherwise public CI builds will fail with 401 Unauthorized errors. (derived from fix-explore-search-ux)