initial-plan.md

HK IPO Verify — Product Vision & Implementation Plan

This is the full product vision. For day-to-day status see progress.md; for the stack/infra/ops details a developer needs, see ../README.md; for the assignment this is built against, see goal.md.

The assignment (goal.md)

The take-home offers four briefs and a set of must-include deliverables. This project leads with Brief A (document-to-insight, front-end-strong) and Brief B (multi-model trust), and borrows the discipline of C (CI + eval gate) and D (hardening & observability) so the result is defensible on all four axes without over-scoping.

Brief A pairs a bounded analysis agent (a fixed step-by-step pipeline, not a free-planning agent) with a strong front-end — both are focuses, not one at the other's expense. The agent drives the analysis, not a hidden pipeline; the front-end is the polished, responsive, accessible surface the brief explicitly asks for. Each step invokes analysis tools only (parse, extract, validate, compare, arbitrate) — plain retrieval of a filing is not an agent action — and the loop hard-stops before any external action. That constraint is the product: an agent whose steps and tool use are fully inspectable and that never auto-acts.

Must-include items (tracked to completion in progress.md):

• A Git repo with a 10-minute README (../README.md).
• A test/eval harness — for the platform angle, the pipeline must go red on a bad change.
• A one-paragraph AI-use disclosure (what Claude/Codex drafted vs. hand-verified) — see the end of this doc.
• For the front-end brief, a deployed link or screenshots of the key states.

Product

HK IPO Verify — browse recent HKEX IPO filings (Application Proof, PHIP, or prospectus), pick one, and get back a validated structured factsheet, a claim-by-claim source-backed summary, a multi-model agreement/disagreement panel, and a hard "never auto-act" boundary.

Scope it as "filing → verified investor-ready factsheet", not "autonomous IPO analysis." HKEX publishes Application Proofs, PHIPs, and prospectuses publicly, so the app sources filings directly from HKEX — users never upload a file — yet we treat the documents as sensitive financial PII to demonstrate privacy discipline (Brief D).

The product signature is trust: a bounded analysis agent whose steps and tool calls are fully inspectable, presented through a polished front-end and held inside hard boundaries — validation, recoverability, and evidence presentation — not autonomy.

Architecture

A small, explicit, inspectable agent behind a polished front-end — not a broad autonomous platform. The core is a bounded, code-driven loop running a fixed pipeline (the agent works step by step but does not choose its own actions) whose tools are the analysis stages below and whose only exit is a hard interrupt before any external action. Retrieving a filing is ordinary I/O, not an agent action — the agent starts once the document is in hand. We are Cloudflare-first: prefer a free-tier Cloudflare service wherever it fits the purpose, and fall back to AWS only where Cloudflare has no good free-tier alternative.

• Frontend: a polished, responsive, accessible SPA on Cloudflare Pages (Bun runtime, free tier, Git deploys, instant rollback) — a first-class focus (Brief A is front-end-strong). Framework is an open call (React, Preact, Solid, or similar). It is the analyst workstation: the factsheet panels, the agreement panel, citations that deep-link to the source, the agent's reasoning/tool trace, and the persistent "never auto-act" banner.
• Filing ingestion: sourced from HKEX, not uploaded. A Cron-triggered Cloudflare Worker refreshes a catalog of recent filings (Application Proof, PHIP, prospectus) into Cloudflare KV on HK trading days; each filing PDF is fetched from HKEX on first view and cached in Cloudflare R2 (free tier, zero egress). A CatalogSource interface keeps the HKEX data source swappable.
• Backend: Python + FastAPI, packaged with uv, on AWS Lambda behind API Gateway (HTTP API), hosting the analysis agent and its tools plus the plain filing-retrieval endpoints (fetch the cached PDF from R2). This stays on AWS because Workers Python is beta and a poor host for FastAPI + PDF parsing — the documented "no good Cloudflare alternative" case.
• Models: a vendor-neutral LLM gateway — one base URL fanning out to Claude, Gemini, and GPT, each with its own API key (the client speaks the OpenAI-compatible /v1/chat/completions path for all vendors). The gateway provider is swappable; keep its name out of code and config. A single gateway client keeps the multi-model fan-out single-path (OpenAI-compatible chat-completions is the working assumption — confirm against the provider's docs before wiring).
• Orchestration: a bounded, code-driven agent running a fixed pipeline whose action space is a fixed, whitelisted set of analysis tools — parse, extract, validate, compare, arbitrate — and nothing external. Retrieving the PDF is not one of these tools; it happens before the loop, and the agent works over an already-in-hand document. The loop is bounded (step cap, deterministic tool schemas) and every reasoning step + tool call is recorded as a trace the UI can replay. Its only terminal action is an interrupt that surfaces the draft factsheet and stops — it never executes a transaction. LangGraph is optional polish for durable execution and the interrupt checkpoint; adopt it only if it earns its place, don't make it load-bearing.
• Infra: Terraform through HCP Terraform; AWS + Cloudflare in one root module at infra/. AWS hosts the pipeline compute and supporting resources (Lambda, API Gateway, CloudWatch) — not models, and not storage/scheduling/static hosting (those are Cloudflare R2/KV/Cron/Pages).
• Observability & security: CloudWatch, request/correlation IDs in structured logs, least-privilege IAM, secrets in env/secrets store (never the repo).

Agent loop

Retrieval comes first and is plain I/O — no agent involved: select a filing from the catalog, and fetch + cache its PDF (R2, on first view). The analysis agent then works over the in-hand document and calls analysis tools; each tool is one deterministic stage. A typical trajectory:

select filing (from catalog) → fetch + cache PDF (R2, first view)   ← plain retrieval
   ┌─ agent starts here (document in hand) ─────────────────────────────────┐
   │ [agent] parse (deterministic) → chunk → [agent] extract (model A + B)   │
   │      → [agent] validate (schema + citations) → [agent] compare/agreement│
   │      → [agent] arbitrate disputed claims (frontier model, disputes only)│
   │      → render payload → INTERRUPT before any external action            │
   └─────────────────────────────────────────────────────────────────────────┘

The arrows inside the box are the agent's typical trajectory, not a hardcoded pipeline: the loop re-plans on tool failure (e.g. one model times out → degrade to single-model output) and stops when validation passes or the step cap is hit. Tools are a fixed whitelist — the agent cannot invent an external action, and it never reaches back out to fetch anything. A scheduled Worker keeps the catalog current; the PDF is downloaded from HKEX and cached in R2 the first time a filing is opened, then reused. Parse text deterministically first; pass only relevant chunks to models. The engineering signal is schema discipline, citation discipline, and an inspectable agent trace — not opaque whole-PDF prompting.

Output schema

Every claim carries page-level citations (and ideally a quoted span), so the UI can deep-link to the source. Indicative shape:

{
  "document": { "title": "...", "issuer": "...", "document_type": "application_proof|phip|prospectus", "currency": "HKD|USD|RMB", "source_url": "...", "sha256": "..." },
  "ipo_facts": { "offer_size": {}, "price_range": {}, "use_of_proceeds": [], "cornerstone_investors": [], "listing_timetable": {} },
  "financial_highlights": [ { "metric": "revenue", "period": "2023", "value": 0, "unit": "HKD", "citations": [] } ],
  "risks": [ { "claim": "...", "severity": "high|medium|low", "citations": [] } ],
  "trust": { "agreement_score": 0.0, "disputed_claims": [] },
  "agent_trace": [ { "step": 0, "thought": "...", "tool": "parse", "args": {}, "observation": "...", "model": null } ]
}

The agent_trace records every reason→act→observe step so the UI can show how the factsheet was produced — which tool ran, which model, what it observed — and so the eval harness can assert the loop stayed inside its whitelist and hit the terminal interrupt.

Frontend — analyst workstation, not a chatbot

A first-class focus (Brief A is front-end-strong): a polished, responsive, accessible SPA — framework is an open call (React, Preact, Solid, or similar). Four responsive panels: Overview (issuer, type, offer range, proceeds), Financial highlights (normalized metrics with units/periods), Key risks (investor-readable, each clickable to source), and a Trust panel (agreement/disagreement, confidence, missing/ambiguous fields, model availability) that also replays the agent trace — each reason→act→observe step, so the reasoning and tool use are auditable, not a black box.

Persistent boundary banner: "Draft analysis from a public filing. Verify the cited source before you act. This system never executes transactions or submits orders."

Three visible degradation states (Brief A reliability guards):

• Full — both models succeeded, citations validated.
• Partial — one model timed out/failed; show the validated single-model output with a degraded badge.
• Rejected — schema or citation validation failed; explain and render no unsupported claims.

Multi-model trust (Brief B)

Explicit fan-out, not hidden routing: two models extract independently, code normalizes both into the same schema, a comparison stage computes exact agreement, semantic disagreement, citation overlap, and unsupported-claim flags. The UI shows agreed claims, disputed claims, claims one model omitted, and mismatched citations. A third arbiter model runs only on disputed claims — smart and cost-controlled, and the UI still shows the original disagreement.

Pick the cheapest viable model per tier from the gateway: a low-cost extractor, a mid-tier verifier, and a frontier arbiter reserved for disputes. Never make a frontier model the default extractor.

Eval harness (Brief C)

A small gold dataset of hand-labeled HKEX examples; measure disagreement-detection recall, false-positive disagreement rate, citation correctness, schema validity, and agent-loop safety (the trace stays inside the tool whitelist and always ends at the interrupt — never an external action). Plain pytest + JSON fixtures is enough if the outputs are inspectable. The CI eval gate must fail on a bad change (e.g. break a prompt so citations drop, or a tool change that lets the loop skip the interrupt) — that proves AI-output regression control, which is what Brief C tests.

CI, release, rollback

GitHub Actions as the visible spine: lint → test → eval → infra (fmt/validate) → deploy. Frontend deploys to Cloudflare Pages on green; backend to a Lambda version + alias. One-command rollback re-points the Lambda alias and reverts the Pages deployment.

Hardening & observability (Brief D)

Treat the data as sensitive financial PII even though fixtures are public: least-privilege IAM, secrets in a secrets store, no real client data in tests, correlation IDs end to end, no raw document text in logs (log document hash, page count, job ID, field-level validation outcomes only), and one meaningful alert tied to trust degradation (rate of unsupported_claim_rejected / citation_validation_failed) rather than generic CPU/memory.

Repo layout

The repo layout is in ../README.md. Target package set:

apps/api/            FastAPI + analysis agent + tools + Pydantic contracts (AWS Lambda)
apps/web/            polished analyst-workstation SPA (Cloudflare Pages)
apps/catalog/        Cron Worker: HKEX catalog refresh into KV
packages/contracts/  shared JSON schema / TS types
fixtures/            public HKEX sample docs + hand-labeled gold cases
tests/               unit + integration + evals (incl. agent-trace safety)

Feature cut

• Must: HKEX catalog of recent filings, lazy fetch + R2 cache of the chosen filing, a bounded analysis agent over whitelisted analysis tools, deterministic parser, validated schema, agent summary with citations + replayable trace, two-model comparison, disagreement panel, terminal "never auto-act" interrupt, CI failing on eval regression, rollback, structured logs + one alarm.
• Nice: arbiter on disputes, keyboard-first citation drawer, README screenshots, hosted demo, synthetic "one model failed" state, LangGraph for durable agent execution.
• Skip: user file upload, live HTML scraping of HKEX (the JSON servlet suffices), managed RAG, memory, long-lived chat, complex auth, OCR for scanned PDFs, and any agent action that reaches outside the tool whitelist (no external or autonomous actions of any kind).

AI-use disclosure

This project was built with Claude Code as a pair-programmer under test-driven, trunk-based discipline. Claude drafted boilerplate, test scaffolding, the initial component and tool structure, and doc prose; I directed the architecture and hand-verified everything that carries risk — the agent's boundary invariants (whitelist + terminal interrupt), all threat-modeling and Terraform/IAM permissions, the model-routing and fallback logic, the schema and citation-validation rules, and every README claim. The trust boundaries the product is about are exactly the parts I did not delegate.