Skip to content

v0.41.11.0.md

Latest commit

 

History

History
61 lines (47 loc) · 3.46 KB

v0.41.11.0.md

File metadata and controls

61 lines (47 loc) · 3.46 KB
Raw

v0.41.11.0 migration — conversation retrieval upgrade

The v0.41.11.0 upgrade ships:

  1. Schema-pack additions: conversation (temporal, extractable) and atom (annotation, NOT extractable) promoted into gbrain-base.
  2. New CLI command gbrain extract-conversation-facts for bulk fact extraction over long-form conversation pages.
  3. Opt-in cycle phase conversation_facts_backfill (default OFF) for autopilot-driven backlog drain.
  4. New doctor check conversation_facts_backlog (3-state; SKIPPED when feature disabled so no opt-out noise).
  5. Schema migration v94: partial index on facts(source_id, source_session) WHERE source LIKE 'cli:extract-conversation-facts%' so the doctor query stays fast on big brains.

gbrain upgrade + gbrain apply-migrations --yes handles the binary + schema. The cycle phase and doctor check require no agent action.

Agent action — when to run the backfill

When the user mentions:

  • "search isn't finding things in my chat history"
  • "my iMessage / Slack import isn't searchable"
  • "the page exists but search misses the answer"
  • "long conversation pages aren't returning what I want"

…then this command is the right tool. Don't run it speculatively — it costs real money against the user's Anthropic budget.

Recommended flow

  1. Estimate cost first. For each source the user wants to backfill:

    gbrain sources audit <source-id> --json | jq '.facts_backfill_estimate'

    Returns {pages, est_segments, est_cost_usd, types}. Show the user the cost estimate before recommending the run.

  2. Run a dry-run to preview segmentation:

    gbrain extract-conversation-facts --dry-run --slug <slug-of-one-conversation-page>

    Confirms the segmenter parses the page correctly without spending any tokens.

  3. Run with explicit cost cap. Default cap is $5. Recommend matching the audit estimate (rounded up):

    gbrain extract-conversation-facts --background --max-cost-usd <cap>
gbrain jobs follow <printed-job-id>

    Use --background so the user can keep working. The Minion job is resumable — if it hits the budget cap mid-run, re-running with a higher cap continues from where it left off.

  4. Verify with doctor:

    gbrain doctor --json | jq '.checks[] | select(.name == "conversation_facts_backlog")'

    Should show OK with backlog: 0 after a successful complete run.

  5. (Optional) Enable autopilot drain if the user has steady conversation ingest:

    gbrain config set cycle.conversation_facts_backfill.enabled true

    The cycle phase will then drain new conversation pages each tick under bounded per-source ($1/cycle) and brain-wide ($5/cycle) budgets.

Caveats

  • Pages over 25MB body are skipped (memory cap). Surface in doctor details; streaming for huge pages is a v0.42+ TODO.
  • If facts.extraction_enabled is false, the command refuses. Pass --override-disabled only when the user explicitly opted out and now wants this one-time run.
  • Extracted facts use source = 'cli:extract-conversation-facts'. To remove them in bulk (rare), the only path today is raw SQL: DELETE FROM facts WHERE source LIKE 'cli:extract-conversation-facts%'. A gbrain forget --where bulk flag is a v0.42+ TODO.
  • The recall-quality eval (under test/eval/conversation-extraction-quality.eval.ts — added in this wave) is env-gated on ANTHROPIC_API_KEY. Run nightly or on-demand for quality verification; the hermetic wiring tests run in CI by default.