Skip to content

Commit 1d984af

Browse files
tcconnallyclaude
andcommitted
fix(capacity): single pooled connection per hot path; no-op-skip decay writes; paginate history; concurrency CI gate (closes #397, closes #399, closes #403)
#397 — nested self.conn() draws collapsed the server at >= pool-size concurrency (32 clients / pool 16: 174 req/s, 30s brownouts, failed writes). apply_recall_side_effects, find_near_duplicate, and store_embedding now have *_with_conn variants used by every caller that already holds a pooled connection (fts5_search recall side-effects, remember's dedup scan and auto-embed store, embed_entity), so each request uses exactly one pooled connection. Callers with no live connection keep the self-drawing wrappers. r2d2's connection_timeout is now tunable via MIMIR_POOL_TIMEOUT_MS (default unchanged, 30s). #399 — decay_tick rewrote every non-archived row every tick (412MB WAL per tick on a 45MB DB @100k). The per-row UPDATE is now skipped when the recomputed score is within DECAY_WRITE_EPSILON (1e-4) of the stored value and no archive transition applies, and the layer-demotion UPDATE's WHERE mirrors its CASE so only rows whose layer actually changes are written. entities_updated now counts rows actually written (entities_checked still reports rows evaluated); JSON shape unchanged. #403 — mimir_history now takes optional limit (default 20, newest first, 0-1000) and offset; the response's total reports the FULL trail size (plus returned/limit/offset) so agents can page a hot key instead of pulling a 10-15MB response into context. #404 (concurrency-gate half) — new .github/workflows/concurrency-gate.yml: lean release build, pool_load_test_http_transport pinned at 2x oversubscription (CLIENTS=32 / POOL=16, 60s wall budget via new MIMIR_LOADTEST_MAX_WALL_SECS) plus the four named concurrency hammer tests. The perf-gate half of #404 remains open. Measured (Windows, lean release, 32 clients / pool 16 / 5600 requests): before 174 req/s, p99 436ms, max 31.2s, 16 errors, 786/800 writes persisted; after 4242 req/s, p99 11.5ms, max 1.19s, 0 errors, 800/800 persisted. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1 parent e15e0e5 commit 1d984af

5 files changed

Lines changed: 399 additions & 29 deletions

File tree

Lines changed: 60 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,60 @@
1+
name: Concurrency gate
2+
3+
# Guards the #397 fix (nested pooled-connection draws) at 2x pool
4+
# oversubscription: 32 concurrent clients against a 16-connection pool, driven
5+
# through the REAL HTTP transport. Before the fix this exact configuration
6+
# browned out (174 req/s, 30s max latency, failed writes) and 4x hard-wedged;
7+
# after it, the run must complete with zero errors, every write persisted, and
8+
# the wall well inside the 60s budget. The four named hammer tests then pin
9+
# the per-subsystem concurrency invariants (audited-writer history, link/edge
10+
# durability, follow increments, pool starvation) so a failure names the
11+
# subsystem that regressed. Lean build (--no-default-features): the pool
12+
# behavior under test is independent of the embedding stack, and the lean
13+
# build keeps the gate fast. (#404)
14+
15+
on:
16+
push:
17+
branches: [main, master]
18+
paths:
19+
- "src/**"
20+
- "Cargo.toml"
21+
- "Cargo.lock"
22+
- ".github/workflows/concurrency-gate.yml"
23+
pull_request:
24+
branches: [main, master]
25+
paths:
26+
- "src/**"
27+
- "Cargo.toml"
28+
- "Cargo.lock"
29+
- ".github/workflows/concurrency-gate.yml"
30+
31+
env:
32+
CARGO_TERM_COLOR: always
33+
34+
jobs:
35+
concurrency-gate:
36+
runs-on: ubuntu-latest
37+
steps:
38+
- uses: actions/checkout@v4
39+
- name: Install Rust
40+
uses: dtolnay/rust-toolchain@stable
41+
- name: Build (lean, release)
42+
run: cargo build --release --no-default-features --tests
43+
- name: Pool load test at 2x oversubscription (32 clients / pool 16)
44+
env:
45+
MIMIR_LOADTEST_CLIENTS: "32"
46+
MIMIR_LOADTEST_WRITES: "25"
47+
MIMIR_LOADTEST_READS: "75"
48+
MIMIR_POOL_MAX_SIZE: "16"
49+
MIMIR_BUSY_TIMEOUT_MS: "5000"
50+
MIMIR_LOADTEST_MAX_WALL_SECS: "60"
51+
run: |
52+
cargo test --release --no-default-features \
53+
pool_load_test_http_transport -- --ignored --nocapture
54+
- name: Concurrency hammer tests
55+
run: |
56+
cargo test --release --no-default-features -- \
57+
concurrent_audited_writers_serialize_without_corrupting_history \
58+
concurrent_links_and_remembers_do_not_lose_edges \
59+
concurrent_follows_do_not_lose_increments \
60+
many_concurrent_linkers_do_not_starve_the_pool

0 commit comments

Comments
 (0)