-
-
Notifications
You must be signed in to change notification settings - Fork 16
Expand file tree
/
Copy pathllms.txt
More file actions
311 lines (255 loc) · 14.3 KB
/
Copy pathllms.txt
File metadata and controls
311 lines (255 loc) · 14.3 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
Remnic (formerly Engram)
Persistent, private, local-first memory for AI agents.
Open source, MIT licensed. https://github.com/joshuaswarren/remnic
All data stays on the user's machine as plain markdown files with YAML
frontmatter. No cloud services, no external databases, no subscriptions.
Works with OpenClaw, Claude Code, Codex CLI, Hermes Agent, Replit, Cursor,
and any MCP-compatible client. Tell one agent a preference — every agent
remembers it.
ARCHITECTURE
TypeScript ESM monorepo. pnpm workspaces. Node 22+.
Packages:
@remnic/core Engine (orchestrator, storage, search, extraction,
graph, trust zones, LCM). Framework-agnostic.
@remnic/cli Standalone CLI binary, 20+ commands.
@remnic/server HTTP + MCP server with multi-token auth, daemon mode.
@remnic/bench Latency benchmarks with CI regression gates.
@remnic/plugin-openclaw OpenClaw adapter (embedded or delegate mode).
@remnic/plugin-codex Codex CLI adapter (hooks + MCP + memory extension).
@remnic/hermes-provider TypeScript HTTP client for remote Remnic instances.
@remnic/import-supermemory Optional Supermemory JSON importer for
`remnic import --adapter supermemory`.
remnic-hermes Python PyPI package, MemoryProvider for Hermes Agent.
OpenClaw adapter note:
The OpenClaw plugin registers `before_prompt_build` with Remnic's
`initGateTimeoutMs` as the per-hook timeout on SDKs that support hook
options. The same config bounds Remnic's internal cold-start init gate for
recall and day summaries. Default: 30000ms; accepted range: 1000-120000ms.
Current OpenClaw compatibility target: reviewed npm build 2026.6.11-beta.1,
with 2026.6.5-beta.3, 2026.6.5-beta.5, 2026.6.5-beta.6, 2026.6.7-beta.1,
2026.6.8-beta.1, 2026.6.8-beta.2, 2026.6.9-beta.1, 2026.6.10-beta.1,
2026.6.10-beta.2, and 2026.6.11-beta.1 retained in the explicit prerelease
peer range and package metadata floor >=2026.4.1. As of June 26, 2026 the
active 60-day floor is April 27, 2026; keep the
more-permissive 2026.4.1 floor unless an upstream breaking change makes it
impossible. Keep openclaw.install.minHostVersion AND openclaw.compat.pluginApi
as a single >=x.y.z comparator (never a || list): OpenClaw's installer
AND-evaluates whitespace tokens and normalizes the host prerelease suffix, so
the floor already admits stable and prerelease hosts (issue #1450). Put
explicit reviewed prerelease versions ONLY in peerDependencies.openclaw,
which is resolved by npm/node-semver (supports ||, but excludes prereleases
from a bare >= range). Keep the manifest on current native
fields: kind="memory", contracts.tools, commandAliases, activation,
setup.requiresRuntime=false, setup.providers[].envVars for optional
OPENAI_API_KEY, and providerAuthChoices. Also keep providerAuthEnvVars and
supports mirrored for older OpenClaw hosts. Do not restore unsupported
top-level securityDisclosure; document privacy/provider behavior in docs
instead.
Three-phase flow:
1. Recall Before each agent turn, inject relevant memories into context.
2. Buffer After each turn, accumulate content until a trigger fires.
3. Extract Periodically, extract structured memories via an LLM call.
Storage: plain markdown + YAML frontmatter. Categories include fact,
decision, preference, correction, relationship, principle, commitment,
moment, skill, rule, entity, and more.
Search: hybrid BM25 + vector + reranking via QMD. Six backend options:
QMD (default), Orama (embedded JS), LanceDB (embedded native Arrow),
Meilisearch (server), Remote (HTTP REST), Noop (extraction only).
Current QMD target: @tobilu/qmd 2.5.3. Remnic probes qmd --version and gates
newer flags; QMD 2.5.3+ uses --format json for query/search subprocess output,
while older supported QMD installs keep legacy --json.
KEY FEATURES
Core (enabled by default):
- Automatic extraction and recall injection
- Entity tracking and relationship graph
- Memory lifecycle (active -> validated -> stale -> archived)
- Episode/note model (time-specific events vs stable beliefs)
- Importance-gated extraction (trivial content never hits disk)
- Inline source attribution (provenance tags in fact body)
Lossless Context Management (LCM):
Proactive session archive into local SQLite + hierarchical summary DAG.
When context gets compacted, LCM injects compressed history back into
recall. Three-level summarization with guaranteed convergence. Full-text
search over archived messages. Zero data loss.
Parallel Specialized Retrieval:
Three agents run in parallel (latency = max, not sum):
DirectFact Scans entity filenames for keyword overlap (<5ms).
Contextual Existing hybrid BM25+vector search.
Temporal Reads date index with recency decay scoring (<10ms).
Zero additional LLM cost. Graceful per-agent degradation.
Semantic Consolidation:
Finds clusters of similar memories via token overlap, synthesizes a
canonical version via LLM, archives originals. Conservative threshold
(80%+), corrections and commitments excluded by default.
Trust Zones:
Quarantine -> working -> trusted tiers with promotion rules and
poisoning defense. Provenance tracking and corroboration scoring.
Extraction Judge (issue #376):
LLM-as-judge post-extraction durability filter. Shadow mode available
for calibration before enabling gating.
Semantic Chunking (issue #368):
Topic-boundary detection via sentence embeddings and cosine similarity
with smoothing. Alternative to recursive chunking.
Page Versioning (issue #371):
Snapshot-based history for memory files. Every overwrite saves a
numbered snapshot. List, inspect, diff, and revert via CLI or API.
OAI-mem-citation Blocks (issue #379):
Recall responses emit <oai-mem-citation> blocks matching the Codex
citation format for memory attribution and usage tracking.
Memory Extension Publisher Contract (issue #381):
Pluggable contract for installing host-specific instruction files into
any AI agent host's extension directory. Generalizes Codex extension
pattern.
Memory Extension Discovery (issue #382):
Third-party memory extensions provide structured instructions that
influence consolidation, auto-discovered from extension directories.
MECE Taxonomy (issue #366):
Mutually Exclusive, Collectively Exhaustive knowledge directory with
resolver decision tree for deterministic memory categorization.
Enrichment Pipeline (issue #365):
Importance-tiered API spend for entity enrichment from external sources
with a provider registry.
Binary Lifecycle (issue #367):
Three-stage pipeline (mirror, redirect, clean) for binary files in the
memory directory with configurable storage backends.
Codex Marketplace (issue #418):
Install via: codex marketplace add joshuaswarren/remnic
Memory OS (progressive opt-in):
Memory boxes, graph recall, compounding (weekly synthesis), shared
context (cross-agent), identity continuity, hot/cold tiering, native
knowledge search, behavior loop tuning.
Advanced (opt-in):
Objective-state recall, causal trajectories, harmonic retrieval,
verified recall, semantic rule promotion, creation memory, commitment
lifecycle.
BUILD / TEST / DEVELOP
Build: pnpm run build
Test: npm run test (node:test with tsx, 672+ tests)
Typecheck: pnpm run check-types
Quality gate: npm run preflight:quick
Eval suite: npm run eval:run
ACCESS LAYER
HTTP API:
Most routes under /engram/v1/... (v1.x compatibility window).
Bearer-token auth, binds to loopback by default.
Routes: health, recall, recall/explain, memories (CRUD), entities,
observe, lcm/search, lcm/status, trust-zones/status, trust-zones/records,
trust-zones/promote, review-queue, maintenance, suggestions,
review-disposition.
Exception: POST /v1/citations/observed (no /engram prefix).
MCP tools:
Exposed via stdio and HTTP transports. 14+ tools covering recall, store,
entity lookup, memory search, LCM expansion, trust zone inspection, and
observation.
Standalone CLI (20+ commands):
init, status, query, doctor, config, daemon, tree, onboard, curate,
review, sync, dedup, connectors, space, benchmark, versions, taxonomy,
enrich, binary.
Operator UI:
http://127.0.0.1:4318/engram/ui/
DOCUMENTATION
README.md Full installation guide, feature list, architecture
CLAUDE.md Project instructions and privacy policy
AGENTS.md Agent guide with 43 review prevention patterns
docs/config-reference.md 90+ configuration properties
docs/api.md HTTP, MCP, and CLI reference
docs/architecture/ Per-feature architecture docs
docs/guides/ Standalone, Codex, Claude Code, local LLM, etc.
docs/guides/openclaw-engram-to-remnic.md
Legacy OpenClaw Engram -> Remnic migration:
package rename, config keys, backups, patches
docs/getting-started.md Quick start
docs/operations.md Operations guide
docs/search-backends.md Search backend comparison
docs/namespaces.md Namespace isolation
docs/import-export.md Data portability
CONFIGURATION
Config resolved from: REMNIC_CONFIG_PATH env var -> ./remnic.config.json ->
~/.config/remnic/config.json. Presets: conservative, balanced, research-max,
local-llm-heavy. 90+ properties. LLM routing: OpenAI API, local LLM (Ollama,
LM Studio), or gateway model chain with multi-provider fallback.
QUICK START (STANDALONE)
npm install -g @remnic/cli
remnic init
export OPENAI_API_KEY=sk-...
export REMNIC_AUTH_TOKEN=$(openssl rand -hex 32)
remnic daemon start
remnic status
remnic query "hello" --explain
QUICK START (OPENCLAW)
openclaw plugins install clawhub:@remnic/plugin-openclaw
remnic openclaw install
launchctl kickstart -k gui/$(id -u)/ai.openclaw.gateway
remnic doctor
OpenClaw 2026.5.30-beta.1 may resolve bare plugin package names through npm
first during the launch cutover. Remnic is published on ClawHub as
@remnic/plugin-openclaw; use explicit clawhub:@remnic/plugin-openclaw for
deterministic installs, and explicit npm:@remnic/plugin-openclaw@<version>
for npm-only fallback or rollback versions.
ClawHub publish note: publish @remnic/plugin-openclaw from the built
ClawPack/npm tarball, not from the raw GitHub source folder. Run
pnpm --filter @remnic/plugin-openclaw build, pnpm run verify:openclaw-clawpack,
clawhub package pack packages/plugin-openclaw, then clawhub package publish
the generated remnic-plugin-openclaw-<version>.tgz with source metadata.
OPENCLAW PLUGIN COMPATIBILITY
Policy: Remnic supports OpenClaw releases from at least the previous 60 days.
Recalculate the floor from the current date before changing package metadata.
As of June 3, 2026, the active floor is April 4, 2026. Remnic intentionally
keeps OpenClaw 2026.4.1 as a more-permissive package metadata floor.
Current manifest shape for OpenClaw 2026.5.31+:
kind: "memory" selects the exclusive memory slot.
setup.providers[].envVars advertises optional plugin-mode OPENAI_API_KEY
discovery. providerAuthChoices keeps the custom onboarding/CLI metadata.
providerAuthEnvVars must remain mirrored for older pre-runtime auth probes.
June 2, 2026 sweep: issues #1258, #1259, #1266, and #1271 were reviewed
against OpenClaw tags through v2026.6.2-alpha.2. The manifest, SDK overview,
and SDK entrypoint docs remained compatible; Remnic's SDK snapshot still
matched every reviewed tag. New additive OpenClaw surfaces are consumed only
through gates: host embedding providers via
hostEmbeddingProviderEnabled, inbound transcript capture via
openclawMessageReceivedCaptureEnabled, bounded reply/thread metadata via
openclawReplyMetadataCaptureEnabled, opt-in quoted-reply extraction context
via openclawReplyMetadataExtractionHintsEnabled, and canonical channel
envelope prefixes via openclawChannelEnvelopeCleaningEnabled. Never make
these static imports or required paths; older OpenClaw hosts must keep using
Remnic's existing embedding, agent_end, and legacy envelope-cleaning paths.
v2026.6.2-alpha.1 and v2026.6.2-alpha.2 are source tags but were not GitHub
release pages or npm packages during that sweep.
June 3, 2026 sweep: issue #1312 was reviewed against OpenClaw source tags
v2026.6.1-beta.3 and v2026.6.3-alpha.1. v2026.6.3-alpha.1 exists upstream,
but it was not a GitHub release page at review time. The diff from
v2026.6.2-alpha.2 touched OpenClaw policy/workboard docs and
provider/exec/runtime internals; the v2026.6.1-beta.3 diff from
v2026.6.1-beta.2 did not touch plugin docs or src/plugins. Optional surfaces
confirmed in this window are resolve_exec_env, CLI backend
ownsNativeCompaction, and ProviderNormalizeTransportContext.modelId. Remnic
does not register an exec-environment hook, does not provide a CLI backend,
and does not implement provider transport normalization, so no runtime
behavior change is needed. Keep these surfaces optional and gated if a future
feature uses them.
supports must remain for OpenClaw 2026.4 / early 2026.5 slot and lifecycle
routing. Top-level securityDisclosure is not a current native manifest field
and must stay omitted.
contracts.tools: required by OpenClaw 2026.5+ plugin descriptor planning and
tool ownership validation. Add every Remnic-owned registerTool name here when
adding OpenClaw tools, including conditional LCM aliases.
Source of truth: packages/plugin-openclaw/openclaw.plugin.json.
Synced copies: openclaw.plugin.json and
packages/shim-openclaw-engram/openclaw.plugin.json.
Guard test: tests/openclaw-plugin-runtime-surfaces.test.ts.
LEGACY OPENCLAW ENGRAM MIGRATION
Use this for @joshuaswarren/openclaw-engram 9.2.x -> @remnic/plugin-openclaw.
Canonical OpenClaw plugin id: openclaw-remnic.
Legacy plugin id: openclaw-engram.
Do not use remnic-workspace as an OpenClaw plugin id; it is the npm workspace
root package name, not the OpenClaw runtime id.
remnic openclaw migrate-engram --yes
This backs up openclaw.json, backs up the legacy openclaw-engram extension
directory, installs @remnic/plugin-openclaw into openclaw-remnic, writes
plugins.entries["openclaw-remnic"], sets plugins.slots.memory to
"openclaw-remnic", and preserves the memoryDir.
CONNECTING OTHER AGENTS
remnic connectors install claude-code
remnic connectors install codex-cli
remnic connectors install replit
pip install remnic-hermes