feat: v3.2.0 - cumulative metrics, interactive setup, and architect prompts

sl4m3 · sl4m3 · commit e97c2f5924b8 · 2026-03-02T05:07:46.000+03:00
diff --git a/README.md b/README.md
@@ -1,6 +1,6 @@
 # LedgerMind
 
-**v3.1.3** · Autonomous Memory Management System for AI Agents
+**v3.2.0** · Autonomous Memory Management System for AI Agents
 
 > *LedgerMind is not a memory store — it is a living knowledge core that thinks, heals itself, and evolves without human intervention.*
 
diff --git a/docs/changelogs/index.md b/docs/changelogs/index.md
@@ -4,6 +4,17 @@ All notable changes to the LedgerMind project since version 2.0.0.
 
 ---
 
+## [v3.2.0] - 2026-03-02
+
+- **Cumulative Metrics:** Implemented a zero-footprint mathematical model for 
+  incremental lifecycle updates.
+- **Interactive Setup:** Added model selection and automated 60s validation 
+  during project initialization.
+- **Architect Standards:** Upgraded documentation prompts to professional AI 
+  Architect and Principal Engineer levels.
+- **Integrity Stability:** Fixed a critical core bug in UnboundLocalError 
+  during file validation.
+
 ## [v3.1.3] - 2026-03-02
 
 - **Real-Time Synchronization:** Instant history synchronization for Claude 
diff --git a/docs/changelogs/latest.md b/docs/changelogs/latest.md
@@ -1,18 +1,18 @@
-# Latest Release: v3.1.3
+# Latest Release: v3.2.0
 
 ## March 2, 2026
 
-LedgerMind v3.1.3 focus on real-time synchronization, hypothesis unification,
-and improved technical log recording.
+LedgerMind v3.2.0 focus on architectural scalability through cumulative metrics
+and enhanced interactive configuration.
 
 ### Highlights
 
-- **Real-Time Synchronization (Stop Hook):** Instant history synchronization
-  for Claude Code via the new `Stop` hook, preventing duplicate entries.
-- **Hypothesis Unification:** Consistent evolutionary path for both
-  procedural and behavioral hypotheses.
-- **Increased Scaling:** Memory event length limit increased to 500KB to
-  support large technical logs and complex code snippets.
-- **Secure Hook Recording:** Robust interaction recording via stdin pipes.
+- **Cumulative Metrics:** Mathematical model for incremental stability and
+  frequency updates without storing historical event IDs.
+- **Interactive Model Selection:** Specify and validate custom models (e.g.,
+  `gemini-2.0-flash`) directly during project initialization.
+- **Architect Prompts:** New high-standard documentation templates for
+  knowledge enrichment.
+- **Bug Fixes:** Resolved critical UnboundLocalError in core integrity checks.
 
-[View full v3.1.3 changelog](./v3.1.3.md)
+[View full v3.2.0 changelog](./v3.2.0.md)
diff --git a/docs/changelogs/v3.2.0.md b/docs/changelogs/v3.2.0.md
@@ -0,0 +1,44 @@
+# Changelog v3.2.0 (March 2, 2026)
+
+LedgerMind v3.2.0 introduces a major architectural shift to a fully cumulative
+metrics model, interactive model management, and professional documentation
+standards.
+
+## Highlights
+
+### 📉 Cumulative Metrics Model
+- **Zero-Footprint Storage:** Implemented a mathematical model for incremental
+  metric updates. The system no longer needs to store long lists of event IDs
+  in Markdown metadata, keeping files clean and efficient.
+- **Weighted Stability:** Stability and frequency scores are now updated
+  proportionally using a weighted average of new evidence deltas against
+  accumulated history.
+
+### 🛠️ Interactive Model Management
+- **Model Selection & Validation:** Added an interactive step during `init` to
+  specify custom models (e.g., `gemini-2.0-flash`) for 'rich' mode.
+- **Robust Validation:** Implemented automatic 60-second validation calls to
+  verify model availability and connectivity before finalizing configuration.
+
+### 📝 Professional Documentation Standards
+- **Architect-Level Prompts:** Completely redesigned LLM prompts for procedural
+  and behavioral enrichment. The system now acts as a "Principal Engineer" or
+  "Software Architect" to produce industry-standard documentation.
+- **Incremental Summarization:** LLM can now integrate new logs into existing
+  manual rationales, building knowledge step-by-step.
+
+### 🛡️ Core Stability
+- **Integrity Fix:** Resolved a critical `UnboundLocalError` in
+  `IntegrityChecker` that affected incremental file updates.
+- **Test Suite Synchronization:** Updated all lifecycle tests to support the
+  new cumulative logic and improved the specialized `lg.py` validator.
+
+## What's Changed
+
+- feat(core): implement fully cumulative lifecycle metrics model by @sl4m3
+- feat(cli): add interactive model selection and validation for rich mode by @sl4m3
+- feat(reasoning): upgrade LLM prompts to professional architect standards by @sl4m3
+- fix(core): resolve UnboundLocalError in IntegrityChecker by @sl4m3
+- fix(tests): synchronize lifecycle tests with incremental signal logic by @sl4m3
+
+**Full changelog:** [v3.1.3...v3.2.0](https://github.com/sl4m3/ledgermind/compare/v3.1.3...v3.2.0)
diff --git a/pyproject.toml b/pyproject.toml
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "ledgermind"
-version = "3.1.3"
+version = "3.2.0"
 
 description = "Monolithic Autonomous Agent Memory Management System"
 readme = "README.md"
diff --git a/src/ledgermind/VERSION b/src/ledgermind/VERSION
@@ -1 +1 @@
-3.1.3
+3.2.0
diff --git a/src/ledgermind/core/reasoning/lifecycle.py b/src/ledgermind/core/reasoning/lifecycle.py
@@ -27,46 +27,66 @@ def calculate_temporal_signals(
         now: datetime
     ) -> DecisionStream:
         """
-        Updates temporal metrics: reinforcement_density, stability, coverage, lifetime_days.
+        Updates temporal metrics incrementally based on a NEW delta of reinforcement_dates.
+        This allows for clearing processed evidence IDs while maintaining accurate history.
         """
         if not reinforcement_dates:
+            # Still update lifetime metrics even with no new evidence
             stream.lifetime_days = (now - stream.first_seen).total_seconds() / 86400.0
-            stream.reinforcement_density = 0.0
-            stream.stability_score = 0.0
             stream.coverage = stream.lifetime_days / self.observation_window_days
             return stream
 
         sorted_dates = sorted(reinforcement_dates)
-        if stream.first_seen > sorted_dates[0]:
+        
+        # 1. Update temporal boundaries
+        if stream.frequency == 0:
+            # Initial state: trust the new dates completely
             stream.first_seen = sorted_dates[0]
+            stream.last_seen = sorted_dates[-1]
+        else:
+            # Incremental state: expand boundaries
+            if stream.first_seen > sorted_dates[0]:
+                stream.first_seen = sorted_dates[0]
             
-        stream.last_seen = sorted_dates[-1]
+            if stream.last_seen < sorted_dates[-1]:
+                stream.last_seen = sorted_dates[-1]
         
         lifetime_secs = (stream.last_seen - stream.first_seen).total_seconds()
         stream.lifetime_days = lifetime_secs / 86400.0
-        
-        # Avoid division by zero
         safe_lifetime = max(stream.lifetime_days, 0.01)
-        
-        stream.frequency = len(sorted_dates)
+
+        # 2. Update Frequency (Cumulative)
+        old_freq = stream.frequency
+        delta_freq = len(sorted_dates)
+        new_freq = old_freq + delta_freq
+        stream.frequency = new_freq
+
+        # 3. Update Reinforcement Density
         stream.reinforcement_density = stream.frequency / safe_lifetime
         
+        # 4. Update Coverage
         stream.coverage = stream.lifetime_days / self.observation_window_days
 
-        if len(sorted_dates) > 2:
+        # 5. Update Stability Score (Weighted Average)
+        # Calculate stability for the NEW delta only
+        delta_stability = 0.0
+        if delta_freq > 2:
             intervals = [(sorted_dates[i] - sorted_dates[i-1]).total_seconds() / 86400.0 
                          for i in range(1, len(sorted_dates))]
             if len(intervals) > 1:
                 var = statistics.variance(intervals)
-                # Lower variance = higher stability. Use an inverse function or normalize.
-                # Assuming variance of 0 is perfect stability (score 1.0).
-                stream.stability_score = max(0.0, 1.0 - (var / (safe_lifetime + 1.0)))
-            else:
-                stream.stability_score = 0.0
-        elif len(sorted_dates) == 2:
-            stream.stability_score = 0.3 # Moderate stability signal for 2 events
+                # Lower variance = higher stability
+                delta_stability = max(0.0, 1.0 - (var / (safe_lifetime + 1.0)))
+        elif delta_freq == 2:
+            delta_stability = 0.3
+        else:
+            delta_stability = 0.0 # Single events don't provide stability signal
+
+        # Merge with existing stability using weighted average
+        if old_freq > 0:
+            stream.stability_score = (stream.stability_score * old_freq + delta_stability * delta_freq) / new_freq
         else:
-            stream.stability_score = 0.0
+            stream.stability_score = delta_stability
 
         return stream
 
diff --git a/src/ledgermind/core/reasoning/llm_enrichment.py b/src/ledgermind/core/reasoning/llm_enrichment.py
@@ -102,9 +102,13 @@ def process_batch(self, memory: Any):
                 if status == "completed":
                     updates = {
                         "rationale": enriched.rationale,
-                        "enrichment_status": "completed"
+                        "enrichment_status": "completed",
+                        "evidence_event_ids": []
                     }
                     
+                    if is_procedural:
+                        updates["procedural"] = None
+                    
                     with memory.semantic.transaction():
                         memory.semantic.update_decision(
                             filename=fid,
@@ -132,9 +136,9 @@ def enrich_proposal(self, proposal: Any, cluster_logs: Optional[str] = None) ->
         # Format input text for LLM
         if is_procedural:
             raw_text = self._format_procedural_text(proposal)
-            prompt = self._build_procedural_prompt(raw_text)
+            prompt = self._build_procedural_prompt(raw_text, existing_rationale=proposal.rationale)
         else:
-            prompt = self._build_behavioral_prompt(proposal.target, cluster_logs)
+            prompt = self._build_behavioral_prompt(proposal.target, cluster_logs, existing_rationale=proposal.rationale)
         
         try:
             enriched_rationale = None
@@ -161,22 +165,91 @@ def _format_procedural_text(self, proposal: Any) -> str:
             text += f"Step {i+1}: {step.action}\nRationale: {step.rationale}\n\n"
         return text
 
-    def _build_procedural_prompt(self, raw_text: str) -> str:
+    def _build_procedural_prompt(self, raw_text: str, existing_rationale: Optional[str] = None) -> str:
+        context_part = ""
+        if existing_rationale and "*Original Data:*" in existing_rationale:
+            # Extract only the LLM-generated part
+            clean_context = existing_rationale.split("*Original Data:*")[0].strip()
+            if clean_context and "Overall Objective" in clean_context:
+                context_part = (
+                    "EXISTING GUIDE CONTEXT:\n"
+                    "--------------------------\n"
+                    f"{clean_context}\n"
+                    "--------------------------\n\n"
+                    "The sequence below contains NEW execution steps that have occurred SINCE the existing guide was written.\n"
+                )
+
         return (
-            "You are an expert AI architect summarizing system behavior.\n"
-            "Analyze the following machine-generated execution logs and convert them "
-            "into a single, coherent, human-readable summary of the actions taken and the rationale behind them. "
-            "Focus on the 'why' and 'what'. Keep it concise but highly informative.\n\n"
-            f"Logs:\n{raw_text}\n\nSummary:"
+            "You are an expert Software Architect and Technical Documentation Specialist with 15+ years of experience. \n"
+            "You excel at turning raw execution traces, command sequences, API calls, tool invocations, and successful action chains into clear, professional, human-readable procedural guides.\n\n"
+            "TASK:\n"
+            "Analyze the following procedural log/sequence of successful actions and transform it into a coherent, easy-to-follow step-by-step instruction guide for a developer or engineer.\n\n"
+            f"{context_part}"
+            "REQUIREMENTS:\n"
+            "- Identify the overall purpose and final outcome.\n"
+            "- Break everything into logical, numbered steps.\n"
+            "- For each step clearly state:\n"
+            "  • WHAT is done\n"
+            "  • WHY it is done (technical or business reason)\n"
+            "- Show dependencies and flow between steps.\n"
+            "- Remove noise, abstract technical details where possible, but keep important parameters and outcomes.\n"
+            "- Use concise, professional language.\n"
+            "- IMPORTANT: If previous context is provided, INTEGRATE the new steps into the existing guide. Re-order, group, or refine the steps to maintain a logical and coherent flow. Update the 'Overall Objective' if needed.\n\n"
+            "OUTPUT FORMAT (strictly follow this structure):\n\n"
+            "1. **Overall Objective**\n"
+            "   One-sentence summary of what this procedure achieves.\n\n"
+            "2. **Prerequisites** (if any)\n\n"
+            "3. **Step-by-Step Procedure**\n"
+            "   1. [Step description]\n"
+            "      • What: ...\n"
+            "      • Why: ...\n"
+            "      • Key details/parameters: ...\n\n"
+            "4. **Key Insights & Recommendations**\n"
+            "   Any architectural observations, potential improvements, or best practices.\n\n"
+            "Now analyze this sequence of NEW actions:\n\n"
+            f"{raw_text}\n\nUpdated Procedural Guide:"
         )
 
-    def _build_behavioral_prompt(self, target: str, logs: str) -> str:
+    def _build_behavioral_prompt(self, target: str, logs: str, existing_rationale: Optional[str] = None) -> str:
+        context_part = ""
+        if existing_rationale and "*Original Data:*" in existing_rationale:
+            # Extract only the LLM-generated part, ignoring the raw data dump
+            clean_context = existing_rationale.split("*Original Data:*")[0].strip()
+            if clean_context and "Detected Technical Shift" in clean_context:
+                context_part = (
+                    "PREVIOUS SUMMARY CONTEXT:\n"
+                    "--------------------------\n"
+                    f"{clean_context}\n"
+                    "--------------------------\n\n"
+                    "The logs below are NEW activities that have occurred SINCE the previous summary.\n"
+                )
+
         return (
-            "You are an expert AI architect analyzing behavioral patterns in a codebase.\n"
-            f"I have detected a high density of activity related to the component '{target}'.\n"
-            "Based on the following execution logs, please provide a high-level summary of what the developer/agent "
-            "has been working on in this area. Identify the main theme, goal, or technical shift reflected in these actions.\n\n"
-            f"Logs:\n{logs[:5000]}\n\nSummary of Behavioral Pattern:"
+            "You are a Principal Engineer and Codebase Archaeologist specializing in reverse-engineering developer intent from activity clusters.\n\n"
+            "TASK:\n"
+            "You are given a dense cluster of NEW events, logs, commits, or activities concentrated around a specific component, module, file, or target. \n"
+            f"Analyze this high-density activity zone for '{target}' and extract the underlying technical shift, main goal, and behavioral pattern of the developer/team.\n\n"
+            f"{context_part}"
+            "REQUIREMENTS:\n"
+            "- Identify the primary technical evolution or refactoring direction.\n"
+            "- Uncover the core objective (what problem is being solved or opportunity pursued).\n"
+            "- Detect recurring patterns in the work style or architectural decisions.\n"
+            "- Base every conclusion on concrete evidence from the provided cluster.\n"
+            "- IMPORTANT: If previous context is provided, INTEGRATE the new findings into a single, updated, and coherent summary. Refine the technical shift and goal if the new logs provide more clarity.\n\n"
+            "OUTPUT FORMAT (strictly follow this structure):\n\n"
+            "1. **Detected Technical Shift**\n"
+            "   One-sentence summary of the main change happening.\n\n"
+            "2. **Primary Goal / Intent**\n"
+            "   What the developer is ultimately trying to achieve.\n\n"
+            "3. **Key Patterns Observed**\n"
+            "   • Pattern 1: description + evidence\n"
+            "   • Pattern 2: ...\n\n"
+            "4. **Architectural & Strategic Implications**\n"
+            "   How this activity affects the larger codebase and possible next steps or risks.\n\n"
+            "5. **Component Evolution Summary**\n"
+            "   Brief before → after picture (if inferable).\n\n"
+            "Now analyze this NEW activity cluster:\n\n"
+            f"{logs[:5000]}\n\nUpdated Summary of Behavioral Pattern:"
         )
 
     def _call_cli_model(self, prompt: str) -> Optional[str]:
diff --git a/src/ledgermind/core/reasoning/reflection.py b/src/ledgermind/core/reasoning/reflection.py
@@ -1,4 +1,3 @@
-
 import logging
 import os
 import json
@@ -173,8 +172,11 @@ def _process_stream(self, fid: str, data: Dict[str, Any], stats: Dict[str, Any],
             event_map = {e['id']: e for e in events}
         
         reinforcement_dates = []
-        all_evidence = set(stream.evidence_event_ids + stats['all_ids'])
-        stream.evidence_event_ids = list(all_evidence)
+        
+        # Update evidence list (queue for LLM enrichment)
+        # Only add IDs that aren't already in the pending list
+        new_ids = [eid for eid in stats['all_ids'] if eid not in stream.evidence_event_ids]
+        stream.evidence_event_ids.extend(new_ids)
         
         # Link procedural instructions if provided
         if procedural_links:
@@ -194,7 +196,8 @@ def _process_stream(self, fid: str, data: Dict[str, Any], stats: Dict[str, Any],
         # Define kinds that constitute a real 'use' or 'reinforcement' of knowledge (PR #30)
         REINFORCEMENT_KINDS = {KIND_RESULT, KIND_ERROR, "call", "task", "prompt", "intervention"}
 
-        for eid in stream.evidence_event_ids:
+        # Extract dates ONLY for the NEW delta from the current cluster
+        for eid in stats['all_ids']:
             if eid in event_map:
                 ev = event_map[eid]
                 if ev['kind'] in REINFORCEMENT_KINDS:
@@ -204,9 +207,7 @@ def _process_stream(self, fid: str, data: Dict[str, Any], stats: Dict[str, Any],
                     except (ValueError, TypeError):
                         pass
         
-        if not reinforcement_dates:
-            reinforcement_dates = [now]
-            
+        # calculate_temporal_signals now treats reinforcement_dates as a DELTA to be merged
         stream = self.lifecycle.calculate_temporal_signals(stream, reinforcement_dates, now)
         
         # Issue #14: Reactivate dormant stream if new events arrived in this cluster
@@ -247,7 +248,7 @@ def _create_pattern_stream(self, target: str, stats: Dict[str, Any], now: dateti
             evidence_event_ids=stats['all_ids'],
             first_seen=now,
             last_seen=now,
-            frequency=len(stats['all_ids'])
+            frequency=0 # Will be updated by calculate_temporal_signals
         )
 
         if arbitration_mode != "lite":
@@ -282,7 +283,7 @@ def _create_pattern_stream(self, target: str, stats: Dict[str, Any], now: dateti
         if not reinforcement_dates:
             reinforcement_dates = [now]
         
-        # Calculate initial temporal signals
+        # Calculate initial temporal signals (cumulative)
         stream = self.lifecycle.calculate_temporal_signals(stream, reinforcement_dates, now)
         stream = self.lifecycle.promote_stream(stream)
         
diff --git a/src/ledgermind/core/stores/semantic_store/integrity.py b/src/ledgermind/core/stores/semantic_store/integrity.py
@@ -53,8 +53,8 @@ def validate(repo_path: str, force: bool = False, fid: str = None, data: Dict[st
         Supports incremental validation if fid is provided.
         'data' can be provided to avoid re-parsing the file.
         """
+        current_hash = IntegrityChecker._get_state_hash(repo_path)
         if not fid:
-            current_hash = IntegrityChecker._get_state_hash(repo_path)
             if not force and IntegrityChecker._state_cache.get(repo_path) == current_hash:
                 return
         
diff --git a/tests/core/reasoning/test_lifecycle.py b/tests/core/reasoning/test_lifecycle.py
diff --git a/tests/lg.py b/tests/lg.py
