All notable changes to vstack are documented in this file. The format
follows Keep a Changelog and the
project adheres to Semantic Versioning from
1.0.0 onward. During the 0.x series, minor bumps may include
breaking changes (see API stability promise in vstack/__init__.py).
One-command CI onboarding.
vstack-config init-ci— scaffolds a ready-to-run GitHub Actions workflow (.github/workflows/vstack-agent-quality.yml) that wires up the full gate: diagnose a trace on each PR,fail-on: high, upload SARIF to code scanning, and post a sticky PR comment. Pins the Action to the installed vstack version.--out/--force/--dry-runlikeinstall-skills. Takes a repo from "installed" to "gating in CI" in one command.
- All tests pass. Additive only; no breaking changes.
vstack-import now reads the two dominant LLM-observability exports.
vstack.ingest.from_langsmith_runs+vstack-import --format langsmith— convert a LangSmith run (with nestedchild_runs— a run tree) or a flat list of runs into anAgentTrace. Mapsrun_type(llm→message, tool→tool_call, chain→thought, retriever→observation, …); usesinputs/outputs/erroras step content; infers goal/outcome from the root run. The CLI accepts a single run, a list, or{"runs": [...]}.vstack-import --format phoenix+ extendedfrom_otel_spans— reads OpenInference attributes (openinference.span.kind,input.value/output.value,llm.input_messages/llm.output_messages), so Arize Phoenix span exports import cleanly (the genericotelformat now covers them too).
- All tests pass. Additive only; no breaking changes.
The Action's CI UX is now complete: gate · annotate · comment.
- GitHub Action
commentinput + output — setcomment: vstack-comment.mdand the Action writes the findings summary (Markdown) to that path; the example workflow posts it as a sticky PR comment viamarocchino/sticky-pull-request-comment, so findings land in the PR conversation (not just the job summary). Complements the existing build gate (fail-on) and code-scanning annotations (sarif). +1 test.
- All tests pass. Opt-in; no breaking changes.
Bring-your-own-traces: import the logs you already have into a vstack trace.
vstack.ingest— trace importers that build the canonicalAgentTracefrom real data:from_chat_messages(messages, …)— OpenAI/Anthropic chat logs ({role, content, tool_calls}); maps system/user/assistant/tool to trace steps, infers goal (first user message) + outcome (last assistant message), flattens multimodal content.from_otel_spans(spans, …)— OpenTelemetry spans (best-effort; readsgen_ai.*attributes as a dict or OTLP{key,value}list, orders by start time, classifies GenAI vs other spans).
vstack-importCLI —vstack-import --format {messages,otel} input.jsonemits anAgentTraceJSON, designed to pipe straight intovstack-diagnose --trace -. (CLI surface: 59 → 60.)_ingestwired into CI (pytest + ruff +mypy --strict).
- All tests pass. Additive only; no breaking changes.
GitHub Marketplace launch of the Action.
action.ymldescription shortened to 96 chars to meet the GitHub Marketplace limit (descriptions must be < 125 characters), so the Action can be published/listed in the Marketplace. No behavior change.
The CI ratchet — gate only on new findings — plus a vdiff correctness fix.
vstack-diagnose --baseline <report.json>— compare the current run against a saved diagnose report and, with--fail-on, gate only on findings that are new relative to the baseline. This is the standard ratchet: a gate won't fail on pre-existing, already-accepted findings, and it tightens as you re-baseline. Prints avs baseline: N new, M pre-existingsummary to stderr.
vstack.vdiff.diff_reportscrashed on real reports. It assumedper_patternwas a name-keyed dict, but actualDiagnoseReports (and their JSON) carryper_patternas a list.diff_reports(and therefore thevstack-vdiffCLI) raisedTypeErroron genuinevstack-diagnoseoutput; it now normalizes both shapes. Regression test added.
_gate_exit_codenow takes a list of severities (so the gate can score either all findings or only the new ones).
- All tests pass.
--baselineis opt-in; no breaking changes.
SARIF output — vstack findings now flow into GitHub code scanning (Security tab + PR annotations).
vstack-diagnose --sarif— emit a SARIF 2.1.0 report instead of Markdown/JSON. Each pattern becomes a SARIF rule; each finding a result whoselevelmaps from severity (critical/high → error, medium/moderate → warning, low/trace → note). Results carry the trace path as their artifact location.vstack.diagnose.to_sarif(report, *, trace_uri=..., version=...)— the public renderer behind the flag.- GitHub Action
sarifinput + output — setsarif: vstack.sarifto also write a SARIF report (single diagnose run) and upload it withgithub/codeql-action/upload-sarif; the example workflow shows the upload. Findings then appear in the Security tab and as inline PR annotations.
- All tests pass (7 SARIF + 1 Action-SARIF test added). No breaking changes —
--sarifis opt-in.
Citability + positioning: a JOSS software-paper draft and a "why vstack" comparison.
- JOSS paper draft —
paper/paper.md(+paper/paper.bib), a Journal of Open Source Software submission draft with a Summary and Statement of Need, grounded in vstack's real OB citations (Lewin, Edmondson, Lencioni, Tversky & Kahneman, Staw, Latané et al., and others). The ORCID is a placeholder flagged for the author to fill before submission. - "vstack vs. eval/observability" comparison table in the README, clarifying that vstack answers why a run failed and what the fix is — complementary to eval (pass/fail) and observability (telemetry).
- "Citing vstack" README section pointing at
CITATIONS.md,CITATION.cff, and the paper.
CITATION.cffbrought current: version0.7.0→0.47.0, "12 invocation surfaces" → 13, and the adapter list updated to all ten frameworks.
- Documentation/metadata only — no code or API changes. All tests pass.
A new CI-gate primitive on the core CLI + launch-grade README demo.
vstack-diagnose --fail-on <severity>— the diagnose CLI is now a self-contained CI gate: it exits3when any finding is at or above the given severity (none…critical), so any CI (not just the GitHub Action) can gate on agent quality with one command. Omit the flag to never fail on findings (current behavior). The GitHub Action and the standalone CLI now share the same severity semantics.- Launch demo assets —
docs/assets/demo.svg(an inline, GitHub- rendered terminal card built from realvstack-hello --offlineoutput) anddocs/assets/demo.cast(an asciinema v2 recording of the session). The README now opens with the demo.
- README hero: added the terminal demo + a one-line "30 seconds, no API key" call to action up top.
- All tests pass (new
--fail-ongate + helper covered). No breaking changes —--fail-ondefaults to off.
A new invocation surface: vstack now ships as a GitHub Action to gate agent quality in CI. Thirteen invocation surfaces.
- GitHub Action (
uses: valani9/vstack@v0.45.0) — a composite action that installs vstack, runsvstack-diagnoseon a trace, and fails the build when any finding is at or above afail-onseverity threshold (none…critical). Inputs:trace,fail-on,mode,recipe,client,shape,version,python-version. Outputs:max-severity,findings-count,report. Writes a findings table to the job summary.client: none(default) runs the deterministic analyzers with no API key — a free smoke gate; set a provider for full LLM-backed findings.action.yml(repo root) +.github/action/gate.py(gate logic, reuses vstack's ownseverity_rank) + 9 unit tests, wired into CI.- Example consumer workflow:
examples/github-action/agent-quality-gate.yml.
- All tests pass. No breaking changes.
Two more shell CLIs + a vstack-doctor consistency fix. CLI surface:
57 → 59.
vstack-synth— generate synthetic agent traces for testing, demos, and onboarding:list(show templates/scenarios) +gen --template <name> [--count N] [--seed S] [--out file]. Pure generation, no LLM.vstack-vdiff—--before a.json --after b.json [--json]diffs twoDiagnoseReports and renders the delta (findings added / removed / changed, severity shifts).
vstack-doctoroptional-extra checks now cover every framework adapter. The list was missingcrewai,smolagents,google-adk, andstrands; all four are added. The probe also now suppresses an optional dep's own import-time warnings and degrades to a WARNING (instead of crashing) if an installed framework fails to import.
- All tests pass. No breaking changes.
Two more framework adapters — vstack's 34 patterns now bind natively to Google ADK and AWS Strands. Twelve adapters total.
vstack.adapters.adk—as_adk_tools()returns one Google ADKFunctionToolper pattern (FunctionTool(func=...)), ready forAgent(tools=...). Framework-gated; install withpip install 'valanistack[adk]'.vstack.adapters.strands—as_strands_tools()returns one AWS Strands@tool-decorated callable per pattern (aDecoratedFunctionToolwith the righttool_name+ spec). Framework-gated; install withpip install 'valanistack[strands]'.- Both verified against the real frameworks (google-adk, strands-agents).
They install as their own extras (ADK's google-cloud deps are heavy), so
they're not bundled into
[adapters]/[all].
- The adapter test suite's
_has_modulehelper now usesimportlib.util.find_spec(availability check) instead of importing the module — importing a heavy framework like google-adk at collection time could fail for reasons unrelated to whether it's installed.
- All tests pass; the ADK/Strands tests skip when their framework isn't installed (CI). No breaking changes.
Two more framework adapters — vstack's 34 patterns now bind natively to
Hugging Face smolagents and Agno, two ecosystems flagged as wanted
in CONTRIBUTING.md.
vstack.adapters.smolagents—as_smolagents_tools()returns one native smolagentsToolsubclass per pattern (name/description/inputs/output_type/forward). Each takes atraceobject plus an optionalmodeand returns the detection. Framework-gated; install withpip install 'valanistack[smolagents]'.vstack.adapters.agno—as_agno_tools()returns one plain Python callable per pattern, each carrying the tool name + anArgs:docstring so Agno introspects it. No Agno import required to use them (framework- free, like the AutoGen adapter) — pass straight toAgent(tools=...).- Both verified against the real frameworks (smolagents 1.26, Agno);
smolagentsadded to the[adapters]bundle. Adapter count: 8 → 10.
- All tests pass. The smolagents test skips when the framework isn't installed (CI), matching the other framework-gated adapters. No breaking changes.
Four analytical shell CLIs that complete the "track findings → debug regressions → visualize" loop. CLI surface: 53 → 57.
vstack-findings-db— persistent SQLite finding store at the shell:add(ingest a diagnose report),list(filter by pattern / severity / agent / run / time),stats(count_by).--db PATH(default~/.vstack/findings.db). The "track findings over time" tool.vstack-trace-diff—--before a.json --after b.json [--json]structurally diffs twoAgentTraces and renders the delta (added / removed / changed steps, outcome + success change, REGRESSION/RECOVERY verdict). The "what changed between a passing and failing run" debugger.vstack-heatmap—--reports reports.json [--by {dimension,trace}] [--format {ascii,html}]renders a pattern severity heatmap.vstack-timeline—--reports reports.json [--format {markdown,sparkline}] [--bucket ...]renders a chronological view of findings.- Each CLI is a thin, fully-typed (
mypy --strict), ruff-clean, unit-tested shell over existing public module functions — no new business logic. The modules already shipped in the wheel, so the CLIs are available topip installusers immediately.
- All tests pass. No breaking changes.
Tooling correctness: vstack-doctor and the shell completions now know
about every CLI instead of a stale subset.
vstack-doctoronly validated 10 of the 53 CLIs. Its CLI list was hardcoded and stale (it didn't even checkvstack-doctoritself, nor any pattern or workflow CLI). It now enumerates the installedconsole_scriptsentry points dynamically, so it validates all 53 CLIs and never goes stale as new ones ship (falls back to the core list only when distribution metadata is unreadable). Added tests for the discovery, the metadata-missing fallback, and thatrun_all_checkscovers every discovered CLI.- Shell completions covered only ~14 of 53 CLIs. The bash, zsh, and
fish completions now cover all 53: a shared completion for the 33
per-pattern CLIs (
analyze/batch/replay/validate/schema/playbooks/compose) plus the 8 workflow CLIs (vstack-diagnose,vstack-recipes,vstack-scorecard,vstack-dashboard,vstack-trace-zoo,vstack-redaction,vstack-export,vstack-aggregate) with their real subcommands/flags.bash -nandzsh -nclean.
- All 3,173 tests pass (1 skipped: crewai not installed). No API changes.
Three new shell CLIs that expose previously library-only modules — the
Trace → Sanitize → Diagnose → AAR → Apply cycle is now fully driveable
from the command line.
vstack-redaction— scrub PII / secrets from a trace at the shell (the cycle's "Sanitize" step).--trace trace.json [--out scrubbed.json]scrubs anAgentTraceJSON (or stdin),--textscrubs raw text,--list-patternsprints the built-inDEFAULT_PATTERNS; per-pattern match counts go to stderr.vstack-export— convert a diagnose report / findings JSON to another format:--format {csv,json,markdown,github,jira}, output to--outor stdout.vstack-aggregate— roll up multiple diagnose reports:--reports reports.json [--top N] [--json]prints top patterns, top agents, and the severity distribution.- Each CLI is a thin shell over the existing public module functions (no new
business logic), fully typed (
mypy --strict), ruff-clean, and unit-tested. CLI surface count: 50 → 53.
- All tests pass; the three modules already shipped in the wheel, so the new
CLIs are available to
pip installusers immediately. No breaking changes.
Type-safety completion: every feature module is now mypy --strict clean
and gated in CI.
- All 36 feature modules now pass
mypy --strict. v0.37.0 added 12 of them to the CI mypy matrix; this release fixes the type annotations in the remaining 24 (_diagnose,_dashboard,_scorecard,_budget,_trace_zoo,_replay,_vcache,_otel,_compose,_vdiff,_vbench,_markers,_synth,_veval,_recipes_dsl,_heatmap,_policy,_budgeter,_tracer,_export,_findings_db,_timeline,_eval_gates,_intervention_tracker) and adds them to the CI matrix. 107 type errors resolved with proper annotations — zero new# type: ignoreand no runtime behavior changes. - The CI
Typecheck (mypy)job now covers the full surface: 15 original surface dirs + all 36 feature modules + 34 pattern libs.
- All 3,131 tests pass (1 skipped: crewai not installed). Typing-only changes; public API and runtime behavior unchanged.
Packaging fix + two new Claude Code skills + a documentation truth-up across the README, the skill docs, and the docs site to reflect the thirty-six library modules and fifty CLIs shipped since v0.7.0.
/vstack-diagnoseskill — the fast path. Throws one trace at the cross-patternvstack_diagnoserunner and returns a ranked report in a single call; maps a one-phrase failure description ("stuck in a loop", "agents arguing") to a named recipe. Routes deeper (/vstack-post-incident,/vstack-audit-crew, …) on what it surfaces./vstack-scorecardskill — turns diagnose reports into a per-agent / per-fleet letter-graded scorecard, renders it (text / markdown / HTML), and compares two scorecards over time to catch regressions. CI-gateable viavstack-scorecard compare --fail-on-regression.- Both new skills are wired into the
/vstackrouter's routing table.
vstack-config install-skillsnow works forpip installusers. The_skills/bundle is force-included into the wheel asvstack/_skills/, and_resolve_skills_sourcechecks the wheel-bundled location first (falling back to the two repo-checkout layouts). Previously the command only resolved against a git checkout, so it failed for anyone who installed from PyPI. The resolver is now unit-tested across all three resolution paths.- CI ruff failure — removed two unused imports in
_otel/tests/test_exporter.pyby replacing a mis-wired test (it claimed to exercisesetup_otelbut called_otel_check_available) with realsetup_otel/is_enabledcoverage that does not pollute global OTel state. - CI mypy failure from upstream numpy churn — numpy 2.5.0 (a transitive
dep of some framework extras) ships inline stubs using the 3.12
type X = ...statement, which mypy can't parse underpython_version 3.11("Type statement is only supported in Python 3.12 and greater"). Added a[[tool.mypy.overrides]]entry that skips following numpy, since vstack has no direct numpy dependency. - Docs build (strict mkdocs) had been failing since v0.36.0 — the
LangChain integration page linked
../composition.md#langchain, but the file lives atconcepts/composition.md. Fixed the link so the hosted docs site can deploy again.
- README brought current to v0.37.0: corrected the stale Docker tags
(
0.7.0→0.37.0), the test count (2,150→3,131), and the Claude Code skills surface (now installed viavstack-config install-skills, 9 skills). Added a Feature modules section cataloguing all thirty-sixvstack.<name>library modules grouped by role, plus a CLI example for thevstack-diagnose/vstack-recipesrunner. - Docs site — synced version markers, test counts, and the skills
catalog (added
/vstack-diagnose+/vstack-scorecard, removed a reference to a non-existent/vstack-aarskill). - CI coverage closed the feature-module gap. The CI pytest, ruff
(check + format), and mypy jobs only covered the original ~15 surface
dirs + the 34 pattern libs — none of the 36 feature modules added
across v0.20–0.36. They are now wired in: all 36 feature-module
dirs run under pytest + ruff (≈1,016 previously-uncovered tests now
gate every push), and the 12 already-
--strict-clean modules (_calibrate,_streaming,_signing,_trace_diff,_cost_sim,_findings_router,_alerting,_redaction,_health,_priority_queue,_snippet,_aggregate) are added to the mypy matrix. The remaining 24 feature modules need type-annotation cleanup before they can joinmypy --strict(tracked inci.yml). - Test count: 3,127 → 3,131 (+4: three resolver tests, one OTel test).
- All 3,131 tests pass (1 skipped: crewai not installed).
- Public API surface strictly expanded. No breaking changes.
Five more feature modules: redaction + health + priority_queue + snippet + aggregate. Thirty-four feature modules total since v0.23.0.
vstack.redaction— PII / secret scrubbing for traces.RedactionPattern(regex + replacement); built-inDEFAULT_PATTERNScovering email, US/CA phone, SSN, credit card, AWS access/secret keys, sk-prefixed API keys, Bearer tokens, JWT, IPv4, URLs with user:pass@ credentials.Redactortracks per- pattern match counts;scrub_trace()returns a redacted copy of the trace with goal/outcome/step content scrubbed (original unmutated).vstack.health— composite health checks.Checkprotocol +CallableCheckadapter.HealthReportaggregates with HEALTHY / DEGRADED / UNHEALTHY status (critical UNHEALTHY → UNHEALTHY; non-critical UNHEALTHY → DEGRADED).HealthMonitorwithtick()(interval-aware) +force_tick()for scheduler- driven probes.vstack.priority_queue— finding priority queue with aging boost.FindingPriorityQueueheap-backed; score = severity_weight (high=100/med=10/low=1) × confidence_multiplier (0.5-1.0) + age_boost (aging_multiplier × hours_elapsed) + manual_boost.boost()/remove_pattern()/snapshot()helpers. Aging prevents low-severity starvation.vstack.snippet— minimal trace excerpts.find_relevant_steps()uses token-overlap (lowercase, stopword-filtered, ≥3 chars) between finding text and step content.extract_snippet()pulls N context steps around relevant steps with omission counts.render_snippet()produces markdown with→markers on relevant steps + elision for long content.vstack.aggregate— cross-report aggregation.aggregate_reports()returnsAggregateSummarywith per-pattern stats (high/med/low- severity_score), severity_counts, agent_counts.
top_n_patterns()/top_n_agents()(optionally severity- filtered),severity_distribution(),cooccurrence_matrix()(pairs that appear in the same report).
- severity_score), severity_counts, agent_counts.
- Test count: 3,014 → 3,127 (+113 from the five new modules).
- All 3,127 tests pass (1 skipped: crewai not installed).
- Public API surface strictly expanded.
Three more feature modules: alerting + eval_gates + intervention_tracker. Twenty-nine feature modules total since v0.23.0.
vstack.alerting— multi-channel alert dispatcher. PluggableAlertSinkprotocol with built-inSlackSink(webhook payload- severity emoji),
PagerDutySink(Events API v2 + vstack→PD severity mapping + dedup_key),WebhookSink(generic JSON POST),EmailSink(RFC 5322 envelope),ConsoleSink(stdout),NullSink(testing).AlertDispatcherfans out per-alert with per-sink retries, severity floors that short-circuit retries, and contextvar-based dry-run mode for tests. All sinks accept a user-suppliedsendercallable so network I/O stays out of the module's dependencies.
- severity emoji),
vstack.eval_gates— CI gate primitives.Gateprotocol; built-in gates:SeverityCountGate(max findings by severity),F1Gate/PrecisionGate/RecallGate(eval metric floors),BaselineComparisonGate(max new high findings + max overall-score drop),CustomGate(user predicate).GateSet.check()evaluates in order, captures gate exceptions as failures, returnsGateResultwithpassedflag +exit_code()for CI scripts.vstack.intervention_tracker— track applied interventions + outcomes.InterventionTrackerrecords remediations with the finding snapshot that triggered them.InterventionOutcomeenum: PENDING / RESOLVED / PARTIAL / NO_EFFECT / REGRESSED / ROLLED_BACK. Query by pattern / applied_by / pending / closed.effectiveness_score()ranks interventions by efficacy (resolved = 1.0, partial = 0.5, no_effect/regressed = 0);rank_patterns_by_effectiveness()surfaces which patterns benefit most from intervention.
- Test count: 2,931 → 3,014 (+83 from the three new modules).
- All 3,014 tests pass (1 skipped: crewai not installed).
- Public API surface strictly expanded.
Three more feature modules: timeline + cost_sim + findings_router. Twenty-six feature modules total since v0.23.0.
vstack.timeline— chronological event view + ASCII sparklines.build_timeline()buckets findings by minute / hour / day / week (UTC).Timelineexposes peak_bucket, velocity per period/hour/day, quiet_bucket count.Bucketstacks severities.render_sparkline()produces unicode block-char sparklines;render_markdown_timeline()produces a tabular report.vstack.cost_sim— what-if cost scenarios for production budget planning.Scenarioparameterizes traces/day, sample rate, mode (quick/standard/forensic), pattern list, and optional failure_upgrade (10% assumed forensic re-run). Built- in per-pattern pricing for all 34 shipped patterns; override viacustom_pricing.simulate()projects daily / monthly / annual cost.compare_scenarios()produces side-by-side markdown table.vstack.findings_router— smart routing of findings to owners / teams with channel metadata (jira_project / github_label / pagerduty_service / slack_channel).OwnerRoutematches on pattern / severity (with floor) / confidence range.FindingsRouterevaluates routes in order, first match wins, falls back to default_owner.Assignmentcarries channel routing for downstream issue creation.
- Test count: 2,863 → 2,931 (+68 from the three new modules).
- All 2,931 tests pass (1 skipped: crewai not installed).
- Public API surface strictly expanded.
Three more feature modules: export + findings_db + trace_diff. Twenty-three feature modules total since v0.23.0.
vstack.export— export findings to CSV / JSON / Markdown / Jira ADF / GitHub PR comment.export_csv(),export_json(),export_markdown()(severity-grouped),export_jira()(Atlassian Document Format),export_github_comment()(PR-comment-sized with 65k char truncation, severity emoji 🔴🟡🔵).vstack.findings_db— SQLite-backed finding store.FindingsDBwith auto-schema, store_finding/store_report, find() with rich filters (pattern / severity / agent_id / run_id / timestamp range, multi-value match, limit), count_by() aggregation, indexes on all filter columns. Context-manager safe with del cleanup. In-memory or persistent (file) backing.vstack.trace_diff— structural comparison of two AgentTraces. Myers-style alignment via SequenceMatcher.diff_traces()returnsTraceDeltawith added / removed / changed / unchanged step diffs, goal_changed / outcome_changed / success_flipped flags,is_regression()/is_recovery()helpers, summary() + to_markdown()- to_dict().
- Test count: 2,799 → 2,863 (+64 from the three new modules).
- All 2,863 tests pass (1 skipped: crewai not installed).
- Public API surface strictly expanded.
Two more feature modules: budgeter + tracer. Twenty feature modules total since v0.23.0.
vstack.budgeter— cost projection + multi-tier budget alerts. Companion tovstack.budgetfor monthly-budget forecasting.Budgetertracks spend events, projects monthly total at current burn rate, computes days-until-limit, and fires once-each multi-tier alerts at 50/75/90/100% thresholds (configurable).forecast_burn()helper for one-shot projections.vstack.tracer— inline trace recorder for live agents. FluentTracerbuilder withthought()/tool_call()/observation()/message()/decision()methods, chainable return-self pattern. Context manager support: exits mark failure on exception.finalize()produces anAgentTrace.
- Test count: 2,761 → 2,799 (+38 from the two new modules).
- All 2,799 tests pass (1 skipped: crewai not installed).
- Public API surface strictly expanded.
Two more feature modules: heatmap + policy. Eighteen feature modules total since v0.23.0.
vstack.heatmap— ASCII + HTML heatmap visualization.build_pattern_dimension_grid()produces pattern × dimension grids;build_pattern_trace_grid()produces pattern × trace grids.render_heatmap()outputs ASCII art with intensity ramp;render_heatmap_html()produces a color-graded HTML table (HSL green→red gradient).vstack.policy— declarative finding-action policies.Rulematches findings on pattern / severity / confidence (exact / list / range).Actionsubclasses:ActionLog,ActionAlert,ActionPage,ActionEscalate,ActionIgnore,ActionCustom.Policyevaluates rules in order; first match wins; default_action for unmatched findings.evaluate_policy()produces orderedDecisionlist. Range matches use{"min": 0.7, "max": 1.0}.
- Test count: 2,719 → 2,761 (+42 from the two new modules).
- All 2,761 tests pass (1 skipped: crewai not installed).
- Public API surface strictly expanded.
Two more feature modules: veval + recipes_dsl. Sixteen feature modules total since v0.23.0.
vstack.veval— pattern-vs-ground-truth evaluation harness.EvalCasecarries trace + expected severity.EvalHarness.run()invokes the pattern, compares predicted vs expected severity, returnsEvalResult.compute_metrics()produces precision, recall, F1, accuracy, and confusion matrix (TP/FP/FN/TN). JSON-serializable results.vstack.recipes_dsl— declarative YAML/JSON DSL for custom recipes.validate_recipe()enforces shape/cluster constraints- pattern format.
load_recipe_from_dict/load_recipe_from_file/load_recipes_from_dir. Built-in minimal YAML parser (falls back to PyYAML if installed).RecipeDSLdataclass with full metadata pass-through.
- pattern format.
- Test count: 2,678 → 2,719 (+41 from the two new modules).
- All 2,719 tests pass (1 skipped: crewai not installed).
- Public API surface strictly expanded.
Two more feature modules: signing + synth.
vstack.signing— HMAC-based integrity signing for reports.Signerclass with sign/verify methods. Canonical JSON serialization for stable hashing. SHA-256 + SHA-512 support.VerificationErroron signature mismatch. Convenience functionssign_report()/verify_report(). Key-order independence verified.vstack.synth— programmatic synthetic trace generator. 8 built-in generators:generate_stuck_in_loop,generate_hallucination,generate_sycophancy,generate_over_apology,generate_premature_completion,generate_tool_misuse,generate_anxious_overhedge,generate_healthy. Each takes parameters (retry_count, turns, etc.) for varying difficulty.generate_batch()produces N traces with parameterized seeds for reproducibility. Template registry withregister_template/get_template/list_templatesfor custom generators.
- Test count: 2,631 → 2,678 (+47 from the two new modules).
- All 2,678 tests pass (1 skipped: crewai not installed).
- Public API surface strictly expanded.
Two more feature modules: vbench + markers.
vstack.vbench— in-process pattern benchmark harness.BenchHarnessruns patterns × traces × reps and tabulates per-run metrics: latency, finding count, severity distribution.BenchResultaggregates withStatistics(mean/median/p95/p99).compare_results()flags regressions and improvements between two results. JSON-serializable.vstack.markers— structured markers for trace steps. Built-in marker constructors:cost_marker,latency_marker,quality_marker,safety_marker.CustomMarkerfor user- defined kinds.attach_marker()stores markers on dict or object-style steps.analyze_markers()aggregates across all steps with cost rollup, slow-step detection (configurable threshold), quality average, blocked-step count, and custom marker grouping.
- Test count: 2,588 → 2,631 (+43 from the two new modules).
pyproject.tomladds_vbench,_markersto force-include + testpaths.
- All 2,631 tests pass (1 skipped: crewai not installed).
- Public API surface strictly expanded.
Two more feature modules: calibrate + streaming.
vstack.calibrate— confidence calibration curves. Pure-Python implementations of:IsotonicCalibration: classical pool-adjacent-violators isotonic regression. Monotonic by construction. Block aggregation algorithm.PlattCalibration: σ(ax+b) sigmoid calibration. Gradient descent on log loss.CalibrationMetrics: Brier score, log loss, expected calibration error (ECE).evaluate_calibration(): compute all metrics in one pass. Curves are JSON-serializable. No dependency on sklearn/scipy.
vstack.streaming— SSE-friendly event stream for live diagnoses.EventStreamemitsrun_started/pattern_started/finding_emitted/pattern_completed/run_completed/errorevents. Listener registration (decorator + add_listener), wildcard listeners, queue-based iteration, exception-swallowing for safe broadcast.SSEStreamWriterconverts events to Server-Sent Events format for HTTP streaming.
- Test count: 2,548 → 2,588 (+40 from the two new modules).
pyproject.tomladds_calibrate,_streamingto force- include + testpaths.
- All 2,588 tests pass (1 skipped: crewai not installed).
- Public API surface strictly expanded.
Two new feature modules + trace zoo expansion.
vstack.compose— declarative pattern pipeline composition. FluentPipelinebuilder withstep(),then(),when_severity(),when(),callback(),diagnose(),with_baseline(). Each builder method returns a new immutable Pipeline.run()against a trace executes the pipeline and aggregates findings into aPipelineResultwith severity summary, top-N findings, and to_dict() serialization.Stopexception lets callbacks halt early. 28 new tests.vstack.vdiff— structured diff between two DiagnoseReports.diff_reports()surfaces added/removed findings, severity changes, intervention changes, and pattern additions/removals.ReportDelta.is_regression()andis_improvement()for CI gates. Markdown + dict serialization. 19 new tests.- trace_zoo expansion from 12 → 33 traces. New entries: silent_failure, decision_paralysis, cold_handoff, consensus_dilution, blame_spiral, performative_empathy, handoff_loss, expert_loafing, groupthink, role_thrash, hub_spoke_fragility, trust_collapse, culture_drift, grpi_break, psych_safety_low, bias_stack, devils_advocate_missing, smart_goal_missing, thomas_kilmann_avoiding, mcgregor_x_micromanage, aar_skipped. Includes team and individual shape coverage across all 5 categories.
- Test count: 2,501 → 2,548 (+47 from new modules + zoo).
pyproject.tomladds_compose,_vdiffto force-include + testpaths.
- All 2,548 tests pass (1 skipped: crewai not installed).
- Public API surface strictly expanded.
Three more feature modules: replay, vcache, otel.
vstack.replay— replay historical diagnose() runs from JSONL logs without re-spending LLM calls.ReplayRecorderwraps any client to capture request/response pairs to disk.ReplayClientreads the JSONL log and returns canned responses on hash match. Strict mode raisesReplayMissErroron miss; permissive mode falls back to sequential matching. Use for regression testing, pattern development, and CI gates.vstack.vcache— LLM response cache with TTL + LRU eviction.LLMCachewraps any client and dedupes identical requests. Per-namespace isolation, configurable capacity and TTL, hit/miss stats with cost-saved tracking.vstack.otel— OpenTelemetry exporter.setup_otel()configures the OTLP exporter; spans auto-emit on everydiagnose()run and pattern call.OTelInstrumentedClientwraps any LLM client to emit spans on every chat call. Graceful no-op if OTel isn't installed.
- Test count: 2,450 → 2,501 (+51 from the three new modules).
pyproject.tomladds_replay,_vcache,_otelto force- include + testpaths.
- All 2,501 tests pass (1 skipped: crewai not installed).
- Public API surface strictly expanded.
Three new feature modules: scorecard, budget, trace_zoo.
vstack.scorecard— per-agent multi-pattern scorecard with letter grades. Aggregates findings from many patterns into per-dimension scores (Reasoning / Coordination / Trust / Workload / Culture), each with an A+ → F letter grade. Includescompute_scorecard(),compare_scorecards()(for drift detection),is_blocking_regression()(for CI gates), and text/markdown/HTML renderers. Newvstack-scorecardCLI withcompute/render/comparesubcommands. 119 new tests.vstack.budget— cost-budget enforcement middleware. Wraps any LLM client with rolling-window limits on cost ($), call count, and token count. Per-minute / per-hour / per-day windows.BudgetEnforcerfor single-budget enforcement;BudgetRegistrywith contextvars-based active-client tracking for per-request budget scoping.BudgetExceededexception carries kind/window/limit metadata. 42 new tests.vstack.trace_zoo— canonical library of named synthetic agent traces. 12 curated traces covering the main failure modes (stuck_in_loop, hallucinated_citation, sycophancy_drift, over_apology_loop, overconfidence_spiral, context_saturation, premature_completion, tool_misuse, refusal_cascade, motivation_collapse, anxious_overhedge, healthy_individual). Each carries category + shape + expected severity + expected dominant pattern. Newvstack-trace-zooCLI withlist/show/get/categories/shapessubcommands. 37 new tests.
- Test count: 2,252 → 2,450 (+198 from the three new modules).
pyproject.tomladds_scorecard,_budget,_trace_zooto the force-include block + testpaths + console scripts.
- All 2,450 tests pass (1 skipped: crewai not installed).
- Public API surface strictly expanded. No removals, no argument-shape changes.
- Wire format unchanged.
Documentation expansion: framework playbooks, concepts, tutorials, cluster demos, migration guides.
- 6 framework integration playbooks under
docs/integrations/: LangChain, LangGraph, CrewAI, AutoGen, Pydantic-AI, LlamaIndex. Each covers the adapter, the 8 patterns most useful for that framework's failure modes, trace-capture mechanics, common pathologies, and a production wiring snippet. - 5 per-cluster combined recipe demos under
examples/clusters/: reasoning, coordination, trust, workload, culture. Each runs the recipes for its cluster against representative traces and prints a comparative summary. - 5 migration guides under
docs/migrations/covering the v0.10 → v0.22 upgrade path. Each names what changed, what broke, and what didn't. - 4 new tutorials under
docs/tutorials/:07_dashboard_deployment: stand up the dashboard server.08_baselines_and_drift: per-pattern + fleet baselines + CI gates.09_async_fanout: production-volume async with rate limiting.10_observability: structured logging, metrics, tracing, alerts.
- 4 new concept deep-dives under
docs/concepts/:trace-shapes: individual / team / org schema details.severity-and-confidence: 2x2 matrix + calibration.llm-clients: client protocol + custom integration.recipes-vs-bundles: composition modes compared.
- Pure-documentation release. No code, schema, or wire-format changes from v0.22.0.
- All 2,252 tests pass (1 skipped: crewai not installed).
- Public API surface unchanged.
Examples gallery expansion: cookbook + per-pattern starters + trace fixtures library.
- 24 new cookbook scripts under
examples/cookbook/covering every named recipe in the catalog that didn't yet have an end-to-end stub-driven script:15_stuck_in_loop,16_agents_arguing,17_silent_failure,18_bad_feedback_loop,19_culture_drift,20_goal_misalignment,21_trust_collapse,22_overconfidence_spiral,23_plan_collapse,24_premature_completion,25_tool_misuse,26_anxious_overhedge,27_motivation_collapse,28_handoff_loss,29_deference_cascade,30_expert_loafing,31_cold_handoff,32_performative_empathy,33_decision_paralysis,34_hub_spoke_fragility,35_role_thrash,36_espoused_actual_drift,37_policy_decay,38_hyper_specialization. Each callsdiagnose()with the named recipe and prints an intervention bundle. - 34 per-pattern starter demos under
examples/patterns/, one per shipped pattern. Each demo is the minimum runnable invocation for that pattern: import + stub + trace +diagnose(patterns=[...])call + summary print. Auto-generated to a consistent shape so the gallery is browsable. - 25 new trace builders in
examples/_shared/traces.py:overconfidence_spiral_trace,premature_completion_trace,tool_misuse_trace,anxious_overhedge_trace,motivation_collapse_trace,hallucination_cascade_trace,silent_dependency_drop_trace,bottleneck_orchestrator_trace,consensus_dilution_trace,refusal_cascade_trace,context_saturation_trace,blame_spiral_trace,cold_handoff_trace,performative_empathy_trace,decision_paralysis_trace,role_thrash_trace,policy_decay_trace,healthy_individual_trace,healthy_team_messages,healthy_culture_samples,expert_loafing_messages,deference_cascade_messages,trust_collapse_messages,schein_drift_samples,hyper_specialized_messages.
- Pure-examples release. No changes to pattern surfaces, schemas, console scripts, or wire formats from v0.21.0.
- All 2,252 tests pass (1 skipped: crewai not installed).
- Public API surface unchanged.
Per-pattern WALKTHROUGH gallery — 34 end-to-end recipe documents.
- 34
WALKTHROUGH.mdfiles, one per pattern, under eachmodule-N-*/NN-name/WALKTHROUGH.md. Each follows the same consistent shape so users can jump from any pattern's README to a runnable recipe pack without re-learning the structure:- When-to-reach decision lens (signals it's the right pattern; signals it's not).
- The framework's named structure (e.g. the four Goleman domains, the five Lencioni dysfunctions, the seven Robbins- Judge dimensions).
- Five concrete scenarios with
StubClientcode that runs without LLM credentials. Each scenario names the trace shape, the call site, the expected output, and the intervention. - CLI walkthrough for the pattern's console script.
- Composition chain naming the next pattern to run based on the detected sub-pattern, with explicit links to the downstream WALKTHROUGHs.
- Async fan-out snippet.
- Baseline drift detection snippet using the pattern's
record_baseline/compare_to_baselinehelpers. - Anti-patterns + FAQ + forensic-mode cost.
- Reference section linking the source, schema, prompts, demo, tests, essay, and pattern README.
- Total documentation surface added: ~10,000 LOC across the 34 walkthroughs. Module-1 (individual): 12 files. Module-2 (team): 18 files. Module-3 (organization): 4 files.
- No code changes. WALKTHROUGHs are pure documentation; the pattern surfaces, schemas, and console scripts are unchanged from v0.20.0.
- All 2,252 tests pass (1 skipped: crewai not installed).
- Wire format unchanged. All 34 patterns unchanged.
- Public API surface unchanged.
CLI + cookbook + docs expansion on top of v0.19.x.
vstack-recipesCLI: terminal browser for the recipe catalog. Supports listing all recipes (default, grouped by cluster), filtering by--cluster/--shape, substring search via--q, free-text trigger routing via--match, and per-recipe detail via--show <slug>(with--json/--mdoutput formats). 13 new tests in_diagnose/tests/test_recipes_cli.py.- Three new cookbook recipes demonstrating v0.19.0 named-recipe
expansions: 12
refusal_cascade(Grant Strengths + HEXACO), 13context_saturation(Yerkes-Dodson forensic + Lewin), and 14blame_spiral(Lewin + Lencioni). docs/PATTERNS_OVERVIEW.md: comprehensive single-page map of all 34 patterns with module groupings, literature-anchor map, and failure-surface → patterns table.docs/RECIPES_OVERVIEW.md: catalog overview of all 33 named recipes with per-cluster tables, invocation examples (Python / CLI / MCP / HTTP), and the free-text router walkthrough.
- Repo-wide
ruff formatsweep: 11 pre-existing files were brought into format compliance (mostly pre-existing tests + a few module files that hadn't been formatted since their last edit). Lint is now green across the full CI scope.
- All 2,252 tests pass (1 skipped: crewai not installed). +13 recipes-CLI tests on top of v0.19.1's 2,239.
- Wire format unchanged. All 34 patterns unchanged. The 33 recipes
from v0.19.0 ship verbatim; the new
vstack-recipesCLI is a read-only browser. - Public API surface strictly expanded (new
vstack-recipesconsole script). No removals.
CI hotfix on top of v0.19.0.
- 8 unused-import F401 errors in cookbook recipes 07/08/10/etc.
(
datetime.datetime,datetime.timezone,vstack.lencioni.AgentMessage,vstack.lencioni.MultiAgentTrace). - 1 unused-variable F841 error in
examples/cookbook/08_groupthink_cascade.py:225(base_time).
CI lints examples/ in addition to the source tree; my local
ruff check runs were scoped to _dashboard/ and missed the
cookbook offenders.
- All 2,239 tests pass unchanged. ruff check + ruff format check clean across the full source tree.
Catalog + dashboard + cookbook expansion. Four substantial additions on top of the v0.18.x diagnose runner + MCP + REST surfaces:
Recipes catalog grew from 8 to 33 named recipes organized into 5 thematic clusters:
- reasoning (12): stuck_in_loop, hallucination_cascade, overconfidence_spiral, sycophancy_drift, refusal_cascade, plan_collapse, premature_completion, tool_misuse, over_apology_loop, anxious_overhedge, motivation_collapse, goal_misalignment.
- coordination (8): agents_arguing, bottleneck_agent, bad_feedback_loop, silent_dependency_drop, handoff_loss, consensus_dilution, deference_cascade, expert_loafing.
- trust (5): silent_failure, trust_collapse, cold_handoff, performative_empathy, blame_spiral.
- workload (5): context_saturation, decision_paralysis, bottleneck_orchestrator, hub_spoke_fragility, role_thrash.
- culture (4): culture_drift, espoused_actual_drift, policy_decay, hyper_specialization.
Recipe.cluster field + list_recipes_by_cluster() for CLI / UI
sectioned listings. Each recipe's triggers list seeds the keyword
router, so recipe_for_trigger("the agent keeps apologizing in circles") now returns over_apology_loop without manual picking.
New dark-mode HTML report generator + FastAPI dashboard server, modeled after the superlog observability aesthetic:
render_report(report)/render_reports_overview(reports): self-contained HTML output with Chart.js panels (findings by severity, findings by pattern, cost by pattern, per-pattern table, top-N findings, error panel). Pure Python; opens in any browser.DiagnoseReport.to_html(report_id="..."): convenience method on the runner output for one-call rendering.vstack.dashboard.server.build_app(): FastAPI app with 7 routes (overview, per-run detail, pattern catalog, recipe catalog,POST /v1/reportsingest,GET /v1/reportslist,/healthz).vstack-dashboardCLI:render(pipe report JSON in, HTML out) +serve(start the dashboard server).- 29 tests across
_dashboard/tests/(render, server, CLI).
examples/_shared/traces.py: 12 reusable synthetic agent traces
(stuck_in_loop_trace, hallucinated_citation_trace,
sycophancy_trace, over_apology_trace,
silent_dependency_drop_messages, groupthink_messages,
silent_dissent_messages, social_loafing_messages,
hyper_specialized_roster, hub_and_spoke_roster,
balanced_team_roster, well_executed_individual_trace).
examples/cookbook/ grew by 8 stub-driven end-to-end recipes
demonstrating the new named-recipe expansions:
04 hallucination_cascade, 05 sycophancy_drift, 06 over_apology_loop,
07 silent_dependency_drop, 08 groupthink_cascade,
09 bottleneck_orchestrator, 10 consensus_dilution,
11 diagnose_full_walkthrough.
Six new markdown walkthroughs covering the surfaces that landed in v0.10.0 → v0.18.0: 01 first_diagnosis (15-minute intro), 02 chaining_patterns (manual composition), 03 framework_integrations (LangChain / LangGraph / CrewAI / AutoGen / LlamaIndex / Pydantic-AI / OpenAI Assistants), 04 building_a_custom_pattern (full skeleton), 05 mcp_deployment (Claude Desktop / Cursor / Cline), 06 fastapi_deployment (production hardening).
vstack-dashboardconsole script + thevstack/dashboard/import path (force-included from_dashboard/lib/).Recipe.clusterfield; default value"general"for backward compat.vstack.diagnose.list_recipes_by_cluster()public function.DiagnoseReport.to_html()convenience renderer.
- All 2,239 tests pass (1 skipped: crewai not installed). +29 dashboard tests on top of v0.18.1's 2,210.
- Wire format of all 34 patterns unchanged.
- Public surface of
vstack.diagnosestrictly expanded (newlist_recipes_by_cluster, newRecipe.clusterfield, newDiagnoseReport.to_html). No removals or signature changes. - The 8 v0.10.0 recipes ship verbatim; the 26 new recipes are pure additions.
CI hotfix on top of v0.18.0.
ruff format --checkfailures on_api/lib/_app.py,_api/tests/test_api_diagnose.py, and_mcp/lib/_server.py: appliedruff formatto bring the new code from v0.17.0/v0.18.0 into project format conventions.mypy --strictfailure on_api/lib/_app.py:821:payload.shapeisstr | None(from theDiagnoseRequestEnvelopePydantic model) butdiagnose()expects theTraceShapeLiteral. The validation block above the runner call already rejects non-canonical values; added a narrowedshape_arg: TraceShape | Nonelocal with a# type: ignorecast so mypy accepts it.
- All 2,210 tests pass unchanged. ruff check clean. mypy
--stricton the touched files clean.
POST /v1/diagnose FastAPI endpoint.
Symmetric exposure of vstack.diagnose.diagnose() on the HTTP
surface. v0.17.0 added the MCP vstack_diagnose tool; this release
adds the matching REST endpoint so callers without an MCP transport
(internal services, dashboards, CI bots) can hit the same
cross-pattern runner.
POST /v1/diagnoseendpoint withDiagnoseRequestEnvelopebody schema (trace + shape + recipe + patterns + mode + model + cache + top) andDiagnoseResponseEnveloperesponse schema (shape + findings + per_pattern + errors + cost + optional cache_stats).DiagnoseRequestEnvelope,DiagnoseResponseEnvelope,DiagnoseFinding,DiagnosePerPatternSummary,DiagnoseCostSummary,DiagnoseCacheStatsPydantic models so OpenAPI clients get typed return values instead ofdict[str, Any].- 8 new tests in
_api/tests/test_api_diagnose.pycovering:- End-to-end ranked-findings response from synthetic patterns.
toptruncation behavior.- Validation 400s for unknown recipe / unknown pattern / recipe+ patterns mutex / unknown shape.
- 422 for missing-trace body.
- OpenAPI spec inclusion + correct response-schema reference.
- All 2,210 tests pass (1 skipped: crewai adapter, extra not installed in dev). +8 new tests on top of v0.17.0's 2,202.
- All existing
/v1/analyze/{name}per-pattern routes are unchanged. The new endpoint sits alongside them, reusing the same auth + rate-limit + size-enforcement + Prometheus middleware stack. - Endpoint runs the runner via
asyncio.to_threadto keep the event loop responsive, with the standardstate.limits.request_timeout_secondsdeadline + 504 timeout response (mirrors the analyze endpoint). - OpenAPI schema is generated automatically from the Pydantic
envelopes;
/openapi.jsonincludes the new path under/v1/diagnose.
vstack_diagnose cross-pattern MCP tool.
The MCP server already exposed one tool per pattern
(vstack_lewin, vstack_lencioni, ... 34 tools). Callers who knew
which pattern to run were well served, but the "I have a
misbehaving agent and do not yet know where to start" case still
required reading the catalogue and picking a tool manually.
This release adds vstack_diagnose as a 35th MCP tool that wraps
vstack.diagnose.diagnose() — the cross-pattern runner that infers
trace shape, picks the default bundle, executes every pattern with
per-pattern error isolation, and returns a severity-ranked findings
report.
vstack.mcp._server.DIAGNOSE_TOOL_NAME = "vstack_diagnose"constant._build_diagnose_tool()synthesizes the MCP Tool definition with an opt-in input schema:trace(required): generic JSON object passed to the runner.shape: optional override of inferred trace shape.recipe: optional named recipe slug (mutually exclusive withpatterns).patterns: optional explicit list of pattern slugs.mode:quick/standard/forensic.model: LLM model identifier override.cache: opt-in shared-response cache wrapper.top: optional truncation of the ranked findings list.
_dispatch_diagnose_call()routes the request throughvstack.diagnose.diagnose(), with input validation guards for unknown recipes, unknown patterns, and recipe/patterns mutual exclusivity._serialize_diagnose_report()renders the DiagnoseReport as indented JSON: shape, ranked findings, per-pattern summary, errors, cost summary, optional cache stats.- 7 new tests in
_mcp/tests/test_server_diagnose.pycovering tool listing, input schema shape, end-to-end dispatch with synthetic patterns, and validation-error paths.
_list_toolsnow returns 35 tools (1 cross-pattern + 34 per-pattern) instead of 34.- The existing
test_list_tools_returns_all_patternsassertion updated to expect 35 tools and includevstack_diagnosein the expected name set.
- All 2,202 tests pass (1 skipped: crewai adapter, extra not installed in dev). +7 new tests on top of v0.16.0's 2,195.
- All 34 per-pattern tools' input schemas are unchanged.
- MCP wire protocol unchanged; the new tool follows the same
name/description/inputSchemaTool shape as the existing tools. - The cross-pattern runner is the same
vstack.diagnose.diagnose()function exposed in v0.10.0+; this release only adds the MCP surface to it.
Lossless per-pattern findings extraction in vstack.diagnose.
The old _coerce_findings reflective normalizer only recognized the
generic findings and top_findings attribute names. Almost no real
vstack pattern exposes either of those, so the runner fell back to
the score-only path and surfaced exactly ONE Finding per pattern
run. That was lossy: a real Lencioni report has FIVE dysfunction
evidence entries; a Bias Stack run has FOUR bias scores; an AAR run
has 1-5 lessons. Each should land as its own ranked finding.
vstack.diagnose.adaptersmodule with a smartextract_findingsfunction that walks the field-name inventory vstack patterns actually use:- 47 evidence-list field names (
dysfunctions,legs,domains,factors,triggers,strengths,pathologies,quadrants,behaviors,loci,terms,traps,needs,biases,characteristics,dimensions,phases,zones,styles,metrics,lessons,branches, plus_evidence-suffix variants and collection accessors). - 24 categorical-title field names so per-item titles populate
correctly from the pattern-specific axis name (
dysfunction,leg,factor,quadrant,behavior, etc.). - 6 severity field names (
severity,severity_of_gap,severity_of_absence,severity_of_overuse,mismatch_severity,risk) so per-pattern severity wire formats normalize to the canonical 7-point scale. - 9 score field names (
score,weight,presence_score,wobble_score,overuse_score,observed_score,fit_score,substantive_score,value,coherence_score) used when severity is absent and we need to derive one.
- 47 evidence-list field names (
vstack.diagnose.register_adapterdecorator for registering per-pattern override adapters when the smart extractor's conventions do not fit a pattern's schema.vstack.diagnose.ADAPTERSdispatch table exposing the registered overrides.- Built-in AAR override adapter:
Lessoncarries no severity field by design (it is a narrative finding, not scored evidence), so the override emits each Lesson as amoderate-severity Finding with pattern slug + description in the title and root_cause + framework anchor in evidence.
vstack.diagnose.runnernow delegates findings extraction toextract_findingsfrom the adapters module. The legacy private name_coerce_findingsis preserved as an alias for backward compatibility.Findingis now defined invstack.diagnose.adapters(was previously inrunner); it is re-exported fromrunnerand from the top-levelvstack.diagnosepackage so all import paths continue to work._score_to_severityband boundaries tightened to match the seven-band calibration the prompt-engineering pass standardized on (0.10/0.25/0.40/0.55/0.70/0.85 thresholds).
- 14 new tests in
_diagnose/tests/test_diagnose_adapters.pycovering:- Multi-finding extraction from
dysfunctions(5),biases(4),lessons(2) shapes. - Pattern-specific severity field name normalization
(
severity_of_gap,severity_of_absence). - Per-pattern adapter override precedence.
- Override-raises-fall-through-to-smart graceful degradation.
- Backward compatibility with legacy
findings=[...]lists andtop_findingsalias. - Score-only fallback path.
- Severity-and-title-at-top-level fallback path.
- None / empty-list / vague-item edge cases.
- Multi-finding extraction from
- All 2,195 tests pass (1 skipped: crewai adapter, extra not installed in dev). This is 14 new tests on top of the 2,181 from v0.15.0.
Finding,extract_findings,_coerce_findings,ADAPTERS,register_adapterare all importable fromvstack.diagnoseAND fromvstack.diagnose.runner(where applicable) for backward-compat.- Wire format of every analyzer's LLM output is unchanged. Only the runner-side extraction logic was lifted.
Prompt-engineering uplift pass COMPLETE — every shipped pattern (34 of 34) now ships the uniform uplift template introduced in v0.13.0.
The remaining 24 patterns received the same uplift the v0.13.0/v0.14.0 top-10 received: explicit OUTPUT SCHEMA literals, one-shot examples, DO NOT rules covering common failure modes, severity calibration anchors, canonical-order enforcement.
| Module | # | Pattern | Anchor literature |
|---|---|---|---|
| 1 (individual) | 2 | Goleman EI | Goleman 2002; Mayer-Salovey 1997; Joseph-Newman 2010 |
| 1 | 3 | Johari Window | Luft 1969; Stone-Heen 2014; Kadavath 2022 |
| 1 | 4 | DANVA Emotion Reader | Nowicki-Duke; Ekman; Plutchik; Russell |
| 1 | 5 | Cognitive Reappraisal | Gross 1998/2014; Sheppes-Suri-Gross 2015 |
| 1 | 7 | HEXACO Personality | Lee-Ashton 2004-2018; Bourdage 2007 |
| 1 | 8 | Grant Strengths | Grant-Schwartz 2011; Kaiser-Kaplan 2009 |
| 1 | 9 | Motivation Traps | Saxberg-Hess 2013; Weiner 1985; Bandura 1977 |
| 1 | 10 | SDT Intrinsic Reward | Deci-Ryan; Pink 2009; Casper 2023 |
| 1 | 11 | McGregor Orchestrator | McGregor 1960; Eisenhardt 1989 agency theory |
| 1 | 12 | Vroom Expectancy | Vroom 1964; Porter-Lawler 1968; Bandura 1977 |
| 2 (team) | 13 | GRPI Working Agreement | Beckhard 1972; Rubin-Plovnick-Fry 1977 |
| 2 | 15 | Social Loafing | Latane 1979; Karau-Williams 1993 |
| 2 | 16 | Heffernan Superflocks | Heffernan 2014; Muir 1996; Page 2007 |
| 2 | 19 | McAllister Trust | McAllister 1995 (cognitive vs affective) |
| 2 | 21 | Glaser Conversation | Glaser 2014 Conversational Intelligence |
| 2 | 22 | Stone-Heen Feedback | Stone-Heen 2014 Thanks for the Feedback |
| 2 | 23 | Plus/Delta Feedback | Joiner Associates; Brown 2018 |
| 2 | 24 | SMART Goal | Doran 1981; Locke-Latham 1990 |
| 2 | 25 | Group Decision Models | Kaner 2014; Vroom-Yetton 1973 |
| 2 | 29 | Thomas-Kilmann | Thomas-Kilmann 1974 conflict modes |
| 3 (org) | 31 | Schein Iceberg Culture | Schein 1985/2010/2017 |
| 3 | 32 | Robbins-Judge 7 Culture | Robbins-Judge OB 17th ed. 2017 |
| 3 | 33 | Org Structure Matrix | Galbraith Star Model; Mintzberg 1983 |
| 3 | 34 | Span of Control | Galbraith 1977; Mintzberg 1983 |
- Every
prompts.pyacross the 24 patterns listed above. Public template constant names and{placeholder}fields are unchanged;assemble_promptsemantics are unchanged; all downstream callers see no API break. - System prompts now ship explicit posture rules, anti-pattern instructions, calibration tables (severity / fit / quality / motivation thresholds), and per-category symptom signatures.
- Each task prompt that the generator parses now ships a literal OUTPUT SCHEMA block with the exact JSON shape.
- Pattern-specific DO NOT rules prevent the most common failure modes observed in the v0.12.x baseline (invented quotes, vague interventions, schema-violating labels, sycophantic mimicry being scored as relationship_management, performative care being scored as affective trust, mid-superflocks "make the top agent better" recommendations, identity-trigger apology spirals being escalated, etc.).
- One-shot examples on each pattern's main scoring prompt demonstrate the textbook failure signature for that pattern with verbatim evidence quotes and named-framework anchoring.
This closes the prompt-layer quality gap across the entire pattern library. The 0.10–0.12 releases shipped the runner, cost tracking, and shared cache; the 0.13–0.14 releases lifted the top 10 prompts; 0.15.0 brings the long tail (24 patterns) to the same bar. The diagnose pipeline now has uniform per-pattern prompt quality regardless of which sub-bundle the caller invokes.
- All 2,181 tests pass unchanged (1 skipped: crewai adapter, extra not installed in dev).
- Wire format of every analyzer's LLM output is preserved.
- Public template constant names +
{placeholder}field sets are unchanged across all 34 patterns. assemble_promptsemantics are unchanged.
Prompt-engineering uplift pass — top 10 patterns completed.
The uplift template (introduced in 0.13.0 on Lencioni) has now been
applied uniformly across the ten highest-leverage patterns. Each
pattern's prompts.py now ships:
- An explicit OUTPUT SCHEMA block with the literal JSON shape the generator parses, so the LLM does not have to reverse-engineer the schema from a one-line "return JSON" hint.
- A one-shot example of a well-calibrated answer demonstrating verbatim evidence quotation and named-source anchoring.
DO NOTrules covering the most common failure modes (invented quotes, name-dropped citations without anchoring, vague interventions, schema-violating labels, scope creep between phases).- Severity calibration anchors tied to score bands (where the pattern uses numeric scoring), mapped down to the wire-format severity labels.
- Canonical ordering rules so downstream parsers do not have to reorder arrays.
| # | Pattern | Anchor literature |
|---|---|---|
| 1 | Lewin Formula | Lewin 1936; Kelley 1967; Cemri et al. 2025 |
| 6 | Yerkes-Dodson | Yerkes-Dodson 1908; Sweller; Liu et al. 2024 |
| 14 | Process Gain/Loss | Steiner 1972; Hill 1982; Diehl & Stroebe |
| 17 | Lencioni Diagnostic | Lencioni 2002 + 2005; Edmondson 1999 |
| 18 | Trust Triangle | Frei & Morriss 2020 |
| 20 | Psych Safety | Edmondson 1999, 2018 |
| 26 | Debate Pathology | Janis 1972; Stoner 1968; Hatfield et al. |
| 27 | Bias Stack | Kahneman/Tversky 1974, 1979, 2011; Staw 1976 |
| 28 | Devil's Advocate | Janis 1972; Schwenk 1990 |
| 30 | AAR Generator | Wharton; US Army TC 25-20; Edmondson 1999 |
- Every
prompts.pyfile across the ten patterns above. Public template constant names and{placeholder}fields are unchanged;assemble_promptsemantics are unchanged; all downstream callers see no API break. - System prompts now ship explicit posture rules, anti-pattern instructions, severity calibration tables, and per-category symptom signatures so the LLM has a consistent grounding across modes (quick / standard / forensic).
The 0.10–0.12 releases shipped the cross-pattern runner, cost tracking, and shared LLM-response cache for the diagnose pipeline. Pattern quality at the prompt layer was the next ceiling on signal-to-noise. This release closes that gap across the top ten patterns by leverage; the remaining 24 patterns will receive the same lift on the same template in future releases.
- All 2,181 tests pass unchanged (1 skipped: crewai adapter, extra not installed in dev).
- Wire format of every analyzer's LLM output is preserved for downstream tooling.
- Public template constant names and
{placeholder}field sets are unchanged. assemble_promptsemantics are unchanged across all ten patterns.
Prompt-engineering uplift pass, starting with Lencioni #17. Each prompt in the pattern now ships an explicit OUTPUT SCHEMA block, a one-shot example of a good answer, "DO NOT" rules covering the most common failure modes, and a seven-level severity calibration table on the system prompt. The wire format is unchanged.
vstack.lencioni(module 17): rewroteprompts.pyfor all six templates (PYRAMID_SCORE_PROMPT,INTERVENTIONS_PROMPT,QUICK_DIAGNOSTIC_PROMPT,FORENSIC_CASCADE_PROMPT,FORENSIC_PSYCH_SAFETY_PROMPT,FORENSIC_INTERVENTIONS_PROMPT). Public template constant names and{placeholder}fields are unchanged;assemble_promptsemantics are unchanged. Downstream callers see no API break.LENCIONI_SYSTEM_PROMPTnow defines a seven-level severity calibration anchored in score bands (none / trace / low / moderate / medium / high / critical) and lists explicit anti-pattern rules (no invented quotes, no invented citations, no refusals on thin traces).- Each task prompt now ships a literal OUTPUT SCHEMA block, so the LLM does not have to reverse-engineer the JSON shape from a one-line "return JSON" hint.
PYRAMID_SCORE_PROMPTships a one-shot example demonstrating good severity calibration plus two distinct verbatim evidence quotes.
The 0.10–0.12 releases gave the pattern bundle a runner, cost tracking, and a shared cache. Pattern quality at the prompt layer was the next ceiling on the diagnose pipeline's signal-to-noise ratio. Lencioni is the first pattern to receive the lift; AAR, Lewin, Bias Stack, Psych Safety, Trust Triangle, Process Gain/Loss, Debate Pathology, Devil's Advocate, and Yerkes-Dodson follow on the same template.
- All 44 Lencioni tests pass unchanged.
- All 84
vstack.diagnosetests pass unchanged (the runner reflectively introspects analyzer output; prompt-text changes do not affect the runner contract). - Wire format of the LLM output (DysfunctionEvidence severity values are still high/medium/low/none) is preserved for downstream tooling.
Shared LLM-response cache across patterns in one diagnose() run.
When a bundle of patterns hits the LLM with identical prompts, the
cache returns the cached completion without paying twice.
vstack.diagnose.CachingLLMClient: thin wrapper around any LLM client that memoizescomplete(prompt, system)calls keyed on(prompt, system, model). SHA-256 keys; LRU eviction atmaxsize=256(configurable).vstack.diagnose.CachingClientStats: hit / miss / insert counters plus ahit_rateproperty andbytes_savedheuristic.diagnose(cache=True, ...)anddiagnose_async(cache=True, ...): opt-in shared cache for the duration of one call. The wrapper is threaded through to every analyzer in the bundle.DiagnoseReport.cache_stats: populated when caching was on,Noneotherwise so callers can tell whether the cache was active.to_markdown()renders a cache line in the cost section whencache_statsis present.__getattr__forwarding on the wrapper: analyzers that read unknown attributes off the client (e.g.last_usage,model) see the real underlying client transparently.
- 8 new tests in
_diagnose/tests/test_diagnose_cache.py:- hash key stability + collision resistance
- wrapper returns cached on second identical call
- wrapper misses on different prompts
- LRU eviction when
maxsizeexceeded - attribute forwarding to inner client
diagnose()without cache: two patterns -> two inner callsdiagnose(cache=True): identical prompts -> one inner callcache=Truewith no client is a graceful no-op
- Total
_diagnosesuite: 84 passing. - Full repository suite: 2181 passing, 1 skipped (crewai).
- Caching is opt-in (
cache=Falseby default). - Pattern analyzers see no API change; the wrapper exposes the same
complete()signature as the underlying client.
Cost tracking in vstack.diagnose. The runner now installs an
in-memory telemetry sink for the duration of each diagnose() call
and aggregates LLM-call events into a CostSummary attached to the
report.
vstack.diagnose.CostSummarydataclass: aggregate token + latency stats with per-pattern and per-model breakdowns.DiagnoseReport.cost: populated automatically when participating analyzers emit telemetry viavstack.aar.record_llm_call.DiagnoseReport.to_markdown()renders a## Cost summarysection whencost.llm_calls > 0(skipped silently otherwise).- The previously-installed telemetry sink is restored after the run, so callers who had their own sink installed see no global-state mutation.
- 5 new tests in
_diagnose/tests/test_diagnose_cost.py:- empty cost summary when no telemetry
- aggregation across multiple emitting analyzers
- per-pattern and per-model breakdowns
- sink restoration after the run
- Markdown render gated by
cost.llm_calls
- Total
_diagnosesuite: 76 passing. - Full repository suite: 2173 passing, 1 skipped (crewai).
- Patterns that don't emit telemetry contribute nothing to the cost summary; the report's other fields are unchanged.
- All existing imports continue to work.
Named recipes layer for vstack.diagnose. Until now, callers had to
either accept the shape-default bundle or hand-pick patterns; this
release adds an opinionated layer in between.
vstack.diagnose.RECIPES: catalog of 8 named bundles, each curated for a specific recognizable failure mode:stuck_in_loop(individual)agents_arguing(team)silent_failure(team)bottleneck_agent(team)bad_feedback_loop(team)culture_drift(org)goal_misalignment(individual)trust_collapse(team)
- Each recipe has a description, applicable shape, ordered pattern list, and trigger-keyword phrases.
diagnose(recipe="stuck_in_loop", ...): pick a recipe by name. Explicitpatterns=still wins if both are supplied.recipe_for_trigger("agent keeps making the same mistake"): free-text keyword matcher.vstack-diagnose --recipe <name>and--match "<text>"CLI flags.vstack-diagnose --list-recipesenumerates the catalog.- Catalog validation runs at import time: a recipe that references an unknown pattern slug raises immediately instead of failing at call time.
- 12 new tests in
_diagnose/tests/test_diagnose_recipes.py. - Total
_diagnosesuite: 71 passing. - Total repository suite: 2168 passing, 1 skipped (crewai).
CLI surface for vstack.diagnose. Adds the vstack-diagnose entry
point so the cross-pattern runner is reachable from a shell, not only
from Python.
vstack-diagnoseCLI: reads a JSON trace from--trace <path>or stdin and prints either Markdown (default) or--jsonoutput of the cross-pattern report.--list: enumerates every shipped pattern with its applicable shapes and one-line summary.--shape {individual|team|org}: force trace-shape inference.--patterns slug [slug ...]: override the default bundle with an explicit list.--client {anthropic|openai|ollama|none}: pick the LLM client. The CLI defaults tononeso no paid call is ever made without explicit opt-in.--mode {quick|standard|forensic}: forward pipeline mode to analyzers that accept one.--top <N>: number of top findings to surface in Markdown.- Permissive trace JSON parsing: required AAR
TraceStepfields (timestamp/type/content) are auto-filled from common alternative names (action,note, etc.) so a single line of test JSON produces a valid trace.
- 6 new CLI tests in
_diagnose/tests/test_diagnose_cli.py. - Total
_diagnosesuite: 59 passing. - Total repository suite: 2156 passing, 1 skipped (crewai).
Curated public API + cross-pattern diagnostic runner. Until this release, using vstack at scale meant importing each of the 34 pattern sub-packages by hand and calling their analyzers separately. 0.8.0 adds a top-level entry point that wraps the patterns into a single call.
diagnose(trace, llm_client=...): runs a curated bundle of patterns against one agent or multi-agent trace and returns a single rankedDiagnoseReport. Bundle selection is shape-aware: a single-agent trace runs the individual-failure-mode patterns; a crew trace runs the team patterns; an org-scale trace runs the org-design patterns. Callers can override the bundle withpatterns=[...].diagnose_async(...): concurrent variant that uses each pattern's async analyzer with a configurable in-flight bound.PATTERNS: a declarative registry of every shipped pattern with module path, main analyzer class name, async variant, applicable trace shapes, summary, and tags. Enumerable for tooling that wants to iterate every pattern without hand-importing.DiagnoseReport.to_markdown(): opinionated single-document render with a "Top findings" section and a per-pattern errors section.- Findings normalization: the runner reflects on common return-shape
attributes (
findings,severity,score, …) so new patterns work without changes to the runner. - Error isolation: one failing pattern does not kill the report.
Failures land in
DiagnoseReport.errorsand the rest of the bundle still produces findings.
vstack.__init__now documents the curated API and exposesdiagnose/PATTERNSvia PEP 562 lazy attributes.vstack.diagnoseis the canonical import path:from vstack.diagnose import diagnose, PATTERNS.
- 53 new tests across
_diagnose/tests/:- 44 registry-shape invariants over every shipped pattern.
- 9 runner tests covering ranking, error isolation, shape inference, explicit-shape override, markdown rendering, and the async runner.
- Total repository test count after this release: 2150 passing.
- All existing single-pattern import paths (
from vstack.aar import AARAnalyzer, etc.) continue to work unchanged. - No pattern source files modified.
Onboarding + launch-polish release. Closes the gap between "library shipped" and "users can pick it up and feel productive in 30 seconds."
vstack-helloCLI: a 30-second first-run demo that builds a synthetic agent-failure trace (8 steps, a recognizable edit-before-read failure mode), resolves an LLM client from the environment, and runs the AAR pattern against the trace.--offlineflag: skip LLM resolution and print a pre-rendered sample AAR. Useful for CI smoke tests, demo recordings, and air-gapped environments.--jsonflag: machine-readable envelope.--no-bannerflag: pipe-friendly output.- Graceful fallback: if no API key is set, the command still prints a complete sample so users see the shape of vstack's output without ever hitting an error path.
.github/workflows/docs.ymlbuilds the mkdocs-material site on every push tomainthat touchesdocs/,mkdocs.yml, or the workflow itself, and publishes to https://valani9.github.io/vstack/.mkdocs.ymlsite_urlupdated to the hosted URL.
.github/dependabot.yml: weekly pip + github-actions + docker updates, with grouped PRs for framework adapters / LLM clients / dev-tooling so reviewer load stays low..github/workflows/codeql.yml: CodeQL static analysis on push + PR + a weekly cron, with thesecurity-extendedandsecurity-and-qualityquery packs enabled..github/ISSUE_TEMPLATE/with three forms (bug / feature / question), plusconfig.ymllinking tovstack-doctor, the security policy, and the hosted docs..github/PULL_REQUEST_TEMPLATE.md: explicit test plan + public-API-impact + release-notes-line checklist.
.github/RELEASE_TEMPLATE.md+.github/render_release_notes.py: the GitHub Release body is now rendered from the matchingCHANGELOG.mdsection + a template, instead of inlined intorelease.yml. Falls back to the[Unreleased]section if the exact-version section is missing.
pyproject.toml: 14 new keywords (agent-debugging,agent-observability,agent-evaluation,llm-evaluation,prompt-engineering,post-mortem,after-action-report,retrospective,agent-failure-modes,multi-agent-systems,model-context-protocol,mcp-server,llmops,agent-ops).pyproject.toml:vstack-helloregistered as a[project.scripts]entry;_hello/libadded to the force-include map and the pytest testpaths.release.yml: the smoke-test step now also importsvstack.helloto catch any wheel-mapping regressions.ci.yml: mypy + test + lint extended to cover_hello/.- Shell completions (bash + zsh + fish) for
vstack-hello's flags.
- mkdocs site builds clean under
--strict(no broken links, no ambiguous nav entries) as a precondition for the Pages workflow.
- Python imports
- 34 per-pattern CLIs
- MCP server (
vstack-mcp) - REST API (
vstack-api) - Docker (
ghcr.io/valani9/vstack:0.7.0) - Claude Code skills (7)
- Framework adapters (LangChain / LangGraph / CrewAI / AutoGen / LlamaIndex / Pydantic AI)
- OpenAI / Anthropic tool JSON
- Open WebUI plugin manifest
- Tier B platform generators (
vstack-config gen-platform) - Browser dev tooling (
vstack-browser) - First-run smoke (
vstack-hello) — new in 0.7.0
Production-hardening release. Adds the security + cache + observability +
diagnostic infrastructure that takes vstack-api from "fine for localhost"
to "ready for thousands of concurrent users."
APIKeyStore+APIKey— SHA-256-hashed in-memory keys with constant-time verification. Load fromVSTACK_API_KEYS/VSTACK_API_KEYS_FILE.InMemoryRateLimiter— sliding-window per-key (or per-IP) limiter withRateLimitDecision+Retry-Aftersemantics.RequestLimits— declarative caps on body size, trace steps, message count, per-string chars, total chars, request timeout. Configured viaVSTACK_API_MAX_*env vars.audit_input_for_injection/safe_pattern_name/safe_path/safe_subprocess_argv/warn_on_suspicious_inputs— defense-in- depth helpers for the parts of vstack that take user input.
InMemoryLRUCache+NullCache+CacheBackendprotocol +CacheEntry+CacheStats.build_cache_key(pattern, mode, model, trace)— SHA-256 over the canonical JSON of the trace + run params; identical traces hit the cache cleanly.resolve_cache_from_env()honorsVSTACK_CACHE=memory|off,VSTACK_CACHE_CAPACITY,VSTACK_CACHE_TTL_SECONDS.
MetricsRegistry+Counter+Histogram+render_prometheus()— hand-rolled Prometheus text-format exporter, noprometheus_clientdependency.record_request/time_request— request-level helpers that populatevstack_requests_total{surface,pattern,mode,status}+vstack_request_duration_seconds{surface,pattern,mode}.REQUEST_ID_HEADERconstant +get_or_create_request_id/set_current_request_id/current_request_id—X-Request-IDround-trip + contextvar binding for log correlation.install_sentry_if_configured()— optionalsentry-sdkintegration viaSENTRY_DSN. Silently no-ops if the SDK isn't installed orSENTRY_DSNis unset.
- Auth middleware —
Authorization: Bearer+X-API-Keysupport; constant-time comparison; rejects with401+WWW-Authenticatewhenrequire_auth=Trueand the key is missing/wrong. - Rate-limit middleware —
429+Retry-After+ theX-RateLimit-Limit/X-RateLimit-Remainingheaders on every response. - Body-size middleware — rejects oversized POSTs with
413before they're decoded. - Security-headers middleware —
X-Content-Type-Options,X-Frame-Options,CSP,Referrer-Policy, conditional HSTS. - Request-ID middleware — generates / echoes
X-Request-ID; binds to a contextvar for log correlation. - CORS middleware — opt-in via
VSTACK_API_CORS_ORIGINS. /readyz+/livez+/metrics— separated K8s probe semantics (readyzflips todrainingon shutdown); Prometheus metrics endpoint at/metrics.- Graceful shutdown — FastAPI lifespan handler drains in-flight
requests on
SIGTERM. - Async analyze path — uses analyzer
*Asyncmirrors when the LLM client hasacomplete; falls back to a thread executor for the sync analyzer so concurrent HTTP requests don't serialize on the event loop. - Cache integration — cache lookup happens BEFORE LLM resolution so a cache hit costs zero LLM round-trips.
- Request timeout — server-side per-request deadline
(
VSTACK_API_REQUEST_TIMEOUT, default 120s); returns504on exceedance.
vstack.memory.atomic_write_text/atomic_write_bytes— tmp- file +os.replacefor crash-safe writes. Wired intosave_config()andLearningStore.update_outcome().vstack.memory.append_locked/shared_read_lock/FileLock— POSIX advisory locks (with Windowsmsvcrtfallback) for cross- process JSONL append + read. Wired intoLearningStore.record()andFileTelemetrySink.record().
- Audits 25+ checks across Python version, vstack install, pattern
registry,
~/.vstack/writability, LLM client resolvability, every documented CLI on PATH, every optional extra, gbrain reachability, Node.js availability for browser, API auth misconfiguration, and PyPI upgrade availability. --jsonfor machine-readable output;--skip-networkfor air-gapped CI;--only-errorsfor terse output.- Exit code 1 when any check is ERROR-level; 0 otherwise.
completions/vstack.bash,completions/_vstack(zsh), andcompletions/vstack.fish— tab-completion for all 10 top-level CLIs + subcommands + key arguments (pattern names, platform names, path kinds, config keys).completions/README.mdinstalls instructions.
docs/operations/deploy.md— minimum production checklist; Docker-only + Kubernetes Deployment manifests; auth + rate limiting + request limits + cache + observability config; what stays in-process vs. needs a shared backend at scale; troubleshooting.docs/operations/security.md— three-ring security model (library guards, configurable API guards, deployment responsibilities), threat model, audit posture, vulnerability reporting.
- 4 new force-include lines (
_security/lib,_cache/lib,_observability/lib,_doctor/lib). - 1 new
[project.scripts]entry:vstack-doctor. - 4 new testpaths.
- Version bump 0.5.0 → 0.6.0.
- +113 new tests across
_security/tests/(53),_cache/tests/(15),_observability/tests/(17),_doctor/tests/(8), and_api/tests/test_api_security.py(21 new hardening + caching tests). - Suite total: 2,088 passing (up from 1,969 in v0.5.0).
- Mypy strict clean across all 14 surface lib dirs (the 10 from
v0.5.0 +
_security,_cache,_observability,_doctor).
Phase 3 surface + depth-pass release. Adds the browser dev tooling
(Tier C #I5), the gbrain integration (semantic search over the
pattern catalogue), the canonical Span-of-Control calibration
baselines, the cross-pattern composition runbook, the in-repo
mkdocs-material documentation site, runnable framework demos under
examples/, the benchmark + comparative-eval harness, and the first
six narrative essays at Substack depth.
- Wraps the upstream
chrome-devtools-mcpserver. Scrape agent traces from LangSmith / Phoenix / Helicone / Langfuse without screen-scraping; screenshot detection reports; drive agents in the browser for evaluation. BrowserSession,open_sessionasync context manager,scrape_trace,screenshot_url,fill_form,KNOWN_DASHBOARDSrecipe catalogue (5 vendors).vstack-browserCLI:scrape,screenshot,toolssubcommands.- New
[browser]optional extra (pip install valanistack[browser]). - 28 new tests; all mock the upstream MCP boundary so Chrome doesn't need to be installed in CI.
- Detects gbrain on PATH; semantic search via the gbrain CLI when available; graceful keyword-fallback when not.
indexed_corpus()builds 34 documents (summary + group + schema info + playbook keys + composition);sync_corpus()writes them into gbrain idempotently.vstack-gbrainCLI:status,sync,search,corpus.- 18 new tests; all mock
shutil.whichandsubprocess.runso gbrain doesn't need to be installed in CI.
- Three pre-computed Span-of-Control baselines covering the textbook crew topologies (small-flat, two-layer, hub-and-spoke). Deterministic because Span-of-Control's math is gated to Python.
_baselines/scripts/generate_canonical.pyregenerates them reproducibly._baselines/README.mddocuments the recipe for users to build per-pattern baselines for the 33 LLM-bearing patterns.
- Repo-root document mapping the five canonical chains (F1 confidently-wrong, T1 audit-crew, S1 bottleneck, C1 culture, D1 calibration). Code + per-chain decision tables + cross-chain transitions + the shared executive-readout template.
- mkdocs-material configuration with full navigation: Home, Quick start, Concepts, Patterns (3 modules), Surfaces (7), Workflows (5), Reference (4), Recipes (4 worked examples), Changelog.
- Build offline with
pip install valanistack[docs] && mkdocs build(writessite/— no hosting required). - 18 pages of structured content; integrates with the existing README + COMPOSITION-RUNBOOK + per-pattern docs.
- 7 demo scripts:
langchain_demo.py,langgraph_demo.py,crewai_demo.py,autogen_demo.py,llamaindex_demo.py,pydantic_ai_demo.py,openai_assistants_demo.py. - Each script is self-contained (~50 lines) and exercises the adapter end-to-end with a tiny canonical Lewin / AAR / Lencioni invocation.
BenchmarkCase,BenchmarkSuite,BenchmarkRunner,BenchmarkReportfor running pattern suites end-to-end.- Canonical 3-case suite (Lewin / AAR / Schein) ships built-in;
custom suites loaded from JSON via
load_suite(path). - Comparative harness: quick / standard / forensic side-by-side with elapsed-ms + severity + dominant-finding agreement signal.
- Stub-LLM friendly: the test suite drives the harness without burning API spend. Real numbers arrive when the user runs with their own LLM client.
vstack-benchCLI:list,run,comparesubcommands.- 30 new tests.
_TEMPLATE.mdcanonical structure.- Six anchor essays at depth (~700-900 words each): Lewin (#01), Lencioni (#17), Edmondson psych safety (#20), AAR (#30), Schein (#31), Span-of-Control (#34). The remaining 28 essays follow the template; quality bar established here.
- 3 new
[project.scripts]entries:vstack-browser,vstack-gbrain,vstack-bench. - 2 new optional extras:
[browser](pullsmcp>=1.20.0),[docs](mkdocs + mkdocs-material). - 3 new force-include lines.
- 3 new testpaths.
- Refactored Dockerfile to install from the release.yml-built
wheel artifact when present in
dist/, with a PyPI fallback for local builds. Bypasses the PyPI CDN-propagation race that bit v0.4.0's docker workflow. - docker.yml now triggers on
workflow_runof Release (success), downloads therelease-distartifact, and feeds it to buildx. No PyPI calls inside the build.
- mypy strict loop covers all 10 surface lib dirs (the 7 from
v0.4.0 +
_browser,_gbrain,_benchmarks). - Test job runs the new test dirs.
- Lint job covers the new dirs +
examples/+_baselines/scripts/. - Release smoke test imports
vstack.browser/vstack.gbrain/vstack.benchmarks.
- +74 new tests (28 browser + 18 gbrain + 30 benchmarks). Suite total: 1,969 passing (up from 1,895 in v0.4.0).
- Mypy strict clean across all 10 surface lib dirs.
Phase 2 of the expansion roadmap lands. v0.4.0 adds framework adapters, a learning store, a telemetry aggregator, and config generators for the remaining Tier B native platforms. vstack is now reachable from LangChain / LangGraph / CrewAI / AutoGen / LlamaIndex / Pydantic AI / Open WebUI / OpenAI Assistants / Anthropic Messages in addition to the v0.2.0+v0.3.0 surfaces.
- Unified, registry-driven adapter module: same 34 patterns, eight
framework-native shapes.
as_openai_tool_schemas()— OpenAI Chat / Assistantstoolsarray. Pure JSON; no extra dependency.as_anthropic_tool_schemas()— Anthropic Messagestoolsarray. Pure JSON.as_autogen_function_manifest()+as_autogen_callables()— Microsoft AutoGen function manifest + Python callables. Pure Python; no autogen import required.as_openwebui_manifest(api_base_url=...)— Open WebUI tool- plugin manifest pointing at a runningvstack-api.as_langchain_tools()—StructuredToolinstances; needsvalanistack[langchain].as_langgraph_nodes()/node_for(pattern_name)— state- delta node factories; needsvalanistack[langgraph].as_crewai_tools()—BaseToolsubclass instances; needsvalanistack[crewai].as_llamaindex_tools()—FunctionToolinstances; needsvalanistack[llamaindex].as_pydantic_ai_tools()—(name, description, func)triples; needsvalanistack[pydantic_ai].
- Shared dispatcher (
run_pattern_dispatch) — every adapter validates input against the pattern's Pydantic model, resolves an LLM client, runs the analyzer, and returns the detection as a JSON-safe dict (or a structured{"error", "message"}envelope). - Adapter spec types (
PatternToolSpec,list_pattern_tool_specs) are framework-neutral and reusable for any custom adapter you want to write.
- Append-only JSONL store at
~/.vstack/learnings.jsonl. LearningRecordschema: pattern, mode, agent_id / crew_id, severity, profile_pattern, dominant_finding, interventions_applied, follow_up_outcome, notes, extra.LearningStore.record / recall / update_outcome / outcomes / iter_records / clear— streaming reads, latest-open-record mutation for outcome tagging.OutcomeAggregaterolls up(pattern, intervention) -> improved / no_change / worse / unknowncounts with animprovement_rateproperty.vstack-learnsubcommands:record,recall,outcome,outcomes,path,clear; all support--json.
FileTelemetrySink— drop-in forvstack.aar.TelemetrySinkthat appends one JSONL line perrecord_llm_callevent to~/.vstack/analytics/telemetry.jsonl. Activate once withenable_file_telemetry()at process start.TelemetryAggregator— streaming roll-ups:per_pattern(),per_model(),per_day(),top_costs(n),total_cost().CostEstimator— baseline $/1k token rates for the major model ids (Claude 4 / 3.5, GPT-5 / 4o / 4-turbo, o1, local). Override per-model rates via therateskwarg.vstack-analyticssubcommands:summary,top-costs,cost,raw,path; all support--json.
- One-command config generators for the platforms that aren't already
covered by
vstack-mcp config-snippet:cursor,cline,continue,roo-code,windsurf,zed,aider,goose,kiro,openclaw,codex-cli,opencode,docker-compose. --write+--out+--forcefor writing the snippet directly to its suggested path.
- New optional extras:
valanistack[langchain],valanistack[langgraph],valanistack[crewai],valanistack[llamaindex],valanistack[pydantic_ai],valanistack[adapters](bundles all five).valanistack[all]now includes everything. - 2 new
[project.scripts]entries:vstack-learn,vstack-analytics. - Force-include extended with
_adapters/lib,_learnings/lib,_analytics/lib; pytest testpaths extended to include the new test dirs.
- mypy strict loop now covers
_adapters,_learnings,_analyticsalongside the v0.2.0 / v0.3.0 surfaces. Loop installslangchain-core langgraph llama-index-core pydantic-aiso the framework-gated tests don't all skip in CI. - Test job runs the new test dirs and installs the same lightweight framework extras. CrewAI stays gated locally (heavier dep tree).
- Lint job covers the new dirs.
- Release smoke test imports
vstack.adapters/vstack.learnings/vstack.analyticsso a dropped force- include can't ship.
_adapters/tests/(51),_learnings/tests/(15),_analytics/tests/(15), plus 7 newgen-platformtests in_memory/tests/bring the suite to 1,895 passing (up from 1,811 in v0.3.0; +84 new).
Phase 1 of the expansion roadmap is complete. v0.3.0 lands four additional invocation surfaces alongside the v0.2.0 MCP server, plus the persistent local state store everything else can build on.
- New
~/.vstack/home directory with subdirs forbaselines/,sessions/,analytics/. Override via theVSTACK_HOMEenv var. vstack-configCLI:get/set/list/unset/path/keys/install-skillssubcommands.- 8 documented preference keys (default_mode, default_model, telemetry, log_level, preferred_llm, api_host, api_port, skills_install_path).
- Helpers exposed for downstream callers:
baseline_path_for(name),get_baselines_dir(),load_config()/save_config().
- Hits the PyPI JSON index for
valanistack, picks the highest stable version (or pre-release if--allow-prereleases), compares against the runtimevstack.__version__, and prints the matching CHANGELOG.md section as migration notes. Never execs pip; the user runs the printed install command. --jsonfor machine-readable output,--quietfor CI.- 19 new tests; HTTP is mocked at the
urllib.requestboundary so CI never touches PyPI.
- HTTP surface for every pattern. Endpoints:
/healthz,/v1/patterns,/v1/patterns/{name}(+ playbooks / citations / composition subpaths),/v1/analyze/{name}(POST). Accepts the trace either flat or wrapped in{trace, mode, model}. - FastAPI auto-generates the OpenAPI spec under
/openapi.jsonand serves Swagger UI at/docs. vstack-apiCLI:serve(default),routes,openapi.- New optional extra
valanistack[api](pulls FastAPI + uvicorn). - 13 new tests via FastAPI's TestClient.
Dockerfileat the repo root installsvalanistack[all], setsVSTACK_HOME=/var/lib/vstack, drops to a non-root user, and defaultsCMDtovstack-mcp serve. tini as PID 1 for clean signal handling.- New
.github/workflows/docker.ymlbuilds multi-arch (linux/amd64 + linux/arm64) on everyv*tag, publishes to GHCR with semver tags +latest, generates SBOM + provenance attestations.
- 7 task-shaped SKILL.md essays:
/vstack(meta router),/vstack-pick-pattern(interview),/vstack-post-incident(AAR + Lewin + downstream chain),/vstack-audit-crew(5-pattern crew health),/vstack-bottleneck(Span + Structure + behavioral),/vstack-culture-check(Schein + Robbins + optional McGregor),/vstack-baseline(calibration roundtrip). - Skills are task-shaped, not pattern-direct — each one composes multiple patterns into a real workflow.
- One-command install via
vstack-config install-skills.
vstack-mcp,vstack-api,vstack-config,vstack-upgradeall registered in[project.scripts]. Together with the 34 per-pattern CLIs from v0.1.0, every pattern + every surface is one command away.
_memory/tests/(16),_upgrade/tests/(19),_api/tests/(13) bring the suite to 1,811 passing (up from 1,756 in v0.2.0).- CI mypy loop extended with
_memory,_upgrade,_api; CI lint + test job paths extended to include the new surface dirs. - Release smoke test now also imports
vstack.mcp,vstack.memory,vstack.upgrade,vstack.apito guard against force-include drift.
First non-CLI invocation surface. vstack can now be driven from any MCP-compatible client (Claude Desktop, Cursor, Cline, Continue, ChatGPT, OpenAI Assistants, and the rest of the MCP ecosystem) in addition to direct Python imports and per-pattern CLIs.
vstack-mcpCLI —vstack-mcp serveruns a local stdio MCP server. Zero hosting cost: the user's MCP client spawns the server as a subprocess, exchanges JSON-RPC over stdin/stdout, and never reaches a network socket.- 34 tools, one per pattern —
vstack_lewin,vstack_aar,vstack_schein_culture,vstack_span_of_control, etc. Each tool's input schema is the pattern's Pydantic input model merged with two optional top-level params (mode,model); each tool's output is the pattern's detection model serialized to JSON. - Resources —
vstack://patterns/indexlists every registered pattern;vstack://patterns/<name>/citationsexposes the per- patternCITATIONS.md;vstack://patterns/<name>/playbooksexposes the failure-mode playbooks dict;vstack://patterns/<name>/compositionexposes the cross-pattern handoff manifest. - Prompts —
vstack_pick_patternis a meta routing prompt that takes a free-form problem description and recommends the right pattern + tool call. Each pattern also gets avstack_<name>_invoketemplate (35 prompts total). - LLM client resolution — env-var-driven: prefer Anthropic if
ANTHROPIC_API_KEYis set, else OpenAI, else Ollama. Override viaVSTACK_MCP_LLM=anthropic|openai|ollama|stub. A structured error response is returned when no API key is configured; the server never crashes silently. - Utility subcommands —
vstack-mcp list-tools,vstack-mcp list-resources,vstack-mcp config-snippet {claude-desktop|cursor|cline|continue|generic}for paste-ready client config.
- New
[mcp]optional dependency:pip install valanistack[mcp]pulls inmcp>=1.20.0. valanistack[all]now also includes the MCP server alongside the three LLM-client extras.vstack-mcpregistered as a new[project.scripts]entry point.
- 222 new tests under
_mcp/tests/exercising registry resolution (all 34 patterns introspect cleanly), tool / resource / prompt enumeration, full server-handler round-trips, and a stub-LLM end-to-end call through the Lewin pattern. Brings the suite total to 1,756 passing.
- New
_mcp/source folder mirroring the per-patternmodule-*/shape: source in_mcp/lib/(force-included asvstack/mcp/at wheel build time), tests in_mcp/tests/. CI's mypy loop now includes_mcpalongside the 34 patterns.
0.1.0 — 2026-05-23
First production-ready release. All 34 patterns from the roadmap ship at the 5-layer quality bar (README + library + demo + benchmark + essay).
vstack.__version__is now defined at the namespace root.py.typedmarker ships in the wheel — downstream consumers now pick up the library's type hints by default.- API stability promise documented in the top-level package docstring.
- Structured logging with run-id correlation:
new_run_id,run_context,get_logger,JsonFormatter,configure_json_logging. Patterns can correlate every log line from a single diagnostic run via a stable identifier, and ship JSON logs to structured backends without per-pattern changes. - Token / cost telemetry:
TelemetrySinkprotocol,InMemoryTelemetrySink,NullTelemetrySink,record_llm_call,time_call,set_default_sink. Default is off (Null sink); when enabled, every LLM call emits an event with model + token counts + elapsed_ms + run_id + pattern. - Prompt-injection input guards:
sanitize_for_prompt,detect_injection,fence. Strip control characters, cap absurd field sizes, detect known impersonation phrases, and provide unambiguous delimiters for fencing user content inside prompt templates. - Async LLM client adapters:
AnthropicAsyncClient,OpenAIAsyncClient,OllamaAsyncClient— same constructor surface as the sync clients, withasync def complete(...). Enables parallel pattern fan-out in server traffic. - Token-usage tracking on every sync + async client:
last_usageattribute returns anLLMUsagedataclass (input/output/total tokens + model). Cost layers can read this after each call without changing thecomplete(...) -> strreturn signature. - Configurable timeouts on all real clients
(
DEFAULT_TIMEOUT_SECONDS = 120.0). LLM calls can no longer hang indefinitely on a stalled provider.
SECURITY.mdwith vulnerability reporting policy.CHANGELOG.md(this file).- Pre-commit hooks config (
.pre-commit-config.yaml): ruff, ruff format, mypy, bandit, pip-audit. - PyPI release workflow (
.github/workflows/release.yml) — publishes to PyPI on git tag push via trusted publisher. - CI extended with: bandit security scan, pip-audit dependency scan, coverage gate, all-34-namespaces import verification in the build job.
- README badges (CI status, PyPI version, Python versions, license).
examples/cookbook/directory with cross-pattern composition examples (orchestrator + diagnostic, parallel fan-out, telemetry wiring).- Per-pattern micro-benchmark harness
(
benchmarks/_perf/run_perf_suite.py) measuring stub-client latency p50/p95 across all 34 patterns for regression detection.
Development Statusclassifier promoted from2 - Pre-Alphato4 - Beta.Python :: 3.13classifier added (already covered by CI matrix).vstack.aar.__version__synced to0.1.0.- Documentation updated to reflect the production-readiness layer.
All existing public APIs from 0.0.14 continue to work unchanged.
The new infrastructure modules are purely additive — patterns that
do not adopt structured logging, telemetry, or input guards still
function exactly as they did in 0.0.14.
0.0.14 — 2026-05-22
Closing batch. Patterns #04, #05, #07, #12 ship — completing the 34-pattern roadmap.
vstack.danva_emotion(#04) — DANVA-style emotion-recognition diagnostic. Per-emotion accuracy + intensity calibration + confusion patterns computed deterministically; LLM contributes only interventions.vstack.cognitive_reappraisal(#05) — Gross's reappraisal vs suppression vs rumination vs avoidance vs expression classifier with two LLM passes (strategy detection + interventions).vstack.hexaco(#07) — Lee & Ashton 6-factor personality fit diagnostic. H-factor (honesty-humility) risk reported separately from overall fit.vstack.vroom_expectancy(#12) — Victor Vroom's Expectancy × Instrumentality × Valence motivation calculus. Product computed deterministically in Python; LLM cannot override the math.
0.0.13 — 2026-05-22
vstack.goleman_ei(#02) — Goleman's 2×2 emotional-intelligence diagnostic (self × other × recognition × regulation).vstack.sdt_reward(#10) — Self-Determination Theory intrinsic reward diagnostic (autonomy, competence, relatedness).vstack.span_of_control(#34) — six deterministic org-structure metrics computed in Python (max_span, mean_span, centralization_index, hierarchy_depth, span_gini, decision_bottleneck).
0.0.12 — 2026-05-22
vstack.org_structure(#33) — six-dimensional org structure matrix + archetype classifier.vstack.motivation_traps(#09) — Saxberg's four motivation traps (values, self_efficacy, emotions, attribution).vstack.glaser_conversation(#21) — conversational-intelligence steering across three states × three levels.
See git history for v0.0.11 and prior batches. Roadmap kickoff at
v0.0.1 with pattern #30 (AAR generator); v0.0.2–v0.0.11 shipped
patterns #17, #18, #03, #13, #27, #20, #29, #22, #28, #01, #19, #15,
#26, #14, #24, #11, #25, #31, #08, #23, #32, #16, #06 incrementally.