Status: draft v0.1 · Owner: Hamza · Date: 2026-06-17 Builds on: PRD-local-builder.md (the three modes + the harness). One-liner: Give the local agent the same kind of tools Lovable / Claude Code / Codex have — fetch, scrape, inspect, run, diff — so it can do real tasks (starting with cleanly cloning a real website's homepage), and wrap it in a goal + eval loop that proves capability and improves itself. All one-shot public via the repo.
North-star eval: Given
https://disrupt.com/, the builder reproduces that homepage — layout, palette, fonts, sections, imagery — closely enough that a human says "yeah, that's the page."
This is the honest acceptance test. Today's builder generates from imagination; a capable builder observes the real page and reconstructs it. The gap isn't model IQ — it's missing tools + no feedback loop. Lovable/Claude/Codex feel smart largely because they can look at the world and check their work. We give the local agent the same.
A 14B asked to "clone disrupt.com" hallucinates, because it can't see the site. Give it the page's real DOM, colors, fonts, and structure as input, and the same 14B reconstructs faithfully — it's now a transcription task, not an invention task. Tools convert "invent it" into "transcribe it," which is exactly where small models are strong (the same lesson as baking in the design system). The model supplies structure; the tools supply ground truth.
| Capability | Lovable/Claude/Codex have | We add (local, sandboxed) |
|---|---|---|
| Fetch a URL | ✅ | fetch_url(url) → HTML (server-side, no CORS limit) |
| Parse / scrape | ✅ | inspect_site(url) → BeautifulSoup + a headless render: DOM tree, text, links, asset URLs |
| Extract design | ✅ (vision) | extract_design(url) → palette (dominant colors), fonts (font-family stack), spacing/radius, section outline, hero image |
| See the result | ✅ (screenshot) | screenshot(html) → render in headless Chromium → PNG |
| Check the work | ✅ | visual_diff(a,b) → similarity score + per-region deltas (clone vs original) |
| Files | ✅ | read/write/list in the project dir (have write, add read/list) |
| Run commands | ✅ | run(cmd) — approve-to-run (have it) |
| Search the web | ✅ | web_search(q) (optional, P2) |
All server-side in agent-server.py (Python already has the ecosystem: requests, beautifulsoup4, Pillow, optional playwright). 127.0.0.1-bound, origin-locked, approve-to-run, workspace-confined — the existing posture extends cleanly.
Shipped (2026-06-17):
screenshotis now a robust headless capture — Playwright's managed Chromium as the primary path (installed on demand), with a system-Chrome subprocess fallback so the zero-dependency install still degrades usefully. A newgitsynctool turns a generated project into a real local git repo (git init+ sensible.gitignore+ commit) and exports a.zipthat includes the.githistory; pushing to a remote stays user-credentialed (the tool prints thegit pushcommand, never runs it). Both are covered by deterministic, offline stress tests undertests/— runtools/test.sh. Element + full-page capture are first-class on the Playwright path.
A traceable, tool-using task — every step visible in the chat (Claude-Code style), artifacts in the panes:
1. Fetch fetch_url(disrupt.com) → raw HTML [✓ Fetched the page]
2. Inspect inspect_site() → DOM, sections, [✓ Mapped 7 sections]
assets, links
3. Extract extract_design() → palette, fonts, [✓ Palette: #0B0B0F …; Font: Inter]
layout, hero img
4. Spec reasoner writes a build spec → concrete plan [✓ Planned the rebuild]
from the extracted facts
5. Build coder builds to the spec + → HTML [◐ Writing the page · N lines]
real tokens (not invented)
6. Render preview + screenshot → PNG [✓ Rendered]
7. Diff visual_diff(clone, original) → score 0–100 [✓ 82% match]
8. Iterate if score < target: feed the → fixes [◐ Closing the gap (round 2)]
biggest deltas back, goto 5
Steps 7–8 are the eval loop made visible — the same mechanism that powers /goal (below). The user watches the score climb. This is "lean back and watch the show."
The user's ask: "set the right /goal command and eval and test cases to ensure we build it right." Two pieces:
A goal is not a vibe; it's a measurable capability statement plus the tests that prove it:
goal: clone-a-homepage
capability: "Given a public homepage URL, reproduce it faithfully."
evals:
- url: https://disrupt.com/ target: visual_diff ≥ 80, palette match ≥ 90%, all hero+nav+sections present
- url: https://linear.app/ target: visual_diff ≥ 75
acceptance: every eval passes its target on a fresh run
/goal runs the evals, scores them, shows a pass/fail scorecard, and — when something fails — captures why (the deltas) as the next work item. It turns "make it good" into a closed loop with a number.
Most goals are written badly (unmeasurable). A write-a-goal skill enforces the good shape: a concrete capability statement, ≥2 real test cases with numeric targets, an acceptance rule, and a non-goals list. (Mirrors how /spec works, but for capabilities + evals rather than features.)
The loop: define goal → run evals → fail → the failing deltas become the build backlog → fix → re-run → pass. The agent improves against the eval, not against a guess. The disrupt.com clone is the first goal; its eval is §4 step 7.
"one shot, literal one shot, then lean back and watch the show."
Everything above ships inside the repo so a brand-new user runs one command and gets the whole thing — models, builder, tools, goals:
local-llm-setup.sh --build→ detects hardware, pulls the right models, starts Ollama, installs the Python tool deps (beautifulsoup4,Pillow, optionalplaywright), launches the agent+builder, opens the browser. Zero further choices (Auto picks models).- Cross-platform parity (
.sh+.ps1), fresh-clone tested, CI-green — per the existing release discipline. - Gating reality: none of v1's routing/spec/self-repair/markdown work reaches a new user until it's baked into these scripts and released. That bake+release is P0 before any of v2 (see §8).
- Tools live in
agent-server.pyas new endpoints (/api/agent/fetch,/inspect,/extract,/screenshot,/diff), each returning structured JSON the builder feeds to the model. Reuses the CORS-lock + approve-to-run + workspace confinement. - Headless render (screenshot/diff) via
playwright(chromium) — an optional dep; degrade gracefully to "no visual diff" if absent so the one-command install never hard-fails. - Tool protocol: extend the existing
<run>/<write>text-tool convention with<fetch url>,<inspect url>, etc. — still text-based, still any-model, still approve-to-run for anything that touches the system. - The model routes: reasoner plans the rebuild from extracted facts; coder transcribes; the diff loop drives iteration.
P0 — make v1 real for new users (this is "what you need atm"). Bake routing + plan-spec + self-repair + markdown + composer/scroll/plan fixes into --build in both scripts; fresh-clone test; cut a release. Until this lands, a new clone gets none of it. Recommended first — it's literally your stated need.
P1 — the toolbelt + clone pipeline. fetch_url, inspect_site, extract_design; the §4 pipeline through step 6 (build from real tokens). Ship the first visible "cloned a page" win.
P2 — the eval loop + /goal. screenshot + visual_diff; /goal scorecard; the disrupt.com goal with numeric targets; the write-a-goal skill; iterate-to-target.
P3 — breadth. web_search, more goals, export (PDF/zip/repo), PersonalDispatch for phone, extract builder into its own public repo.
- "Pixel-perfect" is not the bar; "that's the page" is. Even frontier tools approximate. A 14B + tools gets close and recognizable; the diff score sets honest expectations.
- Some sites resist (heavy JS, auth walls, anti-scraping). The pipeline degrades to "best effort from what we could fetch" and says so — never a silent fake.
playwright/Chromium is a real dependency. Keep it optional so the one-command install stays robust; visual-diff is an enhancement, not a gate.- Respect the source: clone for learning/inspiration; don't ship someone's site as your own. A note in the UI.
- A new user runs one command and, with zero model-picking, asks "clone disrupt.com" and watches fetch → extract → build → diff → a recognizable result.
/goal clone-a-homepagereturns a scorecard with real numbers that improve across rounds.- It all lives in the public repo, fresh-clone tested, nothing leaving the machine.