Skip to content

Latest commit

 

History

History
133 lines (110 loc) · 41.1 KB

File metadata and controls

133 lines (110 loc) · 41.1 KB

fast-md-reader — dev notes (read before continuing)

Native macOS document viewer (reader-first), shipped as Fast Document Reader. Markdown is where it started; it also reads Word .docx/.docm/.dotx/.dotm and OpenDocument .odt (read-only) — text, tables with merged cells and cell-level images/lists, images, links (including internal bookmarks/cross-references), footnotes, headings resolved through all three ways Word and ODT can declare one, clause/list numbering with overrides, equations rendered through the app's own formula engine, charts and SmartArt diagrams (via Word's legacy-picture fallback, or an honest placeholder when none exists), right-to-left text, review comments (shown in a right panel, hidden by default), and presentation resolved from the document's OWN style cascade — paragraph spacing, indent, line-height and alignment; run/theme colour, highlight, font, size, caps/small-caps and underline style; tab stops and their alignment; paragraph shading and borders; table columns that fill the width by the document's own grid proportions, plus cell shading/borders/vertical-alignment/margins and named table styles' banded rows and shaded headers — every absolute value sized to the document's own default so the reader's zoom multiplies on top of it, not a hardcoded app rhythm. Floating/anchored image text-wrap was built, measured, and deliberately not shipped (invariant 31). Plain text (.txt/.csv/.log…) opens verbatim. .rtf was deliberately dropped (35c9485): AppKit's RTF reader loses tables, lists and embedded images, so supporting it honestly meant writing a second parser the size of the Word one. Writing is deliberate and narrow: right-click a block → Edit / Add Below / Move / Delete, held in memory until ⌘S. A headless --extract flag (FastDocReader --extract <file>) runs the SAME office reader without any GUI and prints the document as Markdown to stdout — for feeding a .docx/.odt to an AI without it spending tokens on the zip/XML (invariant 40). Pure Swift/AppKit + TextKit, SwiftPM executable. No web runtime for text. WebKit renders only two things — mermaid diagrams and TeX/KaTeX formulas — and only on a cache miss; both cache to vector PDF. Code highlighting is native (34 languages, single-pass scanner), no JS.

Build / test / run

  • Build app: ./Scripts/make-app.sh [debug|release]FastDocReader.app in repo root (ad-hoc signed).
  • Toolchain (MUST): standalone Command Line Tools has a mismatched SwiftPM ManifestAPI → swift build breaks. Always use Xcode's toolchain. make-app.sh auto-sets DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer if unset.
  • Tests: DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer swift test (721 tests, keep green; 4 skip by default). Two are measurement-only and skip unless FMD_LATENCY_FILE=<a real document> is set; one is the re-runnable corpus probe behind docs/06-research/gap-list.md's frequency figures, skipping unless FMD_CORPUS_DIR=<colon-separated dirs> is set; one is the floating-image-wrap layout-cost spike, skipping unless FMD_FLOAT_PERF=1 (see invariant 31 — that feature was measured and deliberately not shipped, and the spike stays as a test so the finding never has to be re-derived).
  • Run a dev build from the REPOopen ./FastDocReader.app. Do NOT copy it to /Applications: that folder holds properly signed builds only (owner's rule). A dev build there is ad-hoc signed, and macOS ties per-app state to the signature, so every rebuild reads as a different app — which is exactly why Open Recent kept emptying (see invariant 21). /Applications gets a build from notarize.sh or the App Store, nothing else.
  • Quit cleanly, never pkill: osascript -e 'tell application "FastDocReader" to quit'. Force-kill loses recent-docs persistence and can leave state inconsistent.
  • ⌘R (Reload) reloads the document content only — it does NOT pick up a new app build. A code change needs rebuild + relaunch.

Layout / files

  • App/AppDelegate.swift — menu bar built in code (no MainMenu.nib). Any standard shortcut or menu must be added here.
  • App/MarkdownDocument.swiftNSDocument; render pipeline, presize + prerender, lazy media reconcile.
  • App/DocumentWindowController.swift — text view, scroll, tabs, find, source-edit panel, keyboard nav, reading-line highlight, the sidebar split (NSSplitViewController) and every reflow path.
  • Navigation/ReaderTextView.swift — the reading cursor: key handling, unit navigation, gutter-click block select, reading-line band. Navigation/TextNavigator.swift — pure line/sentence/paragraph boundary math (no view). Navigation/AnchorResolver.swift — resolves an internal link's target fragment (bookmark or heading slug) to a scroll position; pure decision logic, no view, so a dead cross-reference resolving to "do nothing" is unit-testable without a window.
  • App/ExternalEditor.swift — "Edit in " for read-only office documents: finds a candidate editor by bundle identifier (never by path — an app that moves would otherwise become a dead button), remembered per format. Split into a pure, fully-tested half and an NSWorkspace/Bundle glue half verified by launching the app.
  • Render/MarkdownRenderer.swift — CommonMark/GFM → NSAttributedString; also detects mermaid, ```math and $$…$$ and turns them into web-block placeholders.
  • Render/WebBlock.swift — the ONE place that knows there are two WebKit engines (.mermaid, .math); every measure/presize/reconcile pass iterates enumerateWebBlocks. Render/WebBlockRenderer.swift — WebKit → cached vector PDF, shared by both.
  • Render/PlainTextRenderer.swift — non-markdown text (.txt/.csv/.log…) rendered VERBATIM, monospaced, one block per non-blank source line. App/DocumentTypes.swift — the single list of openable extensions (Info.plist must mirror it by hand).
  • Navigation/BlockEdit.swift — pure source arithmetic for add/delete/move (no views), feeding applySourceEdit, the one path that changes the text. App/OutlinePanel.swift — the table-of-contents sidebar (LEFT NSSplitViewItem sidebar), built from the heading attributes the renderer already leaves behind.
  • App/CommentPanel.swift — the review-comments sidebar (a RIGHT NSSplitViewItem inspector, hidden by default, ⌥⌘C), mirroring OutlinePanel; lists MarkdownDocument.officeComments. The body highlight + number badge for each comment are drawn behind the glyphs (draw-time, gated by ReaderTextView.commentsVisible), never an inline attachment (invariant 38).
  • Render/RenderTheme.swift — the BASE design system: type scale + rhythm tokens (line-height / spacing / indent ratios) + colour, inherited by three thin per-format branches MarkdownStyle / OfficeStyle / PlainTextStyle. The rhythm constants live here ONCE; renderers read tokens, never re-inline them (invariant 36). Render/Office/OfficeTextBuilder.swift — the ONE format-neutral builder turning the office block vocabulary into typography, applying each block's resolved ParagraphFormat/Span/table geometry (invariant 37).
  • Render/TableBlockBuilder.swift — turns a row/cell vocabulary (markdown GFM + office Cells) into a REAL bordered NSTextTable (GridTextTable, an NSTextTable subclass carrying its column proportions), shared by both renderers, so cell text is real selectable/copyable/searchable document text (invariant 39); resizeTables(in:toWidth:) re-solves every table's columns to absolute integer widths on each reflow. Render/SizedAttachmentCell.swift — the attachment cell that OWNS its reservedSize so an image==nil medium keeps its space (invariant 1), used by images and formulas.
  • Render/Office/OfficeMarkdownSerializer.swift — the --extract serializer: office block vocabulary → Markdown (pure [OfficeBlock] -> String, view-free), conservative mapping with a <raw> fallback for un-mappable tables (invariant 40). App/HeadlessExtract.swift — the --extract CLI itself (reads the file via the single DocumentTypes.readOffice dispatch → serializer → stdout); App/main.swift gates it BEFORE NSApplication so no GUI starts.
  • App/TextEncodingDetector.swift — what a file's bytes actually are (BOM → strict UTF-8 → BOM-less UTF-16 → regional legacy → verified guess → Latin-1), and the bytes to write back in that same encoding.
  • Render/CodeHighlighter.swift — single-pass native highlighter, 34 languages + aliases.
  • Cache/MermaidCache.swift — content-addressed disk cache, keyed by (source + engine version) so the two engines never collide.
  • Resources/mermaid.min.js, Resources/katex.min.js, Resources/katex-inlined.min.css (KaTeX fonts inlined as data: URIs — regenerate with Scripts/build-katex-css.sh). Resources/Info.plist — bundle config. Scripts/make-app.sh — build+bundle+ad-hoc sign.

Hard-won invariants (DON'T regress)

  1. Scrollbar stability = decouple media SIZE from PIXELS. An NSTextAttachment with image==nil (not-yet-loaded / purged) collapses to ~0 height → total height and the scrollbar swing on scroll. Fix in place: SizedAttachmentCell owns reservedSize independent of the image. Loading/purging pixels must redraw only, never touch layout (redrawGlyphs) — if size is unchanged, do NOT call storage.edited/ensureLayout. This cost 4 debugging rounds; keep the size/pixel split intact.

  2. Uncached docs must behave like cached ones. On first open, prerenderAllDiagrams renders every uncached web block — diagram OR formula — to the disk cache (bounded concurrency, cap=min(3,…)), then presizeKnownMedia reserves each exact area and lays out once. After that, sizes never change on scroll. Both engines ride this one pipeline via WebBlock; giving formulas their own path would mean re-earning this property (and getting it wrong = formulas resizing mid-scroll).

  3. invalidateDisplay alone does NOT draw a newly-set attachment image — you need storage.edited(.editedAttributes, …). (Learned the hard way when diagrams went invisible.)

  4. The web-block cache lives in FileManager.default.temporaryDirectory/fast-md-reader/mermaid (the dir name is historical; it holds BOTH mermaid and KaTeX PDFs now, disambiguated by the engine's cacheVersion in the key), NOT ~/Library/Caches. The app is sandboxed, so that resolves INSIDE the container — ~/Library/Containers/ai.ww-w.fast-md-reader/Data/tmp/fast-md-reader/mermaid, not the $TMPDIR your shell sees. When testing "uncached cold start", clear the CONTAINER path; clearing the shell's $TMPDIR copy looks like it worked and proves nothing. Bump the engine's cacheVersion (in WebBlock) when its capture changes — a stale PDF outlives the bug that made it.

  5. Fresh-DB/fresh-open tests are fake-green for the cache/migration class — they create the target state directly and skip the render path. Cold-start behavior must be tested by actually clearing the mermaid cache dir and relaunching.

  6. NSDocumentClass in Info.plist must be module-qualified: FastDocReader.MarkdownDocument (a SwiftPM executable, so NSDocumentController can't find a bare class name).

  7. Open Recent: AppKit's automatic population does NOT attach to a code-built menu → it's populated manually via a menu delegate in AppDelegate reading recentDocumentURLs. macOS prunes deleted files from the list (a deleted test file → correctly shows "No Recent Files").

    • Read recentDocumentURLs ONLY from menuNeedsUpdate (i.e. when the user opens the menu). Under the sandbox each read can consume a limited pool of security-scoped extensions; apps that poll it (timers, validateMenuItem) exhaust the pool and then fail to reopen recent files. Apple's published workaround — cache the list, refresh on activation — would make it worse here, since that fires on every ⌘Tab. On-demand-only is already the safe pattern; verified working sandboxed across repeated relaunches. If recent files ever stop opening, this is the first suspect.
  8. Sandboxed WKWebView needs com.apple.security.network.client even though mermaid loads only a bundled local JS file and the app makes zero network requests — WebKit runs out-of-process and renders nothing without it. Don't "clean up" that entitlement.

  9. Sandboxed = a document's sibling images are unreadable, and macOS NEVER prompts. Measured, not assumed: relative, absolute, and file:// paths all fail sandboxed while remote URLs load; the same binary unsandboxed loads them all. The sandbox denies before TCC is consulted, so no prompt appears, and there is no "Documents folder" entitlement to request (iMovie's Documents/Desktop/Downloads toggles come from its media entitlements — not a route we have). Granting Full Disk Access does nothing: the sandbox still refuses. The only way through is FolderAccess — user picks the folder in an open panel, persisted as a security-scoped bookmark (what Marked 2 / iA Writer / MWeb do; Typora / Obsidian / IntelliJ skip the store instead). Don't re-litigate this; four plausible theories (temp path, ad-hoc signature, TCC identity, launch method) were each disproved.

  10. A blocked image's placeholder is itself an image, so its click check MUST come before the zoom check in ReaderTextView.mouseDown — otherwise clicking "Click to allow…" just enlarges that label.

  11. Remote images are presized by fetching only their header (Range: bytes=0-65535 → ImageIO), mirroring prerenderAllDiagrams: measure everything, then lay out ONCE. Never "reserve a guess and correct it later" — that reflows under the reader, which invariant 1 exists to prevent.

  12. Math is read from the SOURCE, never the parsed text. MarkdownRenderer.scanMathSpans finds $$…$$ in the raw markdown before the parser touches it. Two reasons: (a) markdown claims _/^ as emphasis, so $$a_1^2$$ parses to a12 — silently wrong maths; (b) a \\-then-= line inside a matrix is read as a setext heading, so the parser shreds one formula across several nodes. A block is turned into a formula only if it lies ENTIRELY inside a scanned span. ```math fences are verbatim, so they skip the dance.

  13. KaTeX fonts are inlined as data: URIs in katex-inlined.min.css. The render page loads with baseURL: nil, so a relative url(fonts/…) resolves to nothing and every glyph silently falls back to a system font at the wrong metrics. Inlining removes the resolution step. Snapshot only after document.fonts.ready — KaTeX lays out synchronously but the fonts arrive async, and snapshotting early bakes fallback metrics into the cached PDF forever. The fonts are OFL-1.1, separate from KaTeX's MIT codelicenses/KaTeX-fonts-OFL-1.1.txt + notice in THIRD-PARTY-NOTICES.md are required, and make-app.sh copies both into the bundle (the licence must travel with the fonts).

  14. The reading-line band repaints the whole visible rect on every selection change (setSelectedRange override → setNeedsDisplay(visibleRect)). A bare caret move only invalidates the thin caret sliver, so the band would smear — lit on the old line, dark on the new. Drawn in drawBackground beneath glyphs, and only when the selection is empty (an active selection is its own stronger highlight).

  15. Keyboard nav: fn+arrow arrives as Home/End/PageUp/PageDown (keyCodes 115/119/116/121), NOT as a bare arrow. Matching a bare arrow (123/124) for the paragraph keys was a real bug — it made plain ←/→ jump a paragraph instead of moving one character. Vertically the scheme is two-tier by heading rank: down the doc fn(top-level # heading only, the SHALLOWEST rank present via topHeadingNav) › (every heading, headingNav) › (page); fn+⌘ (keyCode 116/121 WITH .command, so the [.command] cases must precede the catch-all _) jumps to document start/end. Across the line fn(paragraph) › (sentence) › (line). ⇧ extends. Unit boundaries come from TextNavigator; paragraph/sentence stops also honour block starts (MDAttr.blockId) so a heading is never skipped. The page holds still and the cursor moves inside it — scroll follows only when the caret would leave the screen (revealCaret), never top-anchored. The bare single-letter block keys (E/I/D/U/J/T) match by keyCode (e=14·i=34·d=2·u=32·j=38·t=17), NOT by character — a non-Latin input source (Korean, …) turns e into and silently broke every bare-letter shortcut; keyCode is layout-independent. (⌘-modified equivalents already fall back to the Latin layout, so only the bare keys needed it.)

  16. Every block operation goes through applySourceEdit — the single mutation path. Add/delete/move each compute ONE (range, replacement) pair in BlockEdit and hand it over, so re-render, cursor reveal and undo are earned once and nothing can half-apply. A move is deliberately length-preserving (block A + the untouched gap + block B, reordered). Modeless popups and alert sheets re-resolve the block at confirm time — offsets move under them.

  17. Nothing reaches disk until ⌘S. applySourceEdit changes memory and marks the document dirty; data(ofType:) is the only writer. Change tracking is left to NSDocument (it watches the undo manager), so undoing back to the original correctly reports clean — note the undo group closes on the NEXT run-loop turn, so a test asking immediately is asking too early. ⌘R warns before discarding unsaved edits.

  18. A file is written back in the encoding it was READ in. String(decoding:as:UTF8.self) never fails — it substitutes replacement characters and reports success — which is how a Windows CP949 file arrived as a wall of "?". Use TextEncodingDetector, and accept a guessed encoding only when re-encoding reproduces the original bytes exactly (the system's own guess for arbitrary input is Windows-Cyrillic, which doesn't reverse). Text the file's encoding can't represent is REFUSED with an explanation, never written lossily.

  19. An edit re-renders only the blocks it touched (spliceRender), not the document. Full re-render measured 131ms on a 64k-character file and over a second on 1.2MB — paid again on every undo; the splice is 9–29ms. It is safe because a block renders the same alone as in context (verified per block kind in FragmentRenderTests); reference-style link documents are the exception and take the full path. Two things it must keep doing: lift the fragment's block ids clear of existing ones (fragments number from zero, and two neighbours sharing an id merge into one stop for the reading cursor), and run the fragment to where the NEXT block starts so the separator that follows isn't spliced away. SpliceRenderTests asserts the result is identical to a full render after every operation — a divergence there is invisible on screen.

  20. In plain text a block is a LINE, blank ones included; in markdown it is a paragraph/heading/table/fence. A blank line in a .txt is spacing the author typed, not structure — so it must be selectable, deletable and movable, and "add below" inserts exactly one line (the file's own ending, CRLF preserved) instead of copying the neighbouring gap. Because a blank line is nothing but its terminator, a plain-text block's RENDERED range includes that terminator while its SOURCE range stays content-only. PlainTextRenderer must also keep output.string == source exactly — a dropped trailing newline is a silent data change on the next save.

  21. Open Recent emptying after a rebuild is the AD-HOC SIGNATURE, not the code. Measured 2026-07-20: the list survives quitting and relaunching the same binary, and disappears only when a freshly built bundle replaces it. macOS keeps recents in a SharedFileList keyed to the app's signing identity, and an ad-hoc signature's cdhash changes on every build, so each build looks like a different app. A Developer ID / App Store build has a stable identity and keeps its list across updates — don't "fix" this in code. (The store itself is TCC-protected, so a shell can't read it to check; defaults read ai.ww-w.fast-md-reader won't show recents either.)

  22. Save is greyed out unless CFBundleTypeRole is Editor. With Viewer, AppKit decides the app cannot write that type and disables saveDocument: — while save(nil) called directly still writes the file, so the save path looks fine and only the menu is dead. The two text entries (markdown, plain text) are Editor; the two office entries (Word, OpenDocument) are deliberately Viewer, which is what greys Save out for the read-only formats — there the disabled menu is the intended behaviour, not the bug. The close-and-save prompt works either way, which makes this easy to misread as "saving is broken".

  23. The table of contents is rebuilt from BOTH render paths. spliceRender changes headings as surely as a full render does (## typed into a block, a section moved, a heading deleted), and for a while only the full path called reloadOutline() — so the sidebar quietly described the document as it was several edits ago. Anything that changes the text must end at reloadOutline().

  24. Reflow is deferred to the END of a resize / sidebar animation, and the reading position is restored by ANCHOR. Re-wrapping a long document per animation frame is what made both feel like they were fighting the user (suspendReflow). Two things must stay true afterwards: lay the whole document out BEFORE scrolling (a narrower column makes it taller, and until that layout exists the scroll is clamped to the old, shorter height — which is why position held when widening and drifted when narrowing), and restore via readingAnchor()/restore(_:), which keeps the CURSOR's line at its old height if it is on screen and otherwise puts the centre character back at the centre. A plain scroll offset means different text once the wrapping changed.

  25. A spinner must be shown BEFORE the work and forced to draw. setBusy(true) + spinner.display(), then the work on the next run-loop turn. A delayed show scheduled with asyncAfter can never fire: the work owns the main thread, so the timer's turn only comes after the work finished and cancelled it — that version shipped and showed nothing. Gated on document size (>120k chars) so short documents don't flash.

  26. A control next to the traffic lights is a TITLEBAR ACCESSORY, not a toolbar item. This macOS lays toolbar items out trailing with the title leading, so a toolbar button lands on the far right no matter how the identifiers are ordered — .flexibleSpace doesn't move it, and neither does asking for the system .toggleSidebar. NSTitlebarAccessoryViewController with layoutAttribute = .leading puts it where every Mac app keeps it. Centre it with a constraint; the title bar's height is not ours to guess.

  27. The sidebar is a real NSSplitViewItem(sidebarWithViewController:). That is what supplies the inset rounded panel, the system material and the light/dark behaviour. An NSVisualEffectView of our own underneath a real sidebar just hides it.

  28. A local build carries its OWN bundle identifier — ai.ww-w.fast-md-reader.dev — and must keep doing so. make-app.sh rewrites CFBundleIdentifier and the display name (the Dock shows "Fast Document Reader (Dev)") in the COPY inside the bundle; Resources/Info.plist keeps the real identifier. macOS keys per-app state to the identifier, so while they were shared, every local rebuild took the installed release's Open Recent list with it — invariant 21 explains why a rebuild invalidates it, and a shared identifier is what let that reach an app nobody rebuilt. notarize.sh and appstore.sh pass DIST_IDENTITY=1 to keep the real identifier and then verify it survived, refusing to ship a .dev bundle: a flag that protects only when every future caller remembers to set it is not protection. Consequence when testing a cold start — a dev build's sandbox container is …Containers/ai.ww-w.fast-md-reader.dev, so invariant 4's cache path shifts with it.

  29. A unit test on a parser cannot tell you the parser is REACHED. .odt shipped registered, parsed and covered by 24 passing tests — and could not be opened at all: both office call sites hard-coded DocxReader, so ODF bytes went to the Word parser and threw. Every OdtReaderTests case called OdtReader.read directly, which proves the parser and says nothing about the application. 266 tests were green with the feature completely dead, and only launching the app found it. Two rules follow. DocumentTypes.readOffice is the single dispatch table — one place that maps an extension to its reader, throwing on anything unhandled rather than falling through to Word; a second, divergent list is how this recurs. And any new format needs one test through MarkdownDocument's own read path, not just through its parser, plus an actual open in the built app. The same blindness applies to Info.plist (invariant 6) — both are seams no compiler and no unit test can see.

  30. Measure the corpus before believing it. All eight contracts ship word/footnotes.xml, which read as "footnotes matter here" and ranked a whole sprint — but every one holds ONLY Word's separator/continuationSeparator boilerplate, which Word writes into nearly every document. Real footnotes: zero. Likewise the claim that filtering those separators is what prevents phantom notes: removing the filter changes nothing, because only CITED notes are ever appended — the filter is a second layer, not the load-bearing one (confirmed by mutation). When a check passes, break the thing it checks and confirm it fails; a green assertion whose subject is unreachable proves nothing. Two real defects this session were found by exactly that mutation step, and one "verified" claim was withdrawn because of it.

  31. Setting an NSTextAttachment's .image silently discards a custom attachmentCell the moment it's set — AppKit switches to its own bounds-based image layout at that point, replacing whatever cell was there. Found by experiment while building the chart/SmartArt placeholder (OfficeTextBuilder.appendUnsupportedGraphic), which is exactly why that placeholder sizes itself through .bounds alone rather than through invariant 1's SizedAttachmentCell — there is no "not yet loaded" state for a placeholder whose image is never nil to begin with, so nothing can revise its size after the fact.

  32. NSTextContainer.exclusionPaths invalidates the WHOLE container's layout — AppKit does no partial invalidation. Real text-wrap around a floating image was built and measured (FloatWrapExclusionSpikeTests, kept as a reproducible spike): N floating images cost N sequential full re-layouts, measured 45ms with none vs. 301ms with twelve on a 400-paragraph document, and it isn't paid once — resizing the window or toggling the sidebar re-runs the whole pass on top of the reflow already happening there. Under the owner's "light and fast is the core" constraint this was measured, found too expensive, and deliberately not shipped; floating images still render inline, on their own line.

  33. A style's ID is not localised even though its NAME is. The heading reader's own comment argued from a true premise — a Korean Word install shows a style named 제목 1, so matching by name is unsafe — and drew the wrong conclusion from it: that built-in heading styles couldn't be recognised at all. The id (Heading2) stays the same in every locale even when the name doesn't, and skipping it left 103 of 188 heading-bearing documents rendering with zero headings. A true premise can protect a false conclusion — the fix is to check what the premise actually rules out, not everything nearby it.

  34. Measuring document STRUCTURE is not measuring reader BEHAVIOUR, and when they disagree, behaviour wins. An XML-grep prediction of the same heading gap said 36% of documents render flat; running the reader itself through its own read path said 55% (103/188). The grep undercounted because it treated the mere presence of some outline-capable style in styles.xml as protection, when what matters is whether the styles the document's headings actually use resolve to one — only executing the real code path can see that. The grep number was briefly recorded as a "correction" of the higher one; it was the wrong turn, not the fix (docs/06-research/gap-list.md keeps both, visibly, on purpose).

  35. FileManager.enumerator stops silently at the first unreadable subdirectory unless given an explicit error handler. The corpus probe (CorpusProbeTests, invariant 30's tool) under-counted its own input by about a dozen files with no indication anything was skipped — the exact silent-truncation failure this project's working style forbids, caught only because the probe's own count was checked against a second method. Fixed by passing an error handler that counts skips and continues rather than aborting the walk.

  36. The reader's rhythm lives in ONE base design system, never re-inlined per renderer. RenderTheme carries the base tokens (type scale + line-height/spacing/indent ratios + colour); MarkdownStyle/OfficeStyle/PlainTextStyle are thin branches that inherit it and override only what their format needs. The rhythm constants were once duplicated across MarkdownRenderer, OfficeTextBuilder, TableBlockBuilder and PlainTextRenderer (a change in one never reached the others); hoisting them was proven byte-identical by RenderThemeParityTests, a parity harness that snapshots every layout-affecting attribute. Markdown output must stay identical when a token moves — re-inlining a magic number is the regression that harness exists to catch.

  37. Office paragraph/run/table formatting comes from the DOCUMENT's own styles, resolved into ParagraphFormat/Span/table columnWidths and applied by the one format-neutral OfficeTextBuilder — and "the document said nothing → the theme token, byte-identical" is the load-bearing contract. docx resolves docDefaults → w:basedOn chain (cycle-guarded) → direct w:pPr per property; ODF resolves the style:parent-style-name chain; both feed the SAME builder, and absolute points ride the same fontSizeScale (reading-size ÷ document-default) that font size does — .multiple line-heights are ratios and don't scale. An office change that alters output for a document declaring NOTHING is a regression (every design-system sprint proved its unspecified case identical). Tables fill width from w:tblGrid / style:rel-column-width kept as PROPORTIONS and re-solved to ABSOLUTE INTEGER point column widths at the reading column (invariant 39) — never percentages fed to NSTextTable (it recomputes those per row → a merged row's boundary drifts a pixel, see invariant 42) and never a raw twip as an absolute width. Named table styles (w:tblStyle + w:tblStylePr + w:tblLook) resolve per-cell UNDER the direct props: cell-direct > table-direct > table-style > theme. Paragraph shading/borders draw via the CodeCardLayoutManager block-background path (draw-time, NOT exclusionPaths).

  38. Comments are captured but drawn at DRAW-TIME, gated by the panel. docx word/comments.xml and odt office:annotation become MarkdownDocument.officeComments + Span.commentIds on the commented spans (never merged away, like bookmarks — and the office read contract is OfficeReadResult{blocks,comments}, one dispatch, invariant 29). The right comment panel is an NSSplitViewItem inspector (hidden by default); when open, the commented spans get a faint band + a number badge painted BEHIND already-laid-out glyphs (like the reading-line band), never an inline NSTextAttachment — so toggling the panel sets a flag + setNeedsDisplay and MUST NOT reflow the document. A comment-free document is byte-identical and the menu item is disabled.

  39. Tables are a REAL NSTextTable, so cell text is selectable, copyable and searchable — with columns pinned to ABSOLUTE INTEGER point widths, never percentages. TableBlockBuilder lays every cell into a GridTextTable (an NSTextTable subclass that remembers its column PROPORTIONS) as ordinary paragraphs carrying an NSTextTableBlock, so the text is part of the document — which is the whole point: a custom-drawn attachment, however crisply aligned, is a picture the reader can't select, copy or ⌘F. Columns are set as absolute integer point widths (cumulative ROUNDED edges of proportion × reading-column width) — NOT percentages: NSTextTable recomputes a percentage per row, so a column boundary lands on a slightly different fractional pixel in a 4-cell row than in a span-merged one (the "열이 살짝 어긋남" drift). DocumentWindowController.resizeTableColumnsTableBlockBuilder.resizeTables re-solves those absolute widths to the current reading column on every reflow. Because cell content (INCLUDING image/diagram/formula attachments) now lives in the TOP-LEVEL storage as ordinary paragraphs, the existing top-level media passes (presizeKnownMedia/prerenderAllDiagrams/reconcileMedia/measureRemoteImages) size and paint cell media natively — there is no per-table descent any more. History (don't rebuild it): an earlier custom-drawn engine (TableAttachmentCell + TableGeometry + CellText, added c1173cb, removed 2026-07-24) computed its own x-edges to kill that drift, but the drift was a wrong-algorithm choice (percentage widths), NOT an NSTextTable limitation — absolute integer widths fix it while keeping real text, and cost far less code. MarkdownRendererTests/OfficeTextBuilderTests inspect the NSTextTableBlocks (block span/shading/border + GridTextTable.columnProportions + cell text in the top-level string).

  40. --extract is a HEADLESS branch of the same binary, gated in main.swift BEFORE NSApplication. FastDocReader --extract <file> prints Markdown to stdout and exit()s without ever constructing NSApplication (no window, no Dock icon, no AppDelegate) — the arg check must stay first in main.swift or the GUI spins up. Its purpose is token economy for an AI reading a .docx/.odt: it reuses the app's OWN office reader via the SAME DocumentTypes.readOffice dispatch (invariant 29), then OfficeMarkdownSerializer ([OfficeBlock] -> String, pure/view-free, so it is unit-tested without AppKit). The serializer's contract is conservative + honest: map to real Markdown only when unambiguous, and degrade anything a GFM pipe table can't hold (merged cells via colSpan/rowSpan ≠ 1, or non-.paragraph block content in a cell) to a <raw>…</raw> plain-text dump rather than a fabricated grid that reads as correct — a wrong table is worse than none for the AI consuming it. Escaping is deliberately minimal (only | inside a pipe cell, newlines→spaces); an AI tolerates messy text but not a mis-structured table. stdout is ONLY the document (errors → stderr, exit 1 read/parse · 2 usage), so a script can trust the pipe. Coverage: docx family + odt (the office readers' scope — legacy .doc/.rtf are unsupported, same as the reader) plus md/txt verbatim (decoded through TextEncodingDetector, invariant 18). Proven both as pure serializer units AND through the real dispatch (OfficeDocumentTests.testHeadlessExtractSerializesThroughTheRealOfficeDispatch) — a serializer unit test alone can't prove the reader FEEDS it (invariant 29's lesson again).

  41. A markdown feature can ship DEAD, and only rendering it proves otherwise — MarkdownFeatureAuditTests renders every CommonMark+GFM feature through the real MarkdownRenderer.render path and asserts its evidence attribute. Strikethrough (~~x~~) shipped for a long time producing PLAIN text: the parser emits a Strikethrough node (GFM is on — tables prove it), but inlineFragment had no case for it, so it fell through default to unstyled children while the README advertised it. GFM task-list [ ]/[x] had the same fate — swift-markdown consumes the box into ListItem.checkbox, and renderList ignored it, so the ticks vanished entirely. Both are fixed (strikethrough → .strikethroughStyle over the inner span; checkbox → / glyph in place of the bullet). The lesson is invariant 30/34's: a parser unit test (or a grep of the visitor) can't see an unREACHED case — only the render path can, so any new inline/block support MUST add an audit assertion. Footnotes ([^id]) are deliberately NOT rendered as footnotes yet — swift-markdown does not parse them, so a real implementation must source-scan like scanMathSpans (superscript refs + a collected section + srcRange/blockId wiring), tracked as a deferred unit; README must not claim footnotes until then (testFootnotesAreNotYetRenderedButContentSurvives characterizes the current preserve-but-don't-render behaviour).

  42. When a native API "doesn't align / is slow", suspect your INPUTS before reimplementing it. Two rounds of this on tables, both resolved by a one-line input choice, not a custom engine: (a) alignment — feeding NSTextTable PERCENTAGE column widths let it recompute them per row so a boundary drifted a fractional pixel; the "fix" was a whole custom-drawn engine, when the real fix was ABSOLUTE INTEGER widths (invariant 39). The source already stored absolute twips — WE threw the precision away by converting to percent. (b) speed — that custom engine measured each cell with NSAttributedString.boundingRect, which is ~O(n²) in char count on rich CJK text (a 1000-char patent cell: 53ms; its first 100 chars: 0.8ms — a 10× length for a 67× time), making a 13-table document crawl at ~750ms/reflow; a reused NSLayoutManager was O(n) (~4.6ms), but the deeper truth is the engine shouldn't have existed. Both the drift and the O(n²) were self-inflicted by the workaround. NSTextTable with absolute integer widths gives selectable/copyable/searchable text AND alignment AND native incremental layout, in far less code. The general rule: percentage-vs-absolute, twip-vs-point, and the layout API you pick are load-bearing — a "native API is inadequate" conclusion should be the LAST resort, after the inputs are proven right.

Debugging discipline (this app specifically)

  • Synthetic scroll is blocked by accessibility — CGEvent scroll doesn't reach the window. You cannot drive scrolling programmatically; the user reproduces, you read logs. Log total height / frame height / scrollY per reconcile and read them.
  • Temporary instrumentation goes to /tmp/fmd-*.log. Remove all of it (delete DebugLog.swift, strip log calls) before committing and clean /tmp/fmd-*.
  • Verify visual/pixel behavior with a screenshot only when the judgment is truly visual; deterministic size assertions are proven by the logs/code, not screenshots.

Commit / distribution

  • Solo local app → commit directly to main (established pattern: a80271e, 57b485b, bce0ead). No dev branch. Stage by filename (exclude .bkit/).
  • docs/ is gitignored — local only, NOT on GitHub (this repo is public and the AI collaboration logs quote internal discussion verbatim; GitHub can't keep a folder private inside a public repo). The distribution playbooks (docs/NOTARIZATION.md, docs/APP-STORE.md, docs/APP-STORE-METADATA.md) live there too, so they exist on this machine only — don't link to them from README (public readers would hit a dead path), and don't assume a fresh clone has them. (The .gitignore line is now docs/, not just docs/commit-log/ — for a while only commit-log was ignored and the rest was merely untracked, one git add . from being public.)
  • demo/ and licenses/ ARE public (committed): demo/ = the four sample docs shipped for users (code-blocks, math, images, moby-dick — Moby-Dick is public-domain, PG boilerplate stripped; demo photos are PD/CC0, credited in demo/assets/CREDITS.md). licenses/ = the OFL text the fonts require.
  • Two distribution tracks, both arm64-only, both from this SwiftPM build — no Xcode project:
    • Direct (./Scripts/notarize.sh) — Developer ID + hardened runtime + Apple notarization → stapled FastDocReader.zip. Unsandboxed. Recipients double-click; no quarantine step. Run it only when shipping a build to someone, not per build. → docs/NOTARIZATION.md (local)
    • Mac App Store (./Scripts/appstore.sh) — Apple Distribution + embedded profile → signed .pkgaltool. Sandboxed (the store's price). Defaults to validate-only; --upload submits. The store notarizes during review, so this track never calls notarytool. → docs/APP-STORE.md (local)
  • The builds differ ONLY in the sandbox, deliberately (invariant 9) — don't "unify" them.
  • Signing identity / key ids are NOT in the repo — scripts source $KEYCHAIN_DIR/signing.env (default ~/Documents/DEV/ww-w-ai/.keychains/, chmod 600) and fail loudly naming the missing variable. Never re-add them as script defaults.
  • make-app.sh alone is ad-hoc signed — runs on the building machine only. Never ship that bundle. It matches the direct build (unsandboxed); SANDBOX=1 ./Scripts/make-app.sh builds the sandboxed shape to exercise the folder grant.
  • A build carrying the App Store entitlements won't launch locally (RBS error 163, "Launchd job spawn failed") — those identifiers require store installation. Hence Resources/FastDocReader.entitlements = same sandbox, no store identifiers, local testing only.
  • Intel support needs a universal build (swift build --arch arm64 --arch x86_64) before packaging.