Skip to content

feat: incremental (delta) sync for mailbox refresh#7

Open
Aravinda-HWK wants to merge 1 commit into
LSFLK:mainfrom
Aravinda-HWK:feat/imap-delta-sync
Open

feat: incremental (delta) sync for mailbox refresh#7
Aravinda-HWK wants to merge 1 commit into
LSFLK:mainfrom
Aravinda-HWK:feat/imap-delta-sync

Conversation

@Aravinda-HWK

Copy link
Copy Markdown
Collaborator

What

Implements Phase 3 (delta sync) of the Email 2.0 proposal: refreshing a mailbox now fetches only what changed instead of re-listing the whole folder.

Because go-imap v1 exposes no CONDSTORE/MODSEQ, the delta is computed from UIDs + flags rather than a MODSEQ token (the proposal's documented fallback in §10).

How it works

The client sends the UIDs it already holds + its cached UIDVALIDITY; the server returns only:

  • added — full envelopes for UIDs newer than the client's highest known UID (UID SEARCH (since+1):*)
  • flags — current flags for known UIDs still present (client diffs locally to update read/unread)
  • removed — known UIDs that have been expunged/moved
  • resync — set when UIDVALIDITY changed, telling the client to discard its cache and refetch

Changes

Server

  • New GET /api/v1/mailboxes/{mailbox}/changesMailboxDelta
  • imap.MailboxChanges + shared fetchEnvelopes/fetchFlags helpers
  • ListMessages now also returns uidvalidity (baseline for the client)

Client

  • Dexie v2 syncMeta table persists uidvalidity per folder
  • mailboxes.changes() endpoint + MailboxDelta/FlagUpdate types
  • DataContext reconciles added/removed/flag changes idempotently (keyed on UID); new refreshFolder(role) and a ThreadList refresh button wired into Inbox/Sent/Trash

Performance

  • refreshFolder syncs only the viewed folder, so four folder syncs no longer serialize on the session's single IMAP connection
  • IMAPFor drops a redundant per-request NOOP probe (one RTT) — every operation already self-heals via ensureLive

Together these cut a refresh from ~4.25s (four folders contending on one connection) to a single sub-second changes request.

Tests

  • parseUIDList unit tests added
  • go build ./..., go test ./..., and frontend tsc --noEmit all pass

Notes / follow-ups

  • This is the UID+flag-diff approach (no library change). True MODSEQ-based sync would need a go-imap/v2 migration (currently beta).
  • Natural next steps from the proposal: connection pool (§7) for true parallel folder syncs, and Phase 4 (IMAP IDLE → WebSocket push).

Implements Phase 3 of the Email 2.0 proposal: refresh now fetches only
what changed instead of re-listing the whole folder.

Because go-imap v1 exposes no CONDSTORE/MODSEQ, the delta is computed from
UIDs + flags rather than a MODSEQ token:

Server
- New GET /api/v1/mailboxes/{mailbox}/changes endpoint returning a
  MailboxDelta {uidvalidity, total, resync, added, flags, removed}.
- imap.MailboxChanges: UID SEARCH (since+1):* for new envelopes, a
  flags-only UID FETCH over the client's known UIDs to derive flag
  changes + removals; UIDVALIDITY mismatch signals a full resync.
- ListMessages now also returns uidvalidity so the client has a baseline.
- Factored shared fetchEnvelopes/fetchFlags helpers.

Client
- Dexie v2 syncMeta table persists uidvalidity per folder.
- mailboxes.changes() endpoint + MailboxDelta/FlagUpdate types.
- DataContext reconciles added/removed/flag changes idempotently (keyed
  on UID) and exposes refreshFolder(role); a ThreadList refresh button
  wires it into Inbox/Sent/Trash.

Performance
- refreshFolder syncs only the viewed folder, so four folder syncs no
  longer serialize on the session's single IMAP connection.
- IMAPFor drops a redundant per-request NOOP probe (one RTT) since every
  operation already self-heals via ensureLive.

Tests: parseUIDList unit tests; server build + go test and frontend
typecheck pass.
@Aravinda-HWK Aravinda-HWK requested a review from maneeshaxyz June 21, 2026 09:31
@Aravinda-HWK Aravinda-HWK self-assigned this Jun 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant