Skip to content

Log dedup: pending repeat summary is never flushed in non-TTY mode #9

Description

@smolgroot

Context

The log formatter (src/logger.rs) collapses consecutive identical log lines:

  • TTY: the line is redrawn in place with an (xN) counter.

  • Non-TTY (log files, journald, pipes): repeats are suppressed, and a summary line is emitted when the next different line arrives:

    22:41:12 INFO  blocked domain=collector.github.com. source_ip=127.0.0.1
    22:43:46 INFO  blocked domain=collector.github.com. source_ip=127.0.0.1 (repeated 411 more times)
    

Problem

In non-TTY mode, the summary is only written when a new, different event is logged. If the process exits while still receiving repeats — or if nothing different is ever logged — the final (repeated N more times) summary is lost. Those queries are still counted in server stats, but the log undercounts what happened.

Example: a client hammers a blocked domain, then the service is stopped. journald shows only the first blocked line; the N suppressed repeats are never accounted for.

Possible fix

The formatter only runs on new events, so it can't flush on its own. Options:

  • A small background task (e.g. tokio interval) that flushes the pending dedup state if it's been idle for a few seconds.
  • Additionally flush on graceful shutdown (the server already logs DNS server stopping..., which happens to trigger the summary — but only if shutdown is graceful).

The dedup state lives in CharmFormatter behind a Mutex<DedupState>; it would need to be shared with the flusher, and the flusher needs to write through the same stdout path to avoid interleaving.

Notes

  • TTY mode is unaffected (the counter is redrawn live on every repeat).
  • Low severity: only affects the trailing burst of repeats before exit/idle.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions