Skip to content

[task] Research: Analyze validator node logs on mainnet to understand period block time slowdowns #2659

Description

@Unique-Divine

Mainnet block intervals have drifted upward since roughly April 2026. Average time between commits sits closer to 2.6 seconds today, while many individual blocks still land near the nominal ~1.5 second target. That gap means the chain is not uniformly slow—something episodic is stretching a subset of blocks and pulling the average up.

Those episodic stretches matter for more than operator dashboards. Block timestamps feed oracle freshness, EVM block.time, IBC timeouts, and any product logic that assumes a steady cadence. When intervals widen unpredictably, downstream systems see stale prices, delayed finality perception, and harder capacity planning. Validators also feel it in missed proposal windows and noisier alerting.

Evidence from archive and fullnode observer logs already suggests one plausible class of cause: extra consensus rounds, especially RoundStepPropose timeouts when the round-0 block proposal arrives late or not at all. That pattern can inflate wall-clock block time without requiring slow local execution on every node. What remains unclear is why proposals go missing or arrive late on mainnet—whether the bottleneck sits with specific proposers, network propagation, mempool selection, or something else visible only from the validator that was scheduled to propose.

This issue tracks research into that proposer-side gap. The goal is to learn, from validator node logs on mainnet, what happens during periods when block times slow down: which heights spike, which validator held proposer duty, whether timeouts or round advances appear, and whether those events cluster in time or across operators. Answers should separate “consensus waited” from “execution took longer,” because each points to different owners and follow-ups.

Success here is a clearer picture of the slowdown mechanism on production—not a fix proposal. Useful outcomes include a concise characterization of spike behavior (frequency, duration, correlation with proposer identity), confirmation or rejection of leading hypotheses from observer-only data, and a short list of open questions that still need proposer-side telemetry. That baseline lets engineering decide whether the next step belongs in consensus tuning, validator operations, application work, or further instrumentation.

Metadata

Metadata

Assignees

Labels

S-triageStatus: This issue is waiting on initial triage. More Info: https://tinyurl.com/25uty9w5

Type

Projects

Status
⚡ Building 🧱

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions