Skip to content

net-rs: keepalive client tears down warm connections at 60s (StServer timeout) — recurring 60s RST #21

Description

@sandtreader

What we see

Once a dev relay promotes us to hot, the connection survives only ~60s before being torn down, then reconnects (which resets the ~15-min hot-promotion clock, so we churn and rarely stay hot long enough to do useful work). Observed repeatedly in the 2026-06-18 tx-format run against the play.dev relays:

13:34:40  peer-1  Connection reset by peer (os error 104)   (after a tx exchange + 63 keepalives)
13:35:41  peer-2/3  Connection reset by peer
13:36:44  peer-5  keepalive client exited — tearing down connection

This is the "60s RST" that #15 was thought to have closed. The KeepAlive responder is fine — serve_keepalive answered 63 keepalives on peer-1 with correct cookie echoes before the teardown. The problem is on the client side, plus likely a relay-initiated RST.

Root cause (client-side 60s)

Our keepalive client (spawn_keepalive, net-core/src/peer/peer_task.rs) pings every 20s and then keepalive::keep_alive does runner.recv().await? to wait for the relay's reply. That recv happens in the StServer state (we sent MsgKeepAlive: StClient -> StServer), and:

// net-core/src/protocols/keepalive/mod.rs
pub const TIMEOUT_SERVER: Duration = Duration::from_secs(60);
fn timeout(StServer) => Some(TIMEOUT_SERVER)   // 60s

So our client waits exactly 60s for the relay's keepalive response, then errors and tears the whole connection down (spawn_keepalive returns → duplex_task watchdog tears down).

This is the mirror of the bug #15 fixed for the responder: cardano-node doesn't run keepalive on cold/warm peers, so when the relay demotes us from hot back to warm and goes quiet on keepalive, our client hits the 60s StServer timeout and kills an otherwise-healthy warm connection — resetting the hot-promotion clock.

There may also be a relay-initiated RST

Most teardowns were Connection reset by peer (the relay RST'ing us), with leios_notify / txsubmission merely noticing the dead socket first — and several fired right after a successful TxSubmission exchange. So there are possibly two effects: (a) our 60s client timeout on warm demotion, and (b) the relay actively RST'ing us (its recycle policy, or something we send post-tx such as the TxSubmission ack accounting, newly reachable now that #17 lets us get that far).

Suggested fix

Mirror #15's cold/warm tolerance on the keepalive client: don't treat a keepalive timeout as fatal while warm (e.g. only run/await the client toward hot peers, or back off instead of tearing down). Recommend a RUST_LOG=...keepalive=debug,...txsubmission=debug run first to cleanly separate "our 60s timeout" from "relay RST" and capture the last message before each RST.

Context

Surfaced while validating #17 (txsubmission codec) — that fix is confirmed working (zero CBOR decode errors throughout), this is the next thing blocking sustained TxSubmission against the dev relays. Not a codec problem.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions