Size the UDP receive buffer

Start with explicit assumptions

For illustration, use 5,000 packets each second with mean payload bytes of 1,200. Replace both figures with receiver measurements. A receive queue must cover the period during which the application cannot drain its socket, called the stall budget. Include scheduler delay, stop-the-world pauses in linked runtimes, short CPU contention, and worker catch-up after a burst.

For 250 milliseconds at the assumed rate:

packets = 5,000 packets/s × 0.250 s = 1,250 packetspayload = 1,250 packets × 1,200 bytes = 1,500,000 bytesoverhead allowance = 1,250 packets × 512 bytes = 640,000 bytesburst multiplier = 4required queue = (1,500,000 + 640,000) × 4 = 8,560,000 bytesround up = 16 MiB requested SO_RCVBUF

The 512-byte allowance is a conservative planning value for socket-buffer accounting, packet metadata, alignment, and variance. It is not a Solana wire constant. Linux charges receive memory by internal allocation size rather than UDP payload length alone, and the exact charge depends on kernel, architecture, and driver path. Validate it with ss -u -m and live traffic.

A request of 16 MiB covers roughly 2.8 seconds of payload bytes if no overhead is counted, or about 2.0 seconds using 1,712 charged bytes per packet. After applying the four-times burst assumption, it represents about 490 milliseconds at that stressed packet rate. This exceeds the original 250 millisecond stall target with rounding headroom.

Set both the host ceiling and socket request

Linux applies net.core.rmem_max as the ceiling for an ordinary SO_RCVBUF request. Set it above the requested value. For internal bookkeeping, Linux doubles the request and returns that larger figure through getsockopt. A 16 MiB request therefore needs at least a 16 MiB ceiling and normally reads back near 32 MiB.

sudo sysctl -w net.core.rmem_max=33554432sudo sysctl -w net.core.rmem_default=4194304sysctl net.core.rmem_max net.core.rmem_default

Persist the settings through the host's sysctl configuration system. rmem_default affects sockets that do not set their own buffer. The receiver should always set SO_RCVBUF, so rmem_max is the important ceiling. Avoid making the global default enormous because every UDP socket can consume receive memory.

Use socket2 and verify the effective value:

use socket2::{Domain, Protocol, Socket, Type};use std::io; fn make_socket(requested: usize) -> io::Result<Socket> {    let sock = Socket::new(Domain::IPV4, Type::DGRAM, Some(Protocol::UDP))?;    sock.set_recv_buffer_size(requested)?;    let effective = sock.recv_buffer_size()?;    if effective < requested {        return Err(io::Error::other("receive buffer capped below request"));    }    let _verified_readback = effective;    Ok(sock)} fn main() -> io::Result<()> {    let _socket = make_socket(16 * 1024 * 1024)?;    Ok(())}

Because Linux usually doubles the value, checking effective < requested detects a severe cap but does not prove exact success. Record both values and alert when the readback changes across deployments.

Separate kernel and application buffers

SO_RCVBUF protects only the interval before the receive system call copies a datagram into user space. Once received, the datagram occupies an application object or queue. Size that queue independently in packets, not bytes alone.

At four times the example rate, 250 milliseconds is:

5,000 × 4 × 0.250 = 5,000 packet slots5,000 × 1,228 maximum bytes = 6,140,000 bytes of packet storage

Round to 8,192 slots if the queue implementation benefits from a power of two. Include object metadata and allocator overhead in the memory budget. A bounded queue gives a precise overload signal. An unbounded queue trades visible packet loss for growing latency and eventual memory failure.

Do not add kernel and application capacity and claim the sum as one clean stall budget. The receive thread must run to move packets from the kernel into the application queue. If that thread is descheduled, only SO_RCVBUF helps. If workers are slow while the receive thread runs, the application queue helps.

Measure the real burst distribution

Count datagrams in 1 millisecond, 10 millisecond, 100 millisecond, and 1 second windows. Retain the maximum and high percentiles. Feed production is not uniform. Coding shreds and data shreds can arrive close together, and network queues can compress inter-arrival gaps.

Increase the multiplier when the measured maximum exceeds four times the long-window mean. Decrease it only after observing busy periods and verifying zero RcvbufErrors. A large receive buffer does not repair a receiver that is continuously slower than the feed. It delays loss and increases the amount of stale data waiting in the kernel.

Bound latency as well as loss

Timestamp immediately after receive. Export the age of the oldest application-queued datagram. If that age exceeds the strategy's usefulness window, drop stale work intentionally and continue at the head. For raw shred consumers, processing seconds-old data while current packets are being discarded is usually the wrong failure mode.

Run a stall injection test. Under representative traffic, pause reception for four durations: 50, 100, 250, then 500 milliseconds. Observe SO_RXQ_OVFL, Udp:RcvbufErrors, and application sequence gaps. Accept the configured buffer only when it covers the stated target on the deployed kernel and host.