Skip to content

Tokio, epoll, recvmmsg

Solana shred receiver in Rust with Tokio and recvmmsg

Build a Linux UDP receiver with Tokio readiness, SO_REUSEPORT, preallocated recvmmsg batches, source filtering, and complete datagram reads.

Download the source

What this implementation does

The downloadable crate binds the verified IPv4 destination port, asks Linux for an 8 MiB receive buffer, enables SO_REUSEPORT, and waits for socket readiness through Tokio's AsyncFd. On readiness it drains up to 32 datagrams with recvmmsg into 2,048-byte slots. Each message keeps its own source address, length, and truncation flag. Only datagrams from 203.23.178.228 and no longer than 1,228 bytes are counted as stream traffic; the verification challenge is printed separately.

The buffers are allocated once, so the receive loop avoids allocating one buffer for every datagram. recvmmsg still copies packet bytes from the kernel into userspace. True network zero copy requires a different path such as AF_XDP, with NIC and driver support.

Build and run

Download Cargo.toml and main.rs into the usual Cargo layout, then run on a Linux host with the registered UDP port:

bash

mkdir -p srccargo run --release -- 9000

Set the same port in the dashboard, cloud firewall, host firewall, and any NAT mapping. Allow inbound UDP from 203.23.178.228/32. The sender source port is not specified, so filter on source IP and destination port only.

SO_REUSEPORT and worker design

SO_REUSEPORT permits several sockets to bind the same address and port. It does not promise that this feed will be split evenly: a stable source and destination flow can hash to one socket. Start with one receive socket and hand off work to bounded worker queues. Add a reuse-port BPF classifier only after measuring the flow distribution on the target host.

Production boundary

This is an intake probe, not a shred verifier. The hot path reports aggregate bytes and packet counts. A production receiver must export SO_RXQ_OVFL, kernel UDP drops, queue drops, and packet age, then parse and authenticate shreds with a pinned Agave release. See socket setup, drop detection, and the production checklist.

Upstream APIs and source