Receiver requirements
Prepare a host, network path, and operating system for sustained raw shred delivery.
Before you start
- A public IPv4 address and UDP port
- Firewall access for source 64.130.40.90
Use a dedicated Linux host
Receive the feed on a physical host or a virtual machine with predictable CPU scheduling and network performance. The measured feed is 54.3 Mbps, 5,585 packets per second, and 1,216 bytes per packet on average. That average includes both data and coding shreds. Design for at least four times the measured packet rate because packet processing, not link bandwidth, is usually the limiting resource during bursts.
Allocate two physical cores as a starting point. Reserve one core for socket ingestion and one for parsing, verification, recovery, and downstream publication. Four cores give enough room to separate signature verification and decoding. Avoid oversubscribed shared-core instances. A receiver that loses its CPU for 50 milliseconds can fill a small socket queue even when the one-minute traffic graph looks calm.
Provide at least 4 GiB of RAM. The live working set is normally much smaller, but slot buffers, FEC sets, metrics, and downstream queues need bounded headroom. Do not make any queue unbounded. A slow consumer must cause an explicit eviction or load-shed decision, not a gradual process-wide out-of-memory failure.
Use a 1 Gbps network interface. The mean feed occupies about 5.4 percent of that link, leaving room for bursts, RPC traffic, monitoring, and operating-system overhead. The service sends raw UDP from Frankfurt with source address 64.130.40.90. It does not order packets, retransmit missing packets, or apply backpressure.
Verify the path before activation
Route a public UDP port to the receiver. Permit only UDP packets from 64.130.40.90 on that port. If network address translation is present, keep the mapping stable and send the verification challenge through the same path that will carry shreds. Do not place a stateful load balancer in front unless it preserves source addresses and has documented UDP idle and flow limits.
Confirm the interface MTU is at least 1,500 bytes. The maximum delivered datagram is 1,228 bytes, so it fits without IP fragmentation on an ordinary Ethernet path. A tunnel can reduce the effective MTU. Capture traffic during commissioning and reject a design that fragments received UDP packets.
Use these read-only checks on Linux:
bash
ip -brief address
ip route get 64.130.40.90
ip link show dev eth0
ethtool -k eth0
ethtool -S eth0 | sed -n '/drop|miss|error|buffer/ip'
sysctl net.core.rmem_max net.core.rmem_default net.core.netdev_max_backlog
ulimit -nReplace eth0 with the interface selected by ip route. Record the output so later changes are visible. Receive checksum offload and generic receive offload can be useful, but test the actual driver. UDP generic receive offload may present aggregated datagrams differently to specialized APIs. A normal recvmsg or recvmmsg path remains datagram oriented.
Prepare the operating system
Raise net.core.rmem_max before asking the socket for a large SO_RCVBUF. Linux caps an unprivileged request at that sysctl. Linux also reports twice the requested receive buffer because it reserves bookkeeping space. Read the value back with getsockopt and log it. The buffer-sizing page derives a concrete value from burst tolerance.
Keep file descriptor limits above the number of sockets, metrics endpoints, logs, and downstream connections. A single-socket receiver does not need a huge limit, but 65,536 avoids surprises when the same process maintains RPC and storage clients. Disable swap for latency-critical deployments or set a low swappiness value, then confirm the process never enters sustained memory pressure.
Pin the ingest thread to a CPU only after measuring. If you pin it, align the NIC receive queue IRQ and the application thread with deliberate CPU placement. Putting both on one busy core can be worse than leaving scheduling alone. Inspect /proc/interrupts and the NIC queue statistics rather than assuming the interface maps cleanly to CPU zero.
Make time and telemetry reliable
Synchronize the host clock with chrony or another monitored NTP client. Arrival latency measurements are useless when the receiver clock steps or drifts. Use a monotonic clock for durations and wall time only for correlation. Export the clock offset and synchronization state.
Collect packet counts at four boundaries: NIC, kernel UDP stack, application receive loop, and accepted shred parser. The differences locate loss. Also record invalid datagram length, parse rejection, signature failure, duplicate shred, FEC recovery, completed data set, and eviction counts. Store counters as monotonically increasing integers and compute rates outside the hot loop.
Keep protocol dependencies pinned
Use the Agave ledger implementation that matches the network release when parsing shreds. The wire format is protocol data, but helper APIs and serialization crates change between releases. Current Agave code exposes Shred::new_from_serialized_shred, header accessors, signature verification, and Shredder::deshred. Recent entry payloads use wincode; older code commonly used bincode. Pin one tested revision and replay captured datagrams before upgrading.
Start with a capture-only receiver. Save a bounded sample with arrival timestamps and source addresses, then test parsing offline. Move verification and decoding out of the receive thread. The receive loop should copy each datagram into owned memory, attach a monotonic timestamp, and hand it to a bounded queue.
Acceptance test
Run for at least one busy hour. The test passes when Udp:RcvbufErrors, NIC missed or dropped counters, application queue drops, and truncated datagrams remain at zero. Packet rate should be close to the measured 5,585 packets per second over a representative interval, but block production varies. Verify that the only accepted source is 64.130.40.90 and that datagram lengths never exceed 1,228 bytes.
Parameters
When it goes wrong
Udp: RcvbufErrors increases
Cause. The socket receive queue filled before the process drained it.
Fix. Increase SO_RCVBUF, shorten the receive loop, and remove decoding work from the ingest thread.
No datagrams from 64.130.40.90
Cause. The public route, NAT rule, security group, or host firewall does not admit the feed.
Fix. Trace the same UDP port end to end and allow source 64.130.40.90/32.
recvmsg: Message too long
Cause. The application buffer is smaller than a delivered datagram.
Fix. Allocate at least 1,228 bytes per receive slot and inspect MSG_TRUNC.
Questions
- Does the receiver need to run in Frankfurt?
- No. The feed originates in Frankfurt and can be delivered to any verified public destination. Put the receiver near the systems that consume its output, then measure path loss and latency. A receiver in another region still needs enough socket buffering to survive scheduling pauses and network bursts.
- Is a 100 Mbps interface sufficient?
- The measured average is 54.3 Mbps, so 100 Mbps has little burst and operational headroom. Use a 1 Gbps interface. It reduces queue pressure and leaves capacity for monitoring, RPC access, storage, and short traffic bursts without turning link saturation into silent UDP loss.
- Can the receiver run in a container?
- Yes, if the container uses a predictable network path and receives the required socket limits. Confirm the host sysctls, cgroup CPU allocation, receive buffer readback, and packet-drop counters. Host networking removes one forwarding layer, but it does not replace measurement or bounded application queues.