Why UDP has no backpressure

Backpressure is a feedback signal

Backpressure means a downstream component can tell an upstream component to reduce or stop production. TCP has this signal in the advertised receive window. A receiver reports available buffer space in acknowledgements, and a sender limits bytes in flight. Application frameworks can extend the signal through bounded queues and blocked writes.

UDP contains no acknowledgement, receive window, connection state, or delivery report. The sender submits a datagram to its local kernel. A successful send normally means the local stack accepted it, not that the destination application received it. The sender can continue at the same rate while the receiver is paused, overloaded, disconnected, or filtered by a firewall.

This is not an omitted performance feature that a sysctl can enable. It is part of UDP's contract. A custom application protocol can send rate feedback over UDP, but the application must define and enforce it. A one-way raw shred feed does not. Its receiver stays safe by provisioning capacity, bounding work, observing loss, and choosing what to discard.

The queue chain is finite

An arriving packet crosses several finite structures before user code sees it. The NIC receives the frame into a descriptor-backed buffer. The driver and NAPI poll cycle move packet work through the networking stack. The host may use the per-CPU input backlog. IP and UDP processing find the socket. The datagram is charged to the socket receive queue. The application removes it with recvmsg, recvmmsg, or an asynchronous interface.

There may be more queues after that: a decoder ring, a work-stealing executor, a channel, a logger, or a database client. None creates capacity. Each can absorb a temporary mismatch between arrival and service rate, but a sustained mismatch eventually fills every bounded stage. An unbounded user-space queue replaces packet loss with memory growth and stale work.

Operators need a counter at every transition. NIC discards, softnet drops, UdpRcvbufErrors, per-socket drops, application ring rejects, and sequence gaps locate different failures. Looking only at the final business event count collapses all of them into one ambiguous symptom.

A successful send is local acceptance

For an unconnected UDP socket, sendto returns the number of bytes accepted or an error. For a connected UDP socket, send has the same local meaning while allowing some asynchronous network errors to be reported. Neither call waits for destination application capacity. The datagram can be dropped later by the source qdisc, route, switch, firewall, destination NIC, kernel, or socket.

The source can encounter local pressure. A nonblocking send may return EAGAIN when its send buffer cannot accept data. Blocking behavior or ENOBUFS can occur under specific local conditions. Those are sender-host signals, not end-to-end backpressure from a particular subscriber. With unicast fanout, one destination may be losing everything while sends to it continue to succeed.

This distinction affects health checks. A publisher cannot count successful send calls and claim subscriber delivery. End-to-end sequence telemetry or receiver-reported statistics are required. The feed's verification datagram proves reachability at one moment. It does not prove sustained capacity for 54.3 Mbps and 5,585 packets/sec.

Overload becomes loss or age

When a UDP receiver's service rate falls below arrival rate, queue occupancy grows. At the queue limit, packets drop. Which packets survive depends on the congested stage and timing. The application may see bursts, gaps, and reordered processing after recovery. UDP does not select the most useful market event on the receiver's behalf.

Large queues delay the drop point. They also increase potential age. The measured feed carries about 6.79 MB of packet payload per second from 54.3 Mbps. A receive allocation measured in tens of megabytes can hide seconds of backlog. That may look better on a loss graph while producing worse decisions.

A low-latency design sets two independent targets: acceptable loss under defined bursts and maximum packet residence time. Buffers should cover interrupt moderation, scheduler pauses, and short compute spikes. They should not cover sustained undercapacity. When age crosses the strategy's bound, explicit shedding is usually safer than processing an intact past.

TCP backpressure moves the problem upstream

TCP's receive window prevents a sender from continuously filling a receiver's socket buffer. When the application stops reading, the receiver advertises less space and can eventually advertise a zero window. The sender retains unsent or unacknowledged data and its write path slows or blocks. Congestion control adds a separate limit based on path conditions.

The data is not made current by this mechanism. It waits in a sender buffer, receiver buffer, proxy, or application queue. A complete stream can become old. For transaction processing, file transfer, and many request protocols, completeness is the right invariant. For live market observations, old data may be less safe than an explicit gap.

UDP chooses isolation between sender pace and receiver health. One slow subscriber does not force a global stream to slow. The receiver bears the failure. That makes fanout predictable at the sender and demanding at the edge. Neither semantic is free.

Socket receive buffers absorb short pauses

SO_RCVBUF controls a socket's receive-buffer request. Linux caps unprivileged requests using net.core.rmem_max and reports a doubled value through getsockopt and ss for bookkeeping overhead. net.core.rmem_default supplies the default when the application does not set one. Applications should request a known size and verify the effective value.

A socket buffer is charged for kernel packet memory, not only UDP payload bytes. The number of datagrams it holds is therefore lower than buffer bytes divided by 1,216. Small packets have proportionally more overhead. Measure capacity with the real kernel, driver, and packet-size distribution.

When the socket cannot enqueue another datagram, Linux increments UdpRcvbufErrors and the socket's drop count. nstat exposes the protocol total. The last column in /proc/net/udp exposes a per-socket count. ss -u -a -m prints skmem information, including the current receive allocation and drop count on kernels that provide it.

netdev backlog covers a different pause

net.core.netdev_max_backlog limits the per-CPU input queue used when packets arrive faster than the kernel can process them in the current networking cycle. It is upstream of the UDP socket. Raising the socket buffer does not repair drops that already happened in the NIC or networking backlog.

/proc/net/softnet_stat contains per-CPU hexadecimal counters. The second field is packets dropped because the input backlog was full. The third field counts times processing could not complete within the current budget. Exact later fields vary by kernel version, so use kernel documentation or matching tooling when interpreting them. A nonzero historical counter is less useful than its rate during the incident.

net.core.netdev_max_backlog=8192 is a defensible starting experiment for a dedicated high-rate receiver, not a universal optimum. CPU affinity, NAPI budget, driver rings, receive-side scaling, and interrupt moderation determine whether the backlog is reached. Change one layer at a time and replay a known load.

User-space queues need an explicit policy

The receive loop should do bounded work: timestamp, validate basic length, capture identity, and enqueue to a preallocated structure. Expensive decoding, logging, allocation, and storage belong off the ingest path. The handoff queue must have a capacity and a full-queue policy.

Blocking the receive thread applies backpressure only to the user-space producer. The network sender continues. While the thread waits, the kernel socket queue fills and then drops. An unbounded queue lets the thread keep receiving but consumes memory and raises packet age. Neither default is acceptable without measurement.

Common policies include dropping the newest packet, overwriting the oldest unprocessed packet, dropping an entire invalidated group, or marking state unavailable until recovery. The correct policy depends on protocol semantics. Shreds cannot be discarded solely by arrival age if they are still required for a recoverable fec set. The decoder needs enough metadata to shed work coherently.

Batching increases service capacity

One recvmsg call per packet pays a system-call and scheduling cost for each datagram. Linux recvmmsg receives multiple messages in one call and can reduce overhead at thousands of packets per second. io_uring also supports multishot receive operations on suitable kernels, although operational maturity and buffer management deserve testing.

Batching trades latency for throughput if the application waits to fill a batch. A good receive loop asks for a batch but returns with what is available, then processes promptly. Batch size should be a ceiling, not a timer that holds the first packet. With recvmmsg and no blocking timeout, bursts can amortize calls without intentionally delaying quiet periods.

The application should count calls, packets per call, empty polls, queue rejects, and receive-to-decode time. A batch size of 32 may produce an observed mean of 5 during ordinary load. That is useful evidence. Claiming 32-fold syscall reduction from the configured maximum is not.

CPU scheduling is part of capacity

A receiver can have adequate average CPU and still drop during short scheduling gaps. Page faults, memory reclaim, frequency changes, shared-core interrupts, stop-the-world runtime pauses, and noisy neighbors can keep the ingest thread off CPU. The socket buffer then acts as a time reservoir.

Dedicated systems commonly pin the NIC receive queue interrupt and ingest thread to chosen CPUs, keep heavy workers elsewhere, set an intentional CPU frequency policy, lock hot memory where appropriate, and avoid synchronous logs. These changes can reduce variance. They can also reduce scheduler flexibility and worsen performance if IRQ and application work contend on one core.

Observe with perf, pidstat, /proc/interrupts, and scheduler tracing before assigning causes. A growing UdpRcvbufErrors count with no NIC or softnet drops points toward socket drain capacity. A user-space ring rejection with clean kernel counters points farther downstream. CPU utilization averaged across the host will not reveal a saturated receive core.

Application feedback is possible but different

An application protocol can send receiver reports containing sequence gaps, buffer occupancy, or requested rates. Real-time media protocols use this pattern. A sender can adapt packet rate, encoding, redundancy, or destination membership. The feedback is a feature above UDP, not UDP itself.

A shared market-data feed often cannot reduce source rate for one subscriber because every packet belongs to the product. Slowing the global producer would harm healthy receivers. Per-subscriber sampling would change data semantics. The practical response to repeated overload may be to alert, suspend a destination, or require the customer to provision more capacity.

Forward error correction is another response. It spends extra bandwidth so a receiver can rebuild some missing packets without a round trip. Solana coding shreds follow this model. Redundancy reduces the consequence of limited loss, but it raises arrival rate and does not make an undercapacity receiver stable.

Monitoring should identify the first full queue

Collect counters at short intervals and compare deltas. ethtool -S eth0 exposes driver-specific fields such as rx_missed_errors, rx_no_buffer_count, rx_discards, or named per-queue drops, depending on the NIC. /proc/net/softnet_stat identifies host-stack pressure. nstat reports UdpRcvbufErrors. /proc/net/udp and ss identify socket state. The application reports its own queue and sequence results.

The earliest counter that rises is usually closest to the first exhausted stage. Driver names are not standardized, so record the exact ethtool -S output for the deployed driver. Do not assume a field found on Intel hardware exists on an Amazon ENA or Mellanox interface.

Alert on rates and correlated freshness. One lost packet during maintenance has different meaning from a continuously increasing counter. A flat loss counter with rising packet age means buffering or downstream delay, which is equally relevant to a trading system.

Capacity replaces implicit flow control

The receiver must sustain the full steady rate with burst margin. For the measured feed, plan around 54.3 Mbps, 5,585 packets/sec, a 1,216 byte mean packet, and 17.6 TB per 30-day month. Packet rate drives per-packet CPU cost. Bit rate drives link and byte-copy cost. Both matter.

Test above the observed mean. Replay at 1.5 times and 2 times the packet rate for short periods, induce controlled scheduler pauses, and verify that buffers cover the agreed burst without violating maximum age. Continue the test long enough to expose thermal, allocator, and logging effects.

No UDP tuning value substitutes for this headroom. A larger ring or buffer is valuable when service rate exceeds arrival rate over the relevant window. When service rate remains lower, the queue still fills. UDP makes that arithmetic visible and leaves the overload policy with the operator.