UDP vs TCP for market data

The decision is about failure semantics

UDP and TCP can both carry the same bytes. The material difference is what happens when a packet is late, lost, duplicated, or delivered out of order. TCP turns packets into one reliable, ordered byte stream. The receiving application cannot observe byte 50,001 until every earlier byte has arrived. UDP exposes independent datagrams. A receiver may process datagram 50,001 while datagram 50,000 is missing.

That distinction matters more than average latency. Market data has a short useful life. A newer state update often makes an older update worthless. Waiting for the old update can therefore reduce correctness at the decision layer even while improving completeness at the transport layer. Raw Solana shreds follow this logic. They arrive over UDP without ordering, retransmission, or backpressure, and the consumer decides how to handle gaps.

TCP remains the right choice when every byte is required, when the data has no independent sequence identity, or when replay from the sender is the recovery mechanism. UDP is the right starting point when freshness dominates completeness and the application already understands sequence numbers, duplicates, and partial recovery.

TCP provides a byte stream, not messages

One successful TCP send does not correspond to one receive. TCP may split, combine, delay, and retransmit segments while presenting an ordered stream of bytes. An application protocol must add its own record boundaries, commonly a fixed header containing message type and length. A partial read is normal. A production receiver loops until the declared record length has arrived.

This abstraction is useful because it hides path MTU, segment loss, reordering, and duplicate suppression. It also creates coupling. One missing segment blocks delivery of all later bytes on that connection, even if those later bytes contain newer market events. This is transport head-of-line blocking.

TCP implementations include selective acknowledgements and modern loss recovery, so a gap may be repaired quickly on a healthy nearby path. It is still at least one loss-detection interval plus retransmission time. A one millisecond pause is large beside a decoder budget measured in tens of microseconds. A longer retransmission timeout can be devastating. The stream eventually becomes complete, but its trading value may already be gone.

UDP preserves datagram boundaries

One UDP send produces one datagram, and one successful receive returns that datagram as a unit. The kernel does not merge two datagrams into one application message. If the supplied receive buffer is too small, the excess bytes are discarded and the call can report truncation through MSG_TRUNC. The original datagram cannot be completed by another read.

The transport carries source and destination addresses, ports, a length, and a checksum. It does not establish a connection, acknowledge receipt, pace the sender from receiver feedback, or retry loss. Packets can arrive twice or in a different order. The application must attach meaning to each packet independently.

Solana shreds already contain identifiers used for grouping and ordering at the protocol layer. Coding shreds provide erasure recovery within a fec set. Those properties make datagrams a natural transport unit. A receiver can ingest a later shred immediately, record a gap, and decide whether reconstruction can cover it. TCP would impose a second, unrelated ordering constraint below that protocol.

Loss changes the comparison

Zero-loss benchmarks make UDP and TCP look more similar than production. Introduce one dropped packet and their tail behavior separates. TCP withholds later stream bytes until the missing segment is recovered. UDP delivers whatever survived. The UDP application sees an explicit hole, but its newest data continues to advance.

Neither outcome is universally safer. An order book built from mandatory incremental deltas may become invalid after one gap. Continuing to trade on it would be a correctness failure. That feed needs a snapshot or replay path, regardless of transport. A stream of independently verifiable observations can often tolerate a missing element. Solana data and coding shreds also provide recovery above UDP, although recovery has limits and costs CPU time.

The operational question is therefore not whether packet loss is acceptable. Loss exists under either transport. The question is whether the transport should stop later delivery while it repairs loss. Low-latency systems often choose UDP for the live path and a separate reliable channel for snapshots, control messages, or historical replay.

Backpressure protects TCP and delays it

TCP has several flow-control layers. The receiver advertises how much buffer space remains. The sender limits outstanding bytes with the smaller of the receive window and congestion window. Congestion control reduces sending after evidence of loss or queueing. These mechanisms prevent an unconstrained sender from overrunning every receiver indefinitely.

The protection carries a latency cost. A slow consumer fills its receive window, the advertised window shrinks, and the sender stalls. If many subscribers share an upstream production stage, one blocked connection can consume memory or scheduling attention unless the fanout architecture isolates it. TCP also buffers data in kernels and libraries, so a connection may look healthy while delivering increasingly old records.

UDP has no transport backpressure. A sender keeps emitting. A slow receiver loses packets when the NIC ring, kernel backlog, socket receive queue, or application queue fills. This is harsher but visible. A market-data receiver can count the loss, discard stale work, and remain near the head of the stream. Capacity planning replaces flow control.

Congestion control serves a different objective

TCP congestion control seeks stable sharing and path safety. Algorithms such as CUBIC and BBR adjust the amount of data in flight based on acknowledgements, loss, delay, and a model of the path. That behavior is essential on general networks. It is not designed to preserve the latest event at a fixed publication rate.

A UDP feed can send at 54.3 Mbps regardless of whether the destination can receive it. That does not exempt the operator from congestion responsibility. The sender must provision the path, cap the offered rate, and avoid using an uncontrolled public path as if capacity were guaranteed. A private circuit, cross-connect, or well-provisioned datacenter route makes fixed-rate UDP more defensible.

Adding reliability and congestion control above UDP can be useful, but it changes the product. QUIC, for example, runs over UDP while providing encrypted reliable streams and congestion control. It avoids some cross-stream head-of-line blocking, not the reliability delay within one stream. Raw datagrams and reliable streams solve different problems.

Connection state has operational consequences

TCP begins with a handshake and keeps state at both endpoints. Firewalls and NAT devices track sequence state and timeouts. A reconnect creates a new stream position that the application must reconcile. The handshake cost is normally irrelevant for a persistent feed, but reconnect storms and state-table limits are not.

UDP has no handshake at the transport layer. A receiver binds a port and can accept the first datagram immediately. Network devices may still create pseudo-connections in conntrack tables, usually keyed by source address, source port, destination address, destination port, and protocol. Those entries expire on timers rather than FIN or RST packets.

The absence of a transport session makes endpoint verification important. A receiver should filter the expected source IP, such as 64.130.40.90 for the shredstream.sh feed, and bind only the intended destination port. Source IP filtering is not cryptographic authentication. Protocol signatures and application validation remain necessary when authenticity matters.

Fanout favors independent UDP destinations

A TCP publisher maintains one flow-control and congestion state machine per subscriber. It must retain or regenerate bytes for connections progressing at different speeds. Thousands of slow or lossy subscribers can turn a simple fanout process into a queue-management system. Disconnect policies become part of the market-data contract.

A UDP fanout can copy each datagram to every destination and avoid per-destination delivery queues. One slow destination drops locally without delaying another. This isolation is attractive for fixed-rate multicast-like delivery, even when ordinary unicast UDP is used. It also means the publisher receives little direct evidence of subscriber health.

The trade-off moves into operations. UDP needs destination verification, rate accounting, abuse controls, and external health telemetry. The receiver needs gap counters and enough capacity for the full stream. A TCP publisher can observe acknowledgement progress directly, but that signal may encourage buffering old data rather than rejecting a receiver that cannot stay current.

Latency must be measured as a distribution

Average receive latency hides the behavior that chooses the transport. Record p50, p99, p99.9, and maximum latency during controlled loss and bursts. Also record freshness, defined as the difference between the newest sequence produced and the newest sequence processed. A feed that reports a good p50 while falling seconds behind is not healthy.

For TCP, inspect retransmissions, round-trip time, receive-window pressure, and application queue age. The command ss -tin shows fields such as rtt, rto, cwnd, bytes_retrans, and retrans on established sockets. The counters TcpRetransSegs and TcpExtTCPLostRetransmit are available through nstat on common Linux systems.

For UDP, inspect socket drops, UDP receive errors, NIC discards, and sequence gaps. nstat -az UdpInDatagrams UdpInErrors UdpRcvbufErrors reports protocol totals. The final column of /proc/net/udp is the per-socket drops count. Application sequence gaps distinguish upstream or path loss from drops visible on the local host.

Buffering can make reliable data stale

Large buffers are not a complete performance fix. A TCP socket with megabytes queued can retain every byte and still deliver a uselessly old market. A UDP socket with an oversized SO_RCVBUF can create the same problem if the application drains the queue more slowly than the feed arrives. Completeness and freshness need separate service-level objectives.

At 54.3 Mbps, the byte rate is about 6.79 MB/s before link-layer overhead. A 16 MiB receive allocation can hold roughly 2.47 seconds of payload at that mean rate. That is far too much hidden age for many trading systems. The buffer is insurance against short scheduler pauses, not permission to run behind.

Measure queue occupancy or packet timestamps, not only drops. Linux exposes the receive queue in ss output. SO_TIMESTAMPNS or hardware receive timestamps can let the application compute residence time. If residence time grows steadily, increase processing capacity or discard stale work. Raising net.core.rmem_max postpones the symptom but does not change throughput.

A hybrid design is common

The live lane and recovery lane have different requirements. UDP can carry the freshest observations at a fixed rate. TCP, gRPC, or object storage can provide snapshots, configuration, entitlement state, and replay. When the live receiver detects an unrecoverable gap, it marks dependent state invalid, obtains a new baseline over the reliable lane, and resumes.

This design prevents a transient gap from turning the entire live connection into an old stream. It also makes recovery explicit. The application knows when its derived state is unsafe instead of assuming that an open socket implies correctness. The reliable channel can be slower because it is exceptional rather than part of every event.

Hybrid systems cost more to build. Sequence spaces must line up, snapshot boundaries must be unambiguous, and state transitions need tests. For raw shreds, replay may not be available from the UDP service at all. Coding shreds and a second independent source may be the relevant recovery tools. Transport choice does not remove the need for an application recovery model.

Choosing for a Solana shred receiver

Raw shreds favor UDP because each packet is independently identified, later packets remain useful after a gap, and erasure coding can recover some missing data. The receiver is expected to handle duplicates and out-of-order arrival already. Imposing TCP ordering below that machinery can turn one lost segment into a pause across many otherwise usable shreds.

The choice assumes an engineered receiver. The host must sustain the feed rate, allocate socket and kernel queues deliberately, pin work where needed, monitor NIC and UDP drops, and avoid expensive work on the ingest thread. A program that performs blocking database writes inside its receive loop will fail under UDP with visible loss. Under TCP it will fail by accumulating age.

Use TCP when the consumer requires every byte in order and can tolerate repair delay. Use UDP when the consumer can identify gaps, preserve safety after them, and values current data more than an uninterrupted transport abstraction. The protocol should express the application's truth, not conceal it.