Reference
Solana shred and UDP receiver glossary
Define the protocol, Turbine, ledger, networking, Linux, timing, recovery, and receiver vocabulary used when operating a raw Solana shred feed.
A
| Term | Definition |
|---|---|
| Account | Solana’s basic state record, addressed by a public key and containing lamports, owner, executable state, rent epoch information, and opaque data. Shreds can reveal transactions that may later read or write accounts, but account changes exist only after execution. |
| Account key | A public key listed or resolved for a transaction message. Instruction indices refer into the ordered account-key list, so a decoder must preserve order and resolve address-table lookups before interpreting an instruction. |
| Address lookup table | An on-chain account that lets a versioned transaction refer to additional public keys by compact indices. A raw transaction carries the lookup reference and index lists, not the resolved keys, so historical table state is needed for full decoding. |
| Agave | The validator client and codebase descended from the original Solana Labs validator implementation. Raw receivers commonly pin its ledger and shred libraries because variant rules, sanitation, Merkle handling, and serialization details belong to a concrete release. |
| Ancillary data | Metadata returned beside socket payload by recvmsg or recvmmsg, such as timestamps, interface information, and drop counters. The control buffer has its own capacity, and MSG_CTRUNC means some metadata was lost even when payload bytes arrived. |
| Application queue | A user-space buffer between packet intake and later parsing, recovery, or strategy work. It absorbs only temporary rate differences, so it needs a fixed capacity, occupancy metric, overflow policy, and packet-age limit. |
| Asymmetric routing | A condition where forward and reverse traffic use different paths or interfaces. It can bias clock synchronization, invalidate half-RTT assumptions, and cause strict reverse-path filtering to discard valid inbound packets. |
| Availability | The proportion of a defined interval during which a service meets a stated readiness criterion. A valid availability claim needs the observation point, failure definition, exclusions, interval, and denominator, none of which can be replaced by a single successful packet. |
B
| Term | Definition |
|---|---|
| Backpressure | A transport or queue mechanism that slows a producer when a consumer cannot keep pace. UDP has no receiver-driven backpressure, so overload appears as queue growth, packet age, and drops rather than sender slowdown. |
| Bank | Solana’s in-memory state for executing and recording transactions at a slot on a particular fork. Shreds carry proposed ledger data before the corresponding bank has necessarily executed, frozen, or become rooted. |
| Base58 | A text encoding used for many Solana public keys, signatures, and hashes because it avoids visually ambiguous characters. Shred headers contain fixed binary bytes, so converting every field to Base58 on the ingest thread adds avoidable allocation and CPU cost. |
| Batch receive | Reading several datagrams in one system call or completion batch, commonly with recvmmsg on Linux. Batching amortizes syscall overhead but can increase queue residence when the receiver waits too long to fill a batch. |
| Block | The ledger output associated with a produced slot, assembled from entries and transactions. Solana distributes its content incrementally as shreds, so a receiver can observe early pieces before the whole block is complete. |
| Block production | The leader’s work of selecting, executing, recording, shredding, and transmitting transactions during its assigned slots. Production and propagation overlap, which is why raw shreds can arrive while later parts of the block are still being formed. |
| Blockhash | A hash used to identify recent ledger state and constrain transaction lifetime. The recent blockhash is inside the transaction message, while block identity and fork acceptance require later ledger context. |
| Blockstore | The validator’s persistent ledger store for shreds, completed data, coding metadata, and slot relationships. A standalone raw receiver can build lighter state, but its grouping and recovery rules must remain compatible with the protocol. |
| Burst | A short interval in which packet arrivals exceed their longer-term average. Rings and queues can absorb a bounded burst, while sustained overload eventually produces drops or stale work regardless of buffer size. |
| Busy polling | A receive technique where a thread polls an associated NAPI context for a bounded time before sleeping. It can reduce wakeup tails on supported kernels and drivers at the cost of CPU, energy, and possible contention with decode work. |
C
| Term | Definition |
|---|---|
| Capture loss | Packets dropped by a capture socket, kernel buffer, or capture writer while traffic continues elsewhere. A pcap with precise timestamps is not complete evidence unless capture-drop counters and interface loss remain accounted for. |
| Chained Merkle shred | A Merkle shred that commits its FEC set to the preceding set’s root under the variant’s chain rule. Validation needs both the current proof and compatible prior-root context, and the chain material reduces space available elsewhere in the packet. |
| Clock discipline | The process of steering a clock’s offset and frequency toward a selected reference. NTP, PTP, phc2sys, and a servo can discipline clocks, but the measurement still needs an uncertainty budget for asymmetry, timestamp placement, and holdover. |
| Clock domain | A time scale and oscillator context in which timestamp values are comparable. CLOCK_REALTIME, CLOCK_MONOTONIC, CLOCK_TAI, and a NIC PHC are different domains, so subtracting values across them requires an explicit measured relationship. |
| Clock offset | The difference between two clocks at the same physical instant. An uncorrected offset appears directly as one-way latency error and can make a real positive path delay look negative. |
| Clock step | A discontinuous clock correction rather than a gradual frequency adjustment. Any interval spanning a step should be rejected from latency analysis because timestamp order and deltas no longer share one stable mapping. |
| Cluster | A set of Solana validators sharing genesis context and ledger history. Mainnet, test networks, and private deployments can reuse slot numbers, so cluster context must be part of persistent shred identity. |
| Coding shred | A shred carrying Reed-Solomon parity rather than serialized ledger bytes. It can reconstruct missing data shreds inside a compatible FEC set, but its payload is never appended directly to the entry stream. |
| Commitment | A statement about how much cluster voting and rooting supports observed state, such as processed, confirmed, or finalized in RPC contexts. Raw shreds have no commitment guarantee because they arrive before later fork and vote outcomes. |
| Compiled instruction | A transaction instruction encoded with a program-id index, account index list, and opaque data bytes. Its meaning depends on the resolved account keys and the selected program’s instruction schema. |
| Compute unit | Solana’s metering unit for transaction execution work. Raw transaction bytes can include compute-budget instructions, but actual units consumed are execution output and are not present in the shred header. |
| Confirmation | A later consensus property indicating that sufficient voting supports a block under the cluster’s commitment rules. Receiving and authenticating a leader shred does not establish confirmation. |
| Connection tracking | Stateful flow bookkeeping performed by Linux netfilter for protocols including UDP. A one-way feed may remain unassured, and table pressure or state-only firewall rules can drop traffic despite UDP having no handshake. |
| Consensus | The rules and votes by which validators choose and finalize a common ledger history. Turbine moves proposed block data, while consensus later decides which fork persists. |
| CRDS | The replicated data structure carried by Solana gossip for contact information and other cluster records. It is eventually consistent and supports peer discovery, but it is not the raw block-data transport. |
D
| Term | Definition |
|---|---|
| Data complete flag | A bit in a data shred indicating the end of a data unit in the serialized ledger stream. It marks a boundary but does not prove that every earlier data index arrived or was recovered. |
| Data shred | A shred carrying a slice of serialized ledger data plus common and data-specific headers. Receivers order its sanitized data region by data index and use coding shreds to recover missing members. |
| Datagram | One message unit delivered by UDP with its own length and boundaries. A receive call returns one datagram at a time, and a short application buffer truncates that unit rather than continuing it in the next call. |
| Decode latency | Time from a named input boundary to a named decoded output, such as kernel receive to completed transaction parsing. It includes the chosen validation, buffering, FEC, deshredding, and scheduling stages and should not be labeled network latency. |
| Deduplication | Recognizing repeated delivery of the same shred identity and retaining a defined copy, normally the earliest valid one. The key must include cluster context, slot, shred type, and index so data and coding spaces do not collide. |
| Deshredding | Reassembling sanitized data-shred payloads in index order into the serialized ledger byte stream. It excludes coding bytes, headers, padding, Merkle proof nodes, chained roots, and retransmitter signatures. |
| Destination | The customer-controlled IPv4 address and UDP port that receive verification and live traffic. Changing the address requires renewed proof of control and is separate from the fixed service source address. |
| Duplicate conflict | Two different byte strings claiming the same shred identity. The receiver must not silently overwrite one with the other because the pair can indicate corruption, context mixing, or authenticated equivocation. |
E
| Term | Definition |
|---|---|
| Early demultiplexing | A Linux receive optimization that associates eligible packets with a socket earlier in the IP path. It can reduce lookup work, but its effect depends on socket type, routing, namespace, and the active kernel path. |
| eBPF | The Linux in-kernel virtual machine used for verified programs attached to networking and tracing hooks. It supports filters, observability, and XDP processing, but every program adds code whose bounds and packet semantics must be reviewed. |
| Ed25519 | The public-key signature scheme used for Solana identities, transactions, and shred authentication. Verification proves that a selected key signed variant-defined material, not that the resulting block will become canonical. |
| Endianness | The byte order used to serialize multibyte integers. Current shred header integers are little-endian, while network-layer fields such as UDP ports use network byte order. |
| Entry | A Proof of History record containing a hash count, resulting hash, and a vector of transactions. Entry serialization crosses shred boundaries, so a packet boundary is not an entry boundary. |
| Epoch | A range of slots over which schedules and other cluster state are organized. Leader schedule and stake context can change at epoch boundaries, so a receiver needs the correct context for each slot. |
| Equivocation | Authenticated production of conflicting content for the same protocol position. Conflicting duplicate shreds become meaningful evidence only after identity, leader, signature, variant, and cluster context are verified. |
| Erasure coding | Adding mathematically derived parity so missing data shards can be reconstructed from a sufficient subset. Solana uses Reed-Solomon coding within FEC sets to tolerate UDP and propagation loss without waiting for retransmission. |
| Erasure shard | One fixed-size input or parity element in an erasure-code matrix. Data members occupy positions first, and a coding member’s local erasure position is k plus its coding position. |
| Execution | Running transactions against parent state to produce success or failure, logs, balances, and account changes. It happens after ledger data is reconstructed, so raw shred observation cannot supply execution results. |
F
| Term | Definition |
|---|---|
| Fanout | Distribution from one sender to multiple recipients or from one Turbine node to a selected neighborhood. It reduces the origin’s direct send count but makes path position and intermediate behavior part of arrival time. |
| FEC set | A group of data and coding shreds sharing slot, FEC-set index, dimensions, protected shard size, and compatible authentication context. Recovery is valid only after all admitted members agree on those properties. |
| Finality | The consensus state in which a block is rooted strongly enough that the cluster treats reversal as outside normal operation. A shred is earliest-stage proposed data and carries no finality field. |
| Firewall allowlist | A rule admitting traffic only from approved source and protocol fields. For shredstream.sh, the documented network source is 64.130.40.90/32 over UDP to the customer-configured destination port. |
| First-detection share | The fraction of matched identities for which one feed records the earliest valid arrival beyond a declared tie threshold. A defensible report also publishes ties, one-sided missing identities, sample counts, and clock uncertainty. |
| Flow hash | A hash of packet tuple fields used by RSS, RPS, SO_REUSEPORT, or other steering logic. One stable UDP tuple often maps to one queue unless software or hardware supplies additional entropy. |
| Fork | A competing ledger history that shares an earlier ancestor with another history. Shreds can be validly signed yet belong to a fork that later loses consensus support. |
| Forward error correction | The broader communications method of sending redundancy so receivers can recover loss without a round trip. In the shred path, coding shreds provide the redundancy and Reed-Solomon supplies the reconstruction algorithm. |
| Fragmentation | Splitting one IP packet into fragments because it exceeds a path limit or was formed that way by a sender. Losing one fragment loses the entire UDP datagram, and reassembly consumes finite kernel memory before UDP sees the packet. |
G
| Term | Definition |
|---|---|
| Genesis hash | A cryptographic identifier for a cluster’s initial ledger configuration. It provides stronger cluster context than the compact 16-bit shred version and should namespace multi-cluster storage and replay fixtures. |
| Gossip | Solana’s peer-to-peer control-plane protocol for distributing contact records, votes, version information, and other cluster data. It helps nodes learn peers and state, while Turbine carries the high-rate block fragments. |
| GRO | Generic Receive Offload, a Linux mechanism that coalesces compatible incoming packets before later stack processing. It can reduce per-packet CPU but changes batching and timestamp behavior, so UDP receivers must preserve original datagram segmentation semantics. |
H
| Term | Definition |
|---|---|
| Hardware timestamp | A packet timestamp produced by a NIC near physical ingress or egress. It removes much host scheduling ambiguity but lives in a PHC domain that must be disciplined or translated before cross-host subtraction. |
| Hash | A fixed-size digest used for integrity, identity, Merkle trees, blockhashes, and Proof of History. Its security meaning depends on the exact domain separation and input bytes, not the display encoding. |
| Holdover | A clock’s continued operation after its timing source becomes unavailable. Offset uncertainty grows with oscillator stability, temperature, elapsed time, and the quality of the last frequency estimate, so long holdover can invalidate one-way measurements. |
I
| Term | Definition |
|---|---|
| Identity key | A validator’s long-lived public key used to identify it in schedules and cluster records. A receiver resolves the scheduled leader identity for a slot before verifying leader-authenticated shred material. |
| Ingress | The point where traffic enters a network, interface, host, or processing stage. Latency reports must name the ingress boundary because NIC arrival, kernel enqueue, application dequeue, and decoder admission occur at different times. |
| Instruction | A program invocation inside a Solana transaction, with program selection, account references, and opaque data. Full interpretation requires resolved account keys and the correct program-specific schema. |
| Interrupt coalescing | NIC behavior that waits for a packet count or time threshold before raising a receive interrupt. It lowers interrupt rate but can add quiet-traffic delay and create burstier NAPI delivery. |
| Interrupt request | A hardware or virtual signal asking a CPU to service device work. Queue-specific IRQ affinity influences cache locality, NAPI scheduling, softnet pressure, and the receive thread’s competition for CPU time. |
| IP fragment reassembly | Kernel state that collects IP fragments until one original packet can be rebuilt. Missing fragments, threshold pressure, or timeout produces loss before UDP socket counters can observe the datagram. |
| IPv4 | The 32-bit Internet Protocol version used by the documented shredstream.sh source address. IPv4 headers, routing, fragmentation, reverse-path filtering, and NAT all precede the UDP receive queue. |
J
| Term | Definition |
|---|---|
| Jitter | Variation in latency or inter-arrival time across packets. Report its distribution and conditions because a stable mean can coexist with tails caused by coalescing, scheduler pauses, queueing, route change, or clock error. |
| Jumbo frame | An Ethernet frame supported by an MTU larger than the common 1,500-byte setting. It helps only inside an end-to-end controlled domain, and it is not required for a 1,228-byte UDP payload on a normal path. |
K
| Term | Definition |
|---|---|
| Kernel bypass | A receive design that avoids parts of the normal kernel network stack, as with AF_XDP or a userspace driver framework. It can reduce overhead but transfers memory, queue, filtering, and observability responsibilities to the application. |
| Kernel receive buffer | Socket memory used to queue datagrams until user space removes them. SO_RCVBUF requests its capacity, net.core.rmem_max caps ordinary requests, and charged memory per packet exceeds UDP payload bytes. |
L
| Term | Definition |
|---|---|
| Last shred in slot | A data-shred flag marking the final data shred produced for a slot. It identifies an intended end index but does not fill missing earlier indices or prove the slot will survive fork choice. |
| Leader | The validator scheduled to produce ledger data for a slot. Its public key is the expected origin key for leader-authenticated shreds in that slot. |
| Leader schedule | The mapping from slots to leader identities, derived from cluster state and stake rules. A raw receiver needs trusted schedule context to decide which public key must verify each shred. |
| Ledger | Solana’s ordered history of entries and transactions across slots and forks. Raw shreds are transport fragments of serialized ledger data before later replay and consensus establish canonical state. |
| Little-endian | A byte order that stores the least significant byte first. Current shred header integers use this order, so reading them as network byte order produces implausible slots, indices, sizes, and counts. |
| Local queue delay | Time a packet spends in NIC, softnet, socket, or application queues on the receiving system. A same-host timestamp pair can measure this component without claiming it is provider transit time. |
| Loss | Failure of an expected packet or derived unit to become available at the observation point. Good telemetry separates upstream absence, NIC discard, fragment failure, softnet drop, socket overflow, application rejection, and decoder invalidation. |
M
| Term | Definition |
|---|---|
| Malformed shred | A datagram whose length, variant, header, size, index, proof, signature, or recovery metadata fails validation. Reject it before allocation-heavy work and retain only bounded samples so hostile traffic cannot drive logging load. |
| Maximum transmission unit | The largest network-layer packet an interface or path can carry without fragmentation, usually abbreviated MTU. The smallest effective MTU across physical, virtual, tunnel, VLAN, and routed segments governs delivery. |
| Mempool | A pool of pending transactions visible before block inclusion in some blockchain systems. Solana does not expose one general public mempool equivalent, and shreds show leader-produced data rather than a universal queue of future transactions. |
| Merkle leaf | The domain-separated hash input representing one protected erasure shard in a Merkle tree. Its position and construction come from the active shred variant and must match the validator implementation. |
| Merkle proof | An ordered set of sibling hashes used with a leaf and position to reconstruct a Merkle root. A proof that parses structurally still needs correct domain separation, position, root, and leader signature. |
| Merkle root | The single hash committing to all leaves in a Merkle tree. Current Merkle shred validation reconstructs this root and verifies the slot leader’s signature over the variant-defined root material. |
| Merkle shred | A data or coding shred carrying proof material that authenticates membership in a leader-signed Merkle set. Proof depth, chaining, and retransmitter-signature flags change layout and useful payload capacity. |
| Message | The signed transaction content containing a header, keys, recent blockhash, instructions, and version-dependent lookup information. Transaction signatures cover the serialized message rather than execution results. |
| Message header | Three compact counts describing required signers and read-only signed and unsigned accounts. Those counts partition the ordered account-key list and determine signer and writable privileges. |
| Microburst | A short packet-rate spike that can fill a small queue even when one-second averages look safe. High-resolution counters and controlled replay are needed because coarse monitoring can miss the event entirely. |
| MSG_CTRUNC | A receive flag indicating that ancillary control data did not fit the supplied control buffer. Payload may be intact, but missing timestamps or overflow counters make the associated measurement incomplete. |
| MSG_TRUNC | A receive flag indicating that the datagram exceeded the supplied payload buffer. The returned prefix is not a complete shred and must be rejected rather than parsed as a shorter valid packet. |
N
| Term | Definition |
|---|---|
| NAPI | Linux’s polling framework for bounded network receive work after an interrupt announces packets. Poll budgets, queue affinity, interrupt moderation, and CPU contention shape when datagrams reach the UDP socket. |
| NAT | Network address translation that rewrites source or destination addresses and ports while maintaining mapping state. Inbound one-way UDP works best with a static forwarding rule because a receiver cannot create a provider-side handshake. |
| Network interface card | Hardware or a virtual device that sends and receives network frames, commonly abbreviated NIC. Queue count, ring depth, offloads, timestamps, driver behavior, and interrupt layout all affect a receiver before socket code runs. |
| Network namespace | A Linux isolation boundary with its own interfaces, routes, sockets, firewall state, and many network sysctls. Host counters can differ from container counters, so diagnostics must query the namespace that owns the receiving socket. |
| NIC ring | A descriptor queue through which a NIC and driver exchange packet buffers. A full receive ring causes loss before the normal kernel stack, while an excessive ring can retain older packets and hide service delays. |
| Nonblocking socket | A socket whose receive operation returns immediately when no datagram is ready. It supports event loops and batching but still needs correct readiness handling to avoid spin, starvation, or missed drain opportunities. |
| nstat | A Linux tool that reads protocol counters such as UdpInDatagrams, UdpInErrors, UdpNoPorts, and UdpRcvbufErrors. Use deltas over the incident interval because host-wide totals can include older events and unrelated sockets. |
| NTP | The Network Time Protocol used to estimate and discipline time over packet networks. It has wide reach and is useful for millisecond-scale work, but unknown path asymmetry and software timestamps can exceed a microsecond-scale feed advantage. |
O
| Term | Definition |
|---|---|
| One-way delay | Receive timestamp minus send timestamp for the same packet under a shared or calibrated time basis. The report must include timestamp points, clock state, uncertainty, loss, and packet matching, because half-RTT is not a direct substitute. |
| Out-of-order delivery | Arrival of datagrams in a different sequence from their logical indices. UDP permits it, so a receiver needs bounded slot-aware ordering that does not mistake a late packet for a different identity. |
P
| Term | Definition |
|---|---|
| Packet | A bounded unit transferred through a network layer, often used loosely for an Ethernet frame, IP packet, or UDP datagram. State the layer when size, loss counter, or timestamp placement matters. |
| Packet capture | A recorded set of packets and timestamps collected from an interface or capture point. It preserves identity evidence but can drop under load and inherits the selected timestamp source’s clock error. |
| Packet loss | The absence of a packet expected from an identity sequence or controlled sender. It must be localized with NIC, fragment, softnet, UDP, socket, application, and protocol counters rather than attributed from one gap alone. |
| Packet reordering | A change in arrival order caused by parallel paths, queues, batching, scheduling, or retransmission sources. Measure reorder depth and time because an index distance alone does not define how long state must remain open. |
| Packet socket | A Linux socket that reads packets at a link-layer or network hook, commonly used for capture. It observes a different path and buffering model from the UDP application socket, so timestamps and drops are not interchangeable. |
| Parent offset | A data-header field storing the difference between the shred’s slot and its parent slot. Validation prevents underflow and impossible relationships before the value contributes to fork context. |
| Path MTU | The smallest MTU along the route from sender to receiver. It determines whether an IP packet can travel without fragmentation, and it can differ by direction or failover path. |
| Path MTU discovery | A process that learns or enforces packet size against the path limit using routing behavior and control messages. One-way UDP can hide failures when ICMP is filtered, so production-sized probes and counters remain necessary. |
| Peripheral hardware clock | A NIC or device clock used for hardware packet timestamps, commonly abbreviated PHC. Cross-host measurement requires the PHC to be synchronized and its relationship to the chosen system time scale to be known. |
| Pinned release | A specific dependency version or source commit chosen as the parser authority. Recording it beside captures and fixtures prevents a future library upgrade from silently changing variant, sanitation, recovery, or serialization behavior. |
| Port | A 16-bit transport-layer number used with addresses and protocol to identify a socket endpoint. Source and destination ports are independent, and only the customer destination port belongs in a receiver’s binding rule. |
| Proof of History | Solana’s sequential hash construction used to order and time ledger entries, commonly abbreviated PoH. Entries carry hash counts and results, while consensus and replay validate their place in the ledger. |
| Propagation | Movement of a shred from the leader through Turbine and network paths to receivers. Arrival time includes the producer’s send timing, tree position, intermediate nodes, physical path, and local host queues. |
Q
| Term | Definition |
|---|---|
| Queue residence time | Time between enqueue at one stage and dequeue at the next. Monitoring it beside drop counters prevents a deeper buffer from looking healthy while it serves increasingly old packets. |
| QUIC | A reliable encrypted transport built over UDP and used by parts of Solana’s transaction-ingress path. Raw Turbine shred forwarding over UDP is a different contract and does not inherit QUIC streams, acknowledgements, congestion control, or retransmission. |
R
| Term | Definition |
|---|---|
| Raw shred | The original serialized Solana shred bytes before provider decoding into entries, transactions, or filtered events. Raw access preserves protocol detail and early timing while making validation, FEC recovery, ordering, and upgrades the receiver’s responsibility. |
| Raw UDP | Delivery of datagrams without an application framing connection, reliable stream, or decoded schema. It preserves message boundaries but provides no ordering, retransmission, or receiver backpressure. |
| Receive Flow Steering | A Linux mechanism, abbreviated RFS, that steers protocol processing toward the CPU consuming a flow. It can improve locality for many flows but adds tables and cross-CPU work that one dedicated tuple may not need. |
| Receive Packet Steering | A Linux software mechanism, abbreviated RPS, that hashes packets onto selected CPUs after NIC receive. It helps when hardware queues are insufficient, but redundant steering can add interprocessor interrupts and cache movement. |
| Receiver | The host and process that bind the destination UDP port, admit the documented source, and consume datagrams. Its responsibilities include loss accounting, safe parsing, authentication, recovery, ordering, decoding, and upgrade readiness. |
| Recent blockhash | A transaction-message field referencing recent ledger state and constraining how long the transaction remains processable. Observation in a shred does not prove that the blockhash check or transaction execution will succeed. |
| Recovery | Reconstruction of missing data shreds from a sufficient compatible set of data and coding shards. Recovered output is untrusted until it passes the same sanitation and authentication rules as received data. |
| recvmmsg | A Linux system call that receives multiple messages into an array in one call. It reduces syscall overhead at packet rate, while timeout and batch-size choices determine how much latency the receiver trades for amortization. |
| recvmsg | A socket call that receives one message plus ancillary control data. It is the standard path for obtaining timestamps, overflow notifications, interface data, and message flags alongside a UDP payload. |
| Reed-Solomon | An erasure code that treats shards as symbols and derives parity sufficient to reconstruct missing members within its limits. Correct recovery requires matching k, m, shard size, position, and protected byte layout. |
| Reference tick | A saturated value in the low six bits of a data shred’s flags. It provides protocol timing context but is not a wall-clock timestamp or a substitute for a receiver clock. |
| Repair | Solana’s mechanism for requesting missing ledger data outside the initial Turbine delivery. A raw feed with no retransmission does not turn a missing UDP datagram into an automatic provider repair response. |
| Replay | The validator stage that executes reconstructed ledger entries against parent state and checks ledger rules. It produces transaction outcomes and account changes that are absent from raw shred bytes. |
| Retransmission | Sending data again after initial transmission or forwarding a received copy onward. The shredstream.sh wire contract does not retransmit missing customer datagrams, while Turbine nodes can forward shreds as part of network propagation. |
| Retransmitter | A node that receives a shred and forwards it to downstream Turbine peers. Its path position affects arrival, and some Merkle variants can carry separate retransmitter-signature evidence. |
| Retransmitter signature | An additional signature present in supported retransmitter-signed Merkle variants. It has a separate role and location from the leader signature and must never replace leader-origin verification. |
| Reverse path filtering | A Linux source-validation check that compares an inbound packet with the route back to its source. Strict mode suits symmetric single-homed routing, while loose mode can prevent silent drops on approved asymmetric hosts. |
| Root | A slot accepted as a durable ancestor under the validator’s fork and commitment state, or a tree hash depending on context. Say rooted slot or Merkle root because the bare word is ambiguous. |
| RPC | Solana’s request-response interface for querying processed ledger state, accounts, blocks, and subscriptions. RPC data is higher-level and later than raw shreds because reconstruction and usually replay occur first. |
| RSS | Receive Side Scaling, a NIC feature that hashes flows across hardware receive queues. It can distribute many flows, but one stable source and destination tuple commonly stays on one queue to preserve ordering and cache locality. |
S
| Term | Definition |
|---|---|
| Sanitization | Checked validation that a parsed structure satisfies length, range, flag, index, and variant invariants before use. Deserialization alone is insufficient because hostile bytes can decode into structurally possible but invalid values. |
| Scheduler latency | Time between a thread becoming runnable and actually running on a CPU. It contributes to application-dequeue tails and can rise when softirq work, busy polling, logging, or other processes occupy the selected core. |
| Sequence gap | A missing expected identity between observed indices or sender sequence values. It detects absence but needs protocol context and host counters to distinguish upstream loss, local drop, intentional slot boundaries, and invalid packets. |
| Serialization | Converting structured values into a defined byte representation and reversing that process. Solana ledger and transaction structures use versioned binary rules, so field order, compact lengths, and endian choices must match the pinned implementation. |
| Shred | Solana’s signed transport fragment for distributing block data through Turbine. A shred is either data or coding, belongs to a slot and FEC set, and arrives before later replay, confirmation, and finality. |
| Shred data | The sanitized ledger byte slice exposed by a parsed data shred. It excludes every header, padding region, coding payload, Merkle proof node, chained root, and retransmitter signature. |
| Shred index | A 32-bit position inside either the data or coding index space for a slot. Shred type must accompany the numeric value because the two spaces can contain the same index. |
| Shred variant | The packet-format selection encoded by the variant discriminator. It determines role, legacy or Merkle family, proof size, chaining, retransmitter evidence, offsets, capacity, and authentication procedure. |
| Shred version | A 16-bit cluster-lineage value in the common header. It is a cheap cross-cluster filter, not a software release, packet-format semantic version, or proof of authenticity. |
| Signature | Fixed cryptographic evidence that a private key authenticated defined bytes or a defined hash. Verification requires the correct public key and exact variant-specific signed message, and it says nothing by itself about fork survival. |
| Skipped slot | A scheduled slot for which no accepted block becomes part of the ledger history. Seeing no complete valid shred set may be consistent with a skip, but the receiver needs cluster state before assigning the cause. |
| Slot | Solana’s numbered block-production opportunity and ledger position. A slot can produce many data and coding shreds, can sit on a losing fork, or can be skipped. |
| Slot leader | The validator assigned to produce a particular slot. The trusted leader schedule supplies its identity key for shred-signature verification. |
| Socket | A kernel object representing a network endpoint and its protocol state, queues, options, and counters. For UDP receive, binding selects the local tuple but does not create a connection or reserve remote sender behavior. |
| Socket buffer | Kernel-accounted memory holding received packets and metadata for a socket. It absorbs a bounded pause, while excessive capacity can convert visible loss into hidden packet age. |
| Softnet backlog | A per-CPU Linux input queue used when packets cannot finish receive processing immediately. Its drop and budget-exhaustion counters identify pressure before datagrams reach the UDP socket. |
| Source IP | The network-layer sender address observed on an inbound packet. Everything documented as sent by shredstream.sh, challenge and live stream alike, originates from 64.130.40.90. |
| SO_BUSY_POLL | A Linux socket option setting a per-socket busy-poll duration. It confines the CPU-latency trade to the receiver more narrowly than global busy-poll sysctls and still requires compatible NAPI support. |
| SO_RCVBUF | A socket option requesting receive-buffer capacity. Linux caps ordinary requests with net.core.rmem_max and commonly reports a doubled accounting value, so applications should read back what was applied. |
| SO_REUSEPORT | A socket option allowing multiple sockets to bind the same local address and port under defined rules. Linux can distribute datagrams among them, but one flow may remain concentrated unless a reuseport BPF policy or tuple diversity changes selection. |
| SO_RXQ_OVFL | A Linux socket option that can attach a cumulative receive-queue overflow count as ancillary data. Track deltas and handle counter wrap so loss near the socket is not confused with an upstream gap. |
| SO_TIMESTAMPING | A Linux socket interface for selecting software and hardware packet timestamps. Its flags, ancillary layout, hardware filters, and clock domains require version-aware setup and enough control-buffer space. |
| Stake | Voting weight associated with a validator under Solana’s consensus and network-selection rules. Turbine tree construction uses stake-weighted selection, but stake alone does not reveal one node’s exact path without the matching peer set and algorithm. |
| Stake-weighted fanout | Distribution in which peer selection probability or placement depends on stake. It helps spread data through validator capacity while making exact tree reconstruction depend on current stake, eligible peers, seed, and client logic. |
T
| Term | Definition |
|---|---|
| Thread affinity | Restricting a thread to selected CPUs. It can improve cache and queue locality when coordinated with IRQ and NAPI placement, or worsen latency when network and application work compete on one core. |
| Timestamp | A clock reading associated with a named event such as hardware ingress, kernel enqueue, application dequeue, or decode completion. Decimal precision does not establish accuracy, synchronization, placement, or a valid uncertainty bound. |
| Transaction | A signed Solana request containing a message and one or more signatures. Shreds can expose its bytes before execution, but success, logs, account changes, confirmation, and finality are later results. |
| Transaction message | The signed content that defines account access, recent blockhash, instructions, and optional address-table lookups. Legacy and versioned messages have different decode paths and must retain their original serialized form for signature checks. |
| Transaction Processing Unit | The validator pipeline, commonly abbreviated TPU, that receives, verifies, schedules, executes, records, and broadcasts transaction data during leader operation. Raw shreds appear near the output of record and shred stages, not at initial transaction ingress. |
| Turbine | Solana’s tree-like block-propagation protocol for distributing shreds through validator peers. The leader sends to an initial layer, and downstream nodes retransmit to their neighborhoods so the origin does not send every packet to every validator. |
| Turbine layer | A depth in the propagation tree between the leader and downstream recipients. More layers and intermediate scheduling can add arrival time, although physical route and host behavior can outweigh a nominal layer count. |
| Turbine tree | The per-shred or per-context peer arrangement used for fanout and retransmission. Reproducing it requires eligible peers, stake weights, deterministic seed, local-node treatment, and the exact client selection algorithm. |
U
| Term | Definition |
|---|---|
| UDP | The User Datagram Protocol, a connectionless transport preserving message boundaries without delivery, order, retransmission, or flow-control guarantees. Those omissions keep sender and receiver loosely coupled and make loss recovery an application concern. |
| UDP checksum | An integrity field covering the UDP header, payload, and an IP pseudo-header. A valid checksum detects common corruption but does not authenticate the sender or establish shred validity. |
| UDP receive queue | The per-socket queue of datagrams waiting for user space. When its charged memory reaches the limit, Linux drops later datagrams and increments socket and protocol counters. |
| UdpRcvbufErrors | A Linux UDP MIB counter incremented when datagrams cannot be queued because receive-buffer space is exhausted. It is host or namespace wide, so correlate its delta with the target socket’s drops and application gaps. |
| Unicast | Delivery from one network source to one destination address. shredstream.sh sends each customer destination a unicast UDP stream rather than asking the receiver to join a multicast group. |
| Unrecoverable gap | Missing data that cannot be reconstructed from the compatible shards available before the receiver’s closeout policy. The decoder must not bridge the gap by concatenating later bytes because entry boundaries and serialization would be corrupted. |
| User space | The process environment outside the kernel where receiver, recovery, decoder, and application logic run. Kernel counters can be clean while a bounded user-space queue overflows, so both sides need independent telemetry. |
V
| Term | Definition |
|---|---|
| Validator | A Solana node that participates in cluster operation by processing ledger data and, when eligible, voting and producing blocks. Raw receivers can consume shreds without being validators, but they still depend on validator-compatible protocol rules. |
| Variant byte | The one-byte common-header discriminator at offset 64. It must be parsed before variant-dependent offsets because it selects data or coding and the legacy or Merkle layout details. |
| Versioned transaction | A transaction whose message begins with a version marker and can add fields beyond the legacy message form. Version 0 supports address lookup tables, so decoding needs both static message keys and resolved loaded addresses. |
| Vote transaction | A transaction by which a validator records a consensus vote. It travels inside ledger entries like other transactions, while its program meaning and later effect require transaction and bank context. |
W
| Term | Definition |
|---|---|
| Wall-clock time | Civil time represented by a clock such as CLOCK_REALTIME and related to UTC. It can step under correction and is useful for logs, while monotonic or TAI-aligned domains are often safer for duration measurement. |
| Wire format | The exact byte representation exchanged between systems, including field order, widths, endian rules, discriminators, and authentication boundaries. A raw receiver must treat it as versioned input rather than infer it from a packet capture. |
X
| Term | Definition |
|---|---|
| XDP | eXpress Data Path, a Linux hook that runs an eBPF program early in receive processing. It can drop, pass, redirect, or transmit packets before the normal stack, so an incorrect program can make traffic disappear before UDP counters. |
Z
| Term | Definition |
|---|---|
| Zero-copy | A design that reduces data copying between NIC, kernel, and user buffers through shared or registered memory. It can lower memory bandwidth and CPU cost, but buffer ownership, lifetime, backpressure, and packet validation remain explicit responsibilities. |
Questions
- What is the difference between a shred and a transaction?
- A shred is a signed transport fragment of serialized ledger data or recovery parity. A transaction is a signed request contained inside an entry after data shreds are ordered, gaps are recovered, and the ledger stream is decoded. One transaction can cross shred boundaries, and one shred can contain parts of several transactions.
- Which Linux terms matter first when packets are missing?
- Start with NIC ring and driver drops, softnet backlog drops, UdpRcvbufErrors, per-socket overflow, application queue rejects, and sequence gaps. Each names a different stage. The first counter that rises during the same interval locates the loss more reliably than changing every buffer and affinity setting together.
- Why are timing terms defined so narrowly?
- A timestamp means little without its clock domain and placement. Hardware ingress, kernel enqueue, application dequeue, and decode completion include different work. Clock offset, asymmetry, holdover, and steps can exceed the claimed feed difference, so precise vocabulary is part of the measurement instrument rather than editorial preference.