What are Merkle shreds in Solana?

1 signed root covers one FEC set

Merkle shreds change the authentication unit from one packet body to one FEC-set commitment. The leader builds a canonical Merkle tree over every data and coding shred in the set, signs the root with Ed25519, and places that signature in each member's common header.

Every packet carries enough sibling hashes to reconstruct the root from its own leaf. A receiver verifies the branch, obtains the root, and verifies the leader signature over that root. All consistent members of the set arrive at the same result.

The scheme binds source and parity traffic together. That matters because erasure recovery already operates at the FEC-set boundary. Authentication, reconstruction, and duplicate partitioning can refer to one commitment rather than a loose collection of individually signed packet bodies.

Leaves follow a canonical order

The tree is not built from arrival order. Data-shred leaves appear first in sequence, followed by coding-shred leaves in sequence. Header indexes, FEC metadata, and coding positions determine each leaf's canonical location.

Canonical order prevents two receivers from computing different roots from the same set because packets arrived along different network paths. It also makes position validation part of proof verification. A valid sibling list used at the wrong leaf coordinate should not recover the signed root.

A receiver must know the set dimensions to interpret the tree shape. Coding headers provide data and coding counts. Members must agree on those counts and the FEC anchor. Proof verification without structural agreement accepts too little context.

The signature covers the root

The 64-byte signature in a Merkle shred is an Ed25519 signature over the recovered Merkle root. It is not a signature over every byte after the signature field, which is the legacy model.

Equal signatures across different packet identities in one FEC set are expected. The packets share a root. Deduplication must still use slot, type, and index, not signature equality.

Verification needs the public key of the scheduled leader for the claimed slot. A cryptographically valid signature under another key does not make the packet admissible. Version checks, slot-window checks, and variant checks should run before or around the more expensive proof and signature operations.

Each leaf excludes its proof

The authenticated leaf material begins after the signature field and ends before the Merkle proof section. This prevents a proof from recursively authenticating itself.

The exact tail can also include chaining or resigned material according to the variant. Variant-aware layout code determines the leaf range. Using total UDP length as the leaf boundary includes proof bytes and yields a different root.

The leaf hash uses a domain-separation prefix before the packet bytes. Interior nodes use a different prefix before their child hashes. Domain separation prevents the same byte string from being interpreted interchangeably as raw leaf content and an internal tree node.

Hashes are 20-byte values

Merkle proof entries use SHA-256-derived hashes truncated to 20 bytes under the protocol specification. Each tree level contributes one 20-byte sibling value to the branch.

Truncation saves packet space. Proof bytes consume capacity that could otherwise carry ledger data or parity, so the protocol balances authentication overhead against packet size. A proof with six siblings occupies 120 bytes before any optional trailing fields.

Receivers should implement the specified truncation direction and domain prefixes exactly. A generic Merkle library that returns full 32-byte hashes is not wire-compatible without adaptation. A proof can have the right number of nodes and still be wrong because leaf hashing, node ordering, or truncation differs.

Proof size lives in the variant

Merkle variant bit patterns encode proof size in part of the one-byte discriminator. Other bits distinguish data from coding and identify chained or resigned forms.

Proof size controls how many 20-byte siblings are read near the packet tail. It also limits the number of leaves that the branch shape can authenticate. The receiver validates that size against packet length and declared FEC dimensions.

Do not scan backward until bytes look like hashes. Any 20-byte sequence can look hash-like. Decode the variant, calculate the required proof region, account for optional fields, and reject impossible layouts before hashing.

A proof reconstructs, not stores, the root

The packet does not need to carry a separate root field for ordinary Merkle verification. Starting with the leaf hash, the receiver combines each sibling in left-right order determined by the leaf index, hashes the parent, and repeats to the top.

The final computed value is the root signed by the leader. This saves a root field and ensures the branch itself determines the committed value. It also means a parser cannot fetch a root from one offset and trust it without executing the branch.

Cache the verified root at FEC-set scope. Later members still need their own branch checked, but leader signature verification over an identical cached root can be amortized carefully. Cache keys must include slot, FEC anchor, variant context, and the root itself to avoid cross-set reuse.

Chaining links adjacent commitments

Chained Merkle variants carry material that links a set's commitment to the prior chain context defined by the protocol. The link makes it harder to splice independently valid sets into an invented sequence.

The operational consequence is state. Verification may need the expected previous Merkle root or equivalent chain value, not only the current packet and slot leader key. A receiver joining mid-slot can authenticate a current root signature while lacking earlier chain context needed for full continuity validation.

Chaining should have its own status. Marking a packet signature_valid does not imply chain_link_valid. Systems can decide whether leader attribution alone is enough for an early signal, but they should not label partial verification as complete continuity.

Resigned variants add another role

Resigned Merkle variants support an additional signature associated with retransmission behavior defined by the active protocol. That signature is distinct from the leader's root signature and consumes tail capacity.

Receivers must parse the variant before locating proof bytes because the optional signature changes tail offsets. They must also obtain the correct identity and message rules for the retransmitter check. Treating the trailing bytes as one extra proof node shifts every branch calculation.

Not every raw consumer needs to enforce retransmitter provenance for its application decision. Every decoder still needs to understand the layout so it extracts the correct leader-authenticated leaf and proof.

Merkle and erasure coding interact

The tree contains both data and coding leaves, while Reed-Solomon derives coding content from source data representations. Recovery must preserve the commitment represented by the root.

A receiver groups members by one root view, reaches the k-shard threshold, reconstructs missing source material, and then validates recovered data against the set's authentication structure. The supported implementation handles details needed to rebuild or verify proof-related packet form.

Never combine shards that reach different roots because their slot and FEC anchor match. They are different codewords or different commitments. A matrix can output bytes from such a mixture, but those bytes do not inherit either leader signature.

Merkle proofs improve conflict handling

With a root per FEC set, a receiver can name conflicting views compactly. Two packets under the same slot and anchor that recover different valid leader-signed roots are evidence of duplicate production or another serious conflict.

The root also helps prevent arbitrary coding shreds from being inserted into a source set. A coding member must prove inclusion under the same commitment as the data members before contributing an equation.

This does not make conflict resolution automatic. Two valid roots can both be attributable to the leader. Consensus and duplicate-proof mechanisms decide the wider consequence. A raw receiver should avoid mixing them, retain bounded evidence, and report both roots.

Verification order controls cost

Length, variant, version, slot horizon, index bounds, proof-size bounds, and coding-position checks are cheap. Run them before SHA-256 branch work and Ed25519 verification.

Compute the leaf and branch only after the packet has a plausible location. Check a cached root signature where safe. Maintain negative caches carefully, since invalid traffic can otherwise force repeated expensive work. Bound caches by slot horizon so attacker-selected roots do not consume permanent memory.

Batch verification can improve throughput, but latency-sensitive code should not wait to fill a large batch during a quiet period. A short time or count threshold usually gives a better tail-latency trade. Measure queue time separately from cryptographic time.

Payload capacity is variant-dependent

Every 20-byte proof node occupies packet space. Chaining and resigned fields consume more. Merkle data shreds therefore have less room for serialized entries than a comparable legacy packet, with exact capacity determined by variant and proof size.

Do not hard-code one data capacity for all shreds. Read the variant, calculate its layout, honor the data size field, and expose effective payload length as a parsed property.

This variation also changes how many data shreds a given entry stream needs. More packets can change FEC set boundaries and coding traffic. Capacity is a wire-format consequence, not an application-level guarantee based on the feed's mean packet length.

What Merkle verification establishes

A valid branch and leader signature establish that the packet leaf belongs to a particular FEC-set root authenticated by the expected slot leader. A valid chain link can additionally establish the supported continuity relationship.

The proof does not show that the slot is canonical, executed, confirmed, or rooted. It does not show that every other set member is available. One valid coding leaf reveals no transaction bytes by itself. A valid terminal set does not prove earlier sets were received.

Use explicit states: proof_valid, leader_signature_valid, chain_valid, source_recovered, entries_decoded, and commitment_status. Precise state names stop cryptographic membership from being mistaken for consensus finality.

Verification evidence should follow the packet

A hot pipeline often separates hashing, signature verification, FEC storage, and ordered assembly across threads. Passing only a boolean verified loses which root, key, variant, and chain context produced that result.

Attach a compact verification record to the admitted packet. It should identify the scheduled leader key, recovered root, FEC anchor, proof outcome, signature outcome, chain outcome when applicable, and verifier generation. The generation helps invalidate caches when leader-schedule or feature context changes.

FEC state accepts members only when their records name the same authenticated root and compatible set shape. Ordered assembly can retain the root as block-view provenance even after proof bytes are released. Conflict diagnostics can then show that two decoded transaction ranges came from different leader-signed commitments.

Avoid serializing this internal record as though it were an on-wire Solana structure. It is receiver evidence derived from packet bytes and local context. Keeping that distinction prevents a cache field or local timestamp from entering signature and leaf calculations.

Evidence records also make cache audits possible. Sampled packets can be reverified from raw bytes and compared with stored roots, keys, and outcomes. A mismatch identifies stale context or an implementation fault before affected packets reach entry assembly.