Reference

Shred stream data dictionary

Define every network observation, common header, data header, coding header, Merkle element, recovery key, entry, and transaction field used in a raw shred pipeline.

Representation layers

Layer	Structure	Delivered directly	Purpose
Network	IPv4 and UDP observation	Yes, as packet metadata	Route and admit one datagram
Service verification	Verification datagram	During activation	Prove control of a destination
Solana transport	Shred	Yes, as UDP payload	Carry data or recovery material
Recovery	FEC-set state	No, receiver-derived	Reconstruct missing data shreds
Ledger	Entry stream	No, receiver-derived	Restore Proof of History batches
Transaction	Versioned transaction	No, receiver-derived	Decode signed messages and instructions
Execution	Status, logs, balances, account writes	No	Produced later by replay or RPC

There is no documented shredstream.sh wrapper around the raw shred. Offset zero of the live UDP payload is offset zero of the Solana shred.

Network observation fields

Field	Type	Source	Meaning	Required handling
source_ip	IPv4 address	IP header	Sender address	Must equal 203.23.178.228 for documented service traffic
source_port	Unsigned 16-bit port	UDP header	Sender port	Value and stability are not currently specified
destination_ip	IPv4 address	IP header or socket context	Registered customer address	Compare with the configured interface or NAT path
destination_port	Unsigned 16-bit port	UDP header or socket binding	Registered customer port	Bind and firewall the configured value
received_length	Byte count	Receive API	Actual UDP payload length	Reject truncation and values above 1,228
receive_timestamp	Clock-specific timestamp	NIC, kernel, or application	Observation time at a named boundary	Record clock and timestamp type
interface_index	Interface identifier	Packet metadata or capture	Ingress interface	Useful on multihomed receivers
rx_queue	Queue identifier	NIC or socket metadata	Hardware or software receive queue	Use for affinity and drop diagnosis
ancillary_flags	Bit set	Receive API	Truncation and control-message state	Reject MSG_TRUNC and count MSG_CTRUNC

source_ip, source_port, destination_ip, destination_port, timestamps, interface index, and queue are not bytes inside the shred payload.

Verification datagram

Field	Type	Documented value or rule
first_line	ASCII text	SHREDCAST-VERIFY
valid_for	Duration	600 seconds
source_ip	IPv4 address	203.23.178.228
transport	Protocol	UDP
remaining payload format	Byte sequence	not currently specified
token encoding	Encoding	not currently specified
destination identifier encoding	Encoding	not currently specified

Parse the documented prefix and treat the remaining bytes as an opaque challenge unless a separate current interface defines them. Do not apply the shred parser to a verification datagram.

Shred common header

Field	Offset	Width	Logical type	Meaning
signature	0	64	Ed25519 signature	Leader authentication material
variant	64	1	Discriminator byte	Selects role, legacy or Merkle family, proof count, and flags
slot	65	8	Unsigned 64-bit little-endian	Slot number
index	73	4	Unsigned 32-bit little-endian	Position within the data or coding index space
version	77	2	Unsigned 16-bit little-endian	Cluster-lineage shred version
fec_set_index	79	4	Unsigned 32-bit little-endian	First data index of the FEC set

Data-shred header

Field	Offset	Width	Logical type	Meaning
parent_offset	83	2	Unsigned 16-bit little-endian	Slot minus parent slot
flags	85	1	Bit field	Reference tick plus completion bits
reference_tick	In flags bits 0 through 5	6 bits	Saturating unsigned value	Timing reference carried by the data shred
data_complete	In flags bit 6	1 bit	Boolean	Current data unit ends at this shred
last_in_slot	In flags bit 7 with data-complete state	1 bit	Boolean	This is the final data shred in the slot
size	86	2	Unsigned 16-bit little-endian	End offset of header plus ledger data
data	Variant-derived	Variable bytes	Serialized ledger byte slice	Input to ordered deshredding
padding	Variant-derived	Variable bytes	Non-ledger bytes	Never append to the ledger stream

Coding-shred header

Field	Offset	Width	Logical type	Meaning
num_data_shreds	83	2	Unsigned 16-bit little-endian	Data shard count k
num_coding_shreds	85	2	Unsigned 16-bit little-endian	Parity shard count m
position	87	2	Unsigned 16-bit little-endian	Coding position from zero through m minus one
erasure_position	Derived	Integer	k plus position	Local Reed-Solomon shard index
first_coding_index	Derived	Unsigned integer	index minus position	Consistency key for coding members
coding_payload	Variant-derived	Fixed per compatible set	Reed-Solomon parity bytes	Input to recovery, never entry bytes

Variant fields

Field	Applies to	Meaning
shred_type	Every variant	Data or coding role
format_family	Every variant	Legacy or Merkle
proof_size	Merkle variants	Number of proof entries encoded by the discriminator
chained	Merkle variants	Whether a preceding-root commitment is present
retransmitter_signed	Merkle variants	Whether an extra retransmitter signature is present
signed_message	Every variant	Variant-defined bytes or root authenticated by the leader signature

Use the receiver's pinned Agave release for exact numeric discriminants and legal flag combinations. Never accept an unknown combination by masking it into a known family.

Merkle material

Field	Type	Meaning	Presence
leaf	Hash input	Domain-separated hash of the protected shard representation	Derived for every Merkle shred
proof	Ordered hash array	Sibling nodes used to reconstruct the Merkle root	Merkle variants
merkle_root	Hash	Root reconstructed from leaf, proof, and erasure position	Derived for every Merkle shred
chained_root	Hash	Commitment to the preceding FEC set root	Chained Merkle variants
retransmitter_signature	Ed25519 signature bytes	Separate retransmission evidence	Retransmitter-signed variants
leader_signature	Ed25519 signature bytes	Authenticates the variant-defined root	Every Merkle shred

Proof position follows the erasure-shard location rather than global coding index. Exclude proof bytes, chain roots, and retransmitter signatures from ledger data.

Receiver-derived identity fields

Field	Type	Derivation	Use
cluster_id	Trusted configuration	Genesis or deployment context	Prevent cross-cluster key collisions
shred_type	Enum	Parsed variant	Separate data and coding identities
shred_id	Tuple	cluster_id, slot, shred_type, index	Deduplication and timing match
fec_id	Tuple	cluster_id, slot, fec_set_index	Recovery-state lookup
canonical_bytes_hash	Hash	Received UDP payload	Detect conflicting duplicates
leader_pubkey	Ed25519 public key	Trusted leader schedule for slot	Signature verification
parent_slot	Unsigned 64-bit	slot minus parent_offset	Fork and ledger relationship
data_end	Offset	Data header size field	End of ledger bytes in the shred
completion_kind	Enum	Data flags	Data-complete, last-in-slot, or neither
arrival_rank	Unsigned integer	Order among copies of shred_id	First-detection and duplicate metrics

FEC-set state

Field	Type	Meaning
slot	Unsigned 64-bit	Slot shared by every set member
fec_set_index	Unsigned 32-bit	First data index for the set
num_data_shreds	Unsigned integer	Required data shard count k
num_coding_shreds	Unsigned integer	Available parity shard count m
shard_size	Byte count	Protected size shared by compatible members
present_data	Bit set	Data positions currently available
present_coding	Bit set	Coding positions currently available
missing_data	Bit set	Data positions not yet present
recoverable	Boolean	At least k compatible total shards are available
authenticated_root	Hash or absent	Common Merkle context for Merkle sets
recovery_attempts	Counter	Number of reconstruction attempts
completed_at	Timestamp or absent	Time the set became usable
invalid_reason	Enum or absent	Structural, proof, conflict, or recovery failure

Mathematical recovery begins once the receiver holds k or more compatible shards. Every reconstructed data shred still requires full sanitation and authentication.

Ordered data stream

Field	Type	Meaning
slot	Unsigned 64-bit	Slot being assembled
first_data_index	Unsigned 32-bit	First index in the contiguous range
last_data_index	Unsigned 32-bit	Last index in the contiguous range
bytes	Byte sequence	Concatenated sanitized data slices
data_complete_boundaries	Index list	Ends of entry-bearing data units
last_in_slot_index	Optional index	Final data shred when observed
gap_set	Index set	Missing unrecovered data indices

Concatenate data slices only. Coding payload, headers, padding, and Merkle material never enter this byte stream.

Entry structure

Field	Logical type	Meaning
num_hashes	Unsigned 64-bit	Number of Proof of History hashes represented before the entry hash
hash	32-byte hash	Proof of History result for the entry
transactions	Vector of versioned transactions	Transactions recorded in this entry

Ledger serialization defines entry boundaries independently of UDP packets. An entry may continue across multiple shreds, while one fragment may mix bytes belonging to several entries.

Versioned transaction structure

Field	Logical type	Meaning
signatures	Compact-length vector of 64-byte signatures	Signatures corresponding to required signer keys
message	Legacy or versioned message	Signed transaction message
message_version	Legacy marker or version number	Selects message decoding

Message header

Field	Logical type	Meaning
num_required_signatures	Unsigned byte	Count of signer account keys
num_readonly_signed_accounts	Unsigned byte	Signer keys that are read-only
num_readonly_unsigned_accounts	Unsigned byte	Non-signer keys that are read-only

Legacy and versioned message fields

Field	Applies to	Meaning
header	All messages	Signer and read-only counts
account_keys	Legacy message	Ordered static public keys
static_account_keys	Versioned message	Keys stored directly in the message
recent_blockhash	All messages	Recent blockhash used by transaction processing
instructions	All messages	Compact vector of compiled instructions
address_table_lookups	Version 0 message	Lookup-table account plus writable and read-only index lists
loaded_addresses	Version 0, derived	Public keys resolved from address lookup tables

Address lookup contents are not carried inside the shred beyond the lookup references in the transaction. Resolving loaded addresses requires the relevant on-chain lookup-table state.

Compiled instruction fields

Field	Logical type	Meaning
program_id_index	Unsigned byte	Index of the program account in the resolved account-key list
accounts	Compact vector of unsigned-byte indices	Accounts passed to the program
data	Compact byte vector	Program-defined instruction payload

Instruction data has no universal schema. Decode it only with the selected program’s instruction definition and the correct program version.

Fields not present in raw shreds

Field or record	Why it is absent
commitment level	Determined later by cluster voting and fork state
finalized flag	Determined later by consensus
transaction execution status	Produced during replay
log messages	Produced during execution
compute units consumed	Produced during execution
pre and post balances	Produced by execution and RPC metadata
account-state diff	Derived from replay or account updates
block time	Assigned or estimated outside the raw shred header
RPC context slot	RPC response metadata, not shred bytes
provider sequence number	No service envelope is documented
provider receive timestamp	No sender timestamp field is documented

Seeing a signed transaction in a valid leader shred does not mean that transaction succeeded, became rooted, or was finalized.

Questions

Does shredstream.sh add fields around each shred?: No service envelope is documented. One live UDP payload is one raw Solana shred, so payload offset zero is the shred signature. Source address, destination address, ports, receive timestamp, interface, and queue are network or receiver metadata. They are not fields inserted before the Solana bytes.
Are entries aligned to shred boundaries?: No. UDP fragmentation into shreds does not align entry serialization. An entry may occupy several data fragments, and one data shred may carry bytes belonging to multiple entries. The receiver orders sanitized slices, recovers gaps, respects completion boundaries, and runs the ledger entry decoder over continuous bytes.
Do raw shreds contain transaction status and logs?: No. Raw shreds can be reconstructed into entries and signed transactions, but execution status, log messages, compute consumption, account changes, balances, commitment, and finalization arise later during replay and consensus. A transaction observed in a shred remains proposed data until those later stages supply their own evidence.