Reference

Shred feed benchmark methodology

Measure paired shred arrival, first-detection share, loss, clock uncertainty, and latency distributions without converting instrumentation error into a vendor claim.

Current publication stance

shredstream.sh does not currently publish comparative benchmark RESULTS. The service has a single upstream source, so there is nothing valid to race against inside the service. Publishing a comparative benchmark anyway would itself be exactly the kind of unverifiable claim this site exists to avoid making.

That is a methodological boundary, not an apology. A result begins when two independent, named observations can be paired under one instrument. Until then, the honest output is the method.

Define the question before collecting packets

Question	Required timestamps	What the result means
Which feed first exposed the same shred to one receiver	Earliest accepted arrival from each feed on one receiver clock	Relative observation and path advantage
How much earlier was one feed	Paired earliest arrivals on one receiver clock	Distribution of arrival deltas
What was provider-to-receiver one-way latency	Sender egress and receiver ingress on synchronized clocks	Absolute path interval with clock uncertainty
How long did the host retain a packet	Kernel or hardware ingress and application dequeue on one host	Receiver queue and scheduling delay
How long until a transaction was usable	Earliest shred arrival and completed decode decision	End-to-end customer processing interval

Do not call an application-to-application interval “wire latency.” Do not call half of round-trip time “one-way latency.” Name both timestamp boundaries in every chart and table.

Sample unit and identity

The comparison unit is one Solana shred identity: cluster context, slot, shred type, and index. FEC-set index and variant are retained as validation fields. A data shred and coding shred with the same slot and numeric index are not the same sample.

For each feed, retain the earliest valid copy of an identity. Count later copies as duplicates. Reject malformed packets before they enter timing results, but count and report every rejection reason.

The paired population is the intersection observed by all feeds under comparison during the same measurement window. One-sided observations remain loss or coverage outcomes and must not disappear from the report.

Sample size and duration

Use at least 1,000,000 matched shred identities for a report that includes p99.9. That leaves about 1,000 observations beyond the p99.9 threshold, which is still too few for casual conclusions about the maximum. Report the exact matched count rather than the target.

Run for at least 24 continuous hours and cover multiple leaders, busy and quiet slots, route conditions, host housekeeping cycles, and clock-servo behavior. Repeat the run on separate days. A larger sample from one stationary hour does not replace coverage of changing conditions.

Report total datagrams per feed, unique valid identities, matched identities, feed-only identities, duplicates, malformed packets, unsupported variants, and excluded clock intervals. A percentile without those denominators is incomplete.

Clock discipline

The strongest relative race sends all compared feeds into the same host, NIC timestamp domain, queue policy, and capture process. Subtracting two arrivals from one clock removes cross-host clock offset.

When feeds must land on different hosts, use PTP with supported hardware timestamps and a documented grandmaster path. Record PHC identity, servo state, estimated offset, path-delay observations, holdover intervals, clock steps, and PHC-to-system conversion. Bound uncertainty conservatively.

NTP is useful for UTC sanity and many millisecond-scale measurements. Ordinary NTP over a routed network is not adequate evidence for a microsecond-scale advantage unless its measured error bound is smaller than the claimed difference. Do not let PTP and NTP daemons discipline the same clock without an explicit ownership design.

Invalidate an interval when a clock steps, loses its source, changes grandmaster unexpectedly, enters holdover beyond policy, produces negative physical transit, or has uncertainty at least as large as the claimed difference.

Timestamp placement

Timestamp	Placement	Included delay
Hardware RX	NIC ingress	Path through the receiver PHY and hardware timestamp point
Kernel RX	Networking stack timestamp hook	NIC and driver work before the hook
Application dequeue	Immediately after receive call returns	NIC, kernel, socket residence, wakeup, and batching
Decode ready	First valid decoded output	Receive path plus validation, recovery, and decode
Strategy decision	Consumer decision boundary	Complete measured pipeline

Use the same timestamp type for every feed in one comparison. If hardware RX is unavailable for one feed, do not compare it with application dequeue on another and label the difference as network performance.

For each paired shred identity, compare the earliest valid timestamp from every feed.

Outcome	Rule
Feed A first	A arrives earlier than B by more than the tie threshold
Feed B first	B arrives earlier than A by more than the tie threshold
Tie	Absolute delta is no greater than the tie threshold
A only	Valid identity observed on A but not B inside the stated closeout window
B only	Valid identity observed on B but not A inside the stated closeout window
Neither comparable	Identity or timestamp failed a stated validity gate

The tie threshold must be at least the larger of timestamp resolution and measurement uncertainty. First-detection share for A is A-first divided by all decidable paired outcomes. Publish B-first, ties, A-only, and B-only beside it. Do not fold missing packets into wins.

For more than two feeds, record the first feed and all feeds inside the tie threshold. Pairwise shares and an all-feed winner table answer different questions and should both retain the same population definition.

Paired arrival deltas

Compute delta as arrival time for feed A minus arrival time for feed B on the same identity. State the sign convention next to every result. Report p1, p10, p50, p90, p99, p99.9, minimum, maximum, mean, matched count, and a histogram with resolution appropriate to the clock.

Split results by time window, leader, shred type, packet variant, receiver queue, and route state only when each group has enough samples. Publish the grouping rule before inspecting winners. Post-hoc groups selected because they flatter one feed are not evidence.

Loss and completeness

A fast feed that misses more packets may or may not be useful. Report feed-only identities, unrecoverable FEC sets, time to FEC readiness, time to ordered data readiness, and completed-slot coverage. Keep upstream absence separate from NIC, softnet, socket, application-ring, parser, and deduplication loss.

Collect NIC discards, softnet drops and time squeezes, UDP receive-buffer errors, per-socket drops, application queue rejects, decoder rejections, and sequence gaps during the same interval. A host that drops one input more often cannot produce a fair race.

Controls that must remain equal

Control	Requirement
Receiver hardware	Same host or matched documented hardware
NIC path	Same timestamp capability and comparable queue treatment
CPU placement	Fixed and recorded IRQ, NAPI, ingest, and worker affinity
Socket settings	Same receive buffer, timestamp mode, batching, and busy-poll policy
Firewall and conntrack	Same stateful or untracked treatment
Decoder	Same build, validation gates, and identity logic
Warm-up	Same exclusion rule fixed before the run
Storage and logging	Same bounded sampling and off-hot-path behavior
Traffic window	Simultaneous observation
Geography	Same receiving location

If a control cannot be equal, disclose the difference and narrow the conclusion to the whole tested system rather than the provider alone.

Errors that invalidate a result

Methodological error	Why the result fails
Comparing non-overlapping time windows	Leaders, load, and routes differ
Using different receiver clocks without a valid bound	Clock offset can be larger than the measured lead
Comparing different timestamp points	The result includes unequal software stages
Treating half-RTT as one-way delay	Forward and reverse paths need not be symmetric
Dropping one-sided observations	Missing data is removed from the denominator
Timing the first packet from any slot	Different shred identities are compared
Omitting shred type from identity	Data and coding shreds can collide
Keeping a later duplicate	Provider duplication behavior changes the race
Accepting malformed or unverified packets	Early invalid traffic can be scored as a win
Censoring late packets without reporting the rule	Tail latency is made to look smaller
Ignoring local packet drops	Receiver overload is attributed to the feed
Running one feed on a favored queue or core	Host placement becomes the measured advantage
Changing decoder or socket settings mid-run	The population contains different instruments
Mixing clock steps or route changes into one histogram	The aggregate describes multiple systems
Publishing only p50	Tail and loss behavior remain hidden
Publishing only the maximum	One event is presented as a distribution
Selecting leaders or slots after seeing outcomes	The selection is biased by the answer
Reporting fewer than the declared samples	The result no longer follows its registered method
Measuring one upstream against itself	The race lacks independent observations

Minimum publication record

Publish a run identifier, UTC start and end, receiver location, host and NIC, kernel, driver, firmware, queue mapping, socket settings, decoder revision, feed labels, timestamp type, clock state, uncertainty budget, tie threshold, identity rule, closeout window, sample counts, loss counters, quantiles, raw aggregation method, and every exclusion.

Retain enough packet identities and timestamps to reproduce aggregate results without publishing customer payloads. Sign or checksum the result artifact and keep the analysis code revision beside it.

Questions

Why does shredstream.sh publish a method but no comparison results?: The service currently has one upstream source, so there is no independent in-service feed to race. A comparative result produced anyway would be unverifiable. Publishing the method states exactly what evidence would be required and prevents a marketing number from appearing before a valid paired instrument exists.
How is first-detection share calculated?: Match the same cluster, slot, shred type, and index across feeds, then compare each feed’s earliest valid timestamp. Count a win only when the lead exceeds the declared tie threshold. Divide wins by decidable paired outcomes, and publish ties plus one-sided missing identities separately so absence cannot become a win.
Is NTP sufficient for a shred-feed race?: A same-host relative race needs only one stable receiver clock. Across hosts, NTP can support coarse millisecond questions when its measured uncertainty is smaller than the claimed difference. Microsecond claims normally require calibrated PTP and hardware timestamps, with clock state, asymmetry, steps, and holdover included in the uncertainty budget.