Solana shred feeds for analytics platforms

6 stages separate freshness from certainty

An analytics platform turns transactions into prices, volumes, flows, liquidity measures, alerts, and charts. A shred feed can start that transformation before a completed block reaches normal interfaces. The first published metric is provisional because its source block is provisional.

shredstream.sh delivers raw UDP from Frankfurt to one IP and port. The plan is $100 for 30 days. The feed measured 54.3 Mbps, 5,585 packets per second, and 1,216 mean packet bytes on 2026-08-31, totaling 17.6 TB over 30 days. A platform must budget network and packet work before it budgets dashboards.

Low-latency analytics is valuable for machine clients, risk monitors, and live market views. It is dangerous when an API removes the qualifiers that make early data accurate. Freshness and commitment need separate fields, service levels, and user language.

Shreds show proposed execution

Solana has no public mempool. Transactions decoded from shreds are already included in a leader's proposed block. They are earlier than ordinary processed or confirmed delivery, but they are not future intent. A metric built from them describes proposed activity observed at a receiver.

The source transaction may fail. Its slot may be skipped. Its fork may lose. A later transaction in the same slot may reverse a price move. Analytics built from an entry prefix should never be labeled final block analytics.

A clean model has observed, processed, confirmed, and finalized views. Clients can choose. Streaming consumers receive status updates or retractions. Historical endpoints default to a documented commitment rather than silently mixing stages.

Fields turn bytes into market events

Slot, shred index, entry order, and transaction order preserve sequence. This is necessary for price paths and volume windows. FEC set index and variant preserve recovery provenance. Completion flags show whether an entry range or the slot is complete.

Leader signature validates proposed source. Shred version selects the decoder. Account keys and program IDs map instructions to venues, pools, mints, vaults, and known entities. Instruction data supplies direction, limits, deposits, withdrawals, borrows, repayments, and other supported semantics.

Token balances and economic transfers are not always stated in one instruction. Multi-program transactions can wrap SOL, route through several pools, create accounts, and close accounts. A correct parser needs protocol-specific rules or execution output from a later source. The fast path should publish only metrics it can derive defensibly.

Transaction signature is the event identity for deduplication and commitment reconciliation. Arrival, entry-ready, decoded, aggregated, and published timestamps show where freshness was spent. A single timestamp named time conceals too much.

A defensible latency budget

The first five stages total 2.6 to 24.4 milliseconds. That range describes arrival of required bytes through availability on a low-latency serving stream. It does not include propagation from the leader to Frankfurt, network travel to the receiver, or the later commitment interval.

Detection is subscription classification. Decode is the expensive stage because bytes become entries, transactions, resolved account keys, and supported events. Decision applies metric rules and confidence. Signing is zero for an ordinary unsigned API. Submission means publish to the platform's serving layer. Land means the source event reaches a stated chain commitment, which the platform observes but does not cause.

These definitions prevent a category error. An analytics service does not sign or land the observed transaction. It may sign a data batch, and it may wait for the chain. The table preserves the requested six-stage frame while stating which stages differ from trading.

Work a provisional price example

Assume a decoded swap sends 50 SOL into a supported constant-product pool and receives 4,900 token units. The direct execution ratio is 50 divided by 4,900, or about 0.010204 SOL per token before separating fees and route effects.

The platform verifies that the accounts match the registered pool, identifies input and output mints, applies decimals, and checks whether the transaction contains another swap leg. If this is the only relevant leg, it publishes a provisional trade with source slot, signature, sequence, amount, derived price, parser version, and observed commitment.

A one-minute volume aggregate adds 50 SOL and 4,900 tokens at observed status. When the transaction becomes confirmed, the platform moves its contribution from observed to confirmed without adding it again. If the fork loses, it subtracts the observed contribution and emits a retraction.

An all-commitment chart that adds on observation and never retracts will overstate volume. A confirmed-only chart can remain stable but arrives later. Both are valid products when labeled.

Price is a method, not one division

Pool trades can include fees, transfer-fee tokens, exact-input or exact-output limits, multi-hop routes, and inner instructions. Reserve changes may include deposits or protocol fees unrelated to the displayed trade. A platform should document whether price comes from instruction amounts, token balance deltas, reserve deltas, or a later execution trace.

Volume should exclude self-transfers and avoid double-counting route legs according to a published rule. Liquidity can mean reserve value, executable depth within a percentage, or total deposited value. Market capitalization can use circulating supply, total supply, or another definition. Early delivery does not resolve ambiguous metrics.

Method version belongs in the data. If a parser correction changes volume, historical rows should identify which calculation produced them. Backfills must not silently blend old and new semantics.

Slow clients cannot slow UDP

Raw UDP provides no backpressure. Neither should one analytics websocket. The receive pipeline, decode pipeline, aggregation log, and client fanout need separate bounded queues. Slow clients can be disconnected or moved to snapshots plus deltas under a documented rule.

If client fanout shares a process queue with decode, a popular dashboard can cause packet loss. Persist or publish into an independent log before serving broad queries. Record queue age at every boundary. A low packet-arrival timestamp is meaningless when the event waits 500 milliseconds in aggregation.

Batching trades latency for throughput. A one-millisecond aggregation window may reduce publish calls while staying inside the budget. A 100-millisecond window may be suitable for human charts but not machine alerts. Expose different products rather than one hidden compromise.

Loss and recovery affect metric confidence

UDP has no ordering or retransmission. Duplicate shreds and missing indexes are expected receiver conditions. Use FEC coding shreds for recovery when possible. Mark unrecovered ranges and prevent incomplete state from appearing complete.

At 5,585 packets per second, batch system calls and memory reuse matter. Monitor interface drops, kernel receive-buffer errors, application drops, gap counts, recovery duration, decode errors, parser quarantine, aggregate lag, and publish lag. Store enough raw provenance to reproduce disputed metrics.

The feed source is 64.130.40.90. The firewall should allow inbound UDP from that address to the destination port. The 600-second challenge tests reachability before activation. It is not a sustained throughput or completeness test.

A secondary confirmed source should reconcile signatures, transaction order, success status, and account effects. Differences become corrections and parser tests, not hidden cleanup.

Freshness needs named clocks

Report receiver arrival to publish latency separately from estimated chain event time to receiver arrival. The latter requires trustworthy timestamps at different locations and is harder to measure. Clock synchronization error can exceed an attractive sub-millisecond claim.

Use a monotonic clock for intervals. Use synchronized wall time for cross-host joins. Record whether the timestamp came from the kernel, receive loop, decoder, or publisher. Publish percentiles and sample counts over a dated window.

An API can expose source_slot, commitment, observed_at, decoded_at, published_at, parser_version, and completeness. Clients then decide whether a 15-millisecond observed metric or a later confirmed metric fits their use.

Product analytics must admit corrections

Human interfaces should show a provisional marker without turning every fork event into noise. Machine streams need explicit update and retract event types. Cache keys must include commitment. Alerts should state whether they fired on observed or confirmed state.

Billing, compliance, and customer statements should use the required canonical commitment, not the fastest view. Research can store every branch. Trading dashboards may prefer observed state with visible confidence. One data pipeline can support all three only when it retains provenance.

Do not infer transaction intent from account labels alone. A known exchange address can handle internal operations. A large transfer can be collateral movement rather than a sale. Labels need sources, capture dates, and confidence separate from shred facts.

When raw shreds are not the right input

Daily reports, finalized portfolio statements, and broad historical research do not require millisecond proposed data. A decoded or archival source can lower engineering cost and offer replay. Raw shreds fit live products whose users can act on provisional state and whose operator can reconcile it.

Frankfurt-only delivery to one IP and port is not a global analytics edge by itself. There is no ordering, retransmission, backpressure, historical API, or guaranteed regional redundancy in raw delivery. Build those layers only where the product needs them.

Acceptance needs a correction test

Run the provisional stream beside a confirmed reference for a representative busy window. For every supported event, compare signature, slot, transaction order, success status, account mapping, amounts, and derived metric contribution. Count false positives, missing events, retractions, parser quarantines, and differences caused by method rather than transport.

Inject a losing fork into replay. The observed stream should publish it under provisional status, the confirmed view should exclude it, and clients should receive a deterministic retraction. Restart the publisher between observation and retraction to prove that correction state survives process failure.

Measure freshness from kernel arrival to client receipt with a receiver on the client side. Server publish time alone omits queueing and fanout. Report p50, p95, p99, maximum, sample count, and the percentage of events that required recovery. Segment results by program because one expensive parser can distort a global percentile.

Set an error budget for incomplete input. A platform might require 99.99 percent of supported canonical events to appear in its confirmed stream during the window, while permitting a documented provisional retraction rate. The exact target belongs to the operator. Publishing it makes the trade between speed and correction visible.

Finally, force a slow client, a full disk, a parser exception, and a burst above the measured mean. Packet receive must stay live or enter an explicit degraded state. An analytics platform is ready when failure changes a visible confidence label, not when every happy-path chart updates quickly.

The credible claim is measured and narrow. Required bytes can become a provisional metric in roughly 3 to 24 milliseconds inside a prepared platform. Commitment still takes chain time. Accuracy comes from preserving the distinction.