A complete documentation of how Shark Snips Consensus is computed, weighted, and verified.
This document is public methodology by design. We believe a Consensus product is only credible if its methodology is fully open, replicable, and stress-tested by skeptical users.
TL;DR for the degens
- Consensus combines predictions from published marketplace models, house models, and public available community models. Assigned tiers set starting weights; models without a recognized tier default to Standard.
- Tier-based weights are active. Automatic Brier-based tier promotion is planned.
- Creator concentration is adjusted toward a 5% cap, with feasibility and iteration limits described below. Forked-model correlation deduplication is planned.
- Lower Brier is better: 0.0 is perfect and 0.25 is the coin-flip baseline. Consensus currently enforces no maximum Brier score for eligibility.
- Free includes on-device building and backtesting. Pro includes up to two long runs per user per week, subject to shared availability. Open Pricing to compare Free and Pro access.
Every prediction logged. Every miss logged. The track record is the methodology.
What Consensus is
Sharksnip Consensus is a weighted aggregate of eligible predictions from published marketplace models, house models, and public available community models. Pack Breaker tournament submissions do not currently contribute; that integration is planned. For each upcoming game in supported sports, we compute:
- Consensus probability — what the weighted aggregate of models thinks
- Consensus pick — the implied recommendation (cover spread, take the over, etc.)
- Dispersion — how much the constituent models disagree
- Market divergence — how much the Consensus diverges from the Vegas line
The product is updated as new model predictions arrive (typically multiple updates per day in-season).
Current model weights
A published marketplace model uses its assigned Consensus tier to set its starting weight. A missing or unrecognized tier defaults to Standard. House models and public community models without a published marketplace card use Standard. The creator-cap adjustment can change final weights before aggregation. Brier scores do not automatically change weights or exclude models.
Eligibility — what contributes
Three source types can contribute to Consensus:
Source type 1: Marketplace models
Published marketplace models can contribute eligible predictions. Publication is the inclusion condition; the calculation does not check a separate Consensus opt-in. Unpublished marketplace cards are not admitted as published cards.
Source type 2: House models
Shark Snip's registered house models can contribute eligible predictions.
Source type 3: Public community models
Community models marked public and available can contribute eligible predictions even without a published marketplace card.
A contribution must supply a finite probability between 0 and 1 for a supported market on an upcoming game. There is no maximum allowed Brier score for Consensus eligibility. Brier measures prediction error; it is not an eligibility cutoff.
Pack Breaker integration (planned)
Pack Breaker tournament submissions do not currently contribute. Resolving a tournament or opting in at submission time does not add its predictions to Consensus. Private community models do not qualify through the public-model source; creators choose whether to make their models public.
Tier weights
The live calculation applies these starting weights before the creator-cap adjustment:
| Assigned tier | Starting weight |
|---|---|
| Verified Sharp | 5 |
| Established | 2 |
| Standard | 1 |
| Probationary | 0.25 |
A missing or unrecognized tier defaults to Standard. Contribution weights are separate from Free and Pro access. Featured Creator status alone does not set a Consensus tier or starting weight.
Automatic Brier-based tier promotion is planned. There is no Brier-score cutoff or automatic Brier-based promotion or exclusion in the current calculation.
Single-creator weight cap
Implemented weight adjustment and derivation
The aggregator attempts to reduce creator concentration using a configured cap (default 5%). It scales all of an affected creator's model weights proportionally; it does not select models through a dashboard or zero out additional models once a quota is reached.
For one over-cap creator, let S_c be its combined weight, S_o the weight of the other creators, and c the cap. Holding the other weights fixed, solve:
S_c_final / (S_c_final + S_o) = c
S_c_final = c * S_o / (1 - c)
scale factor = S_c_final / S_c
For a set O of simultaneously over-cap creators, hold the combined weight W_rest of the other creators fixed. If each creator in O receives share c of the new total:
T_new = |O| * c * T_new + W_rest
T_new = W_rest / (1 - |O| * c)
target weight for each creator in O = c * T_new
The joint solve requires a positive denominator. Each affected creator's model weights are multiplied by its target weight divided by its current combined weight.
Limits and fallback behavior
- Capping is applied only for
0 < c < 1. - If the remaining field has no positive weight, the implementation stops capping instead of collapsing every weight to zero.
- The near-infeasible-denominator guard uses independent per-creator solves, with at most five outer passes.
- The joint branch exits after its solve. It does not independently prove that previously uncapped creators remain below the cap after the total changes.
- Therefore, 5% is not an unconditional maximum for every returned snapshot. With fewer than 20 positively weighted creators, requiring every creator to have at most 5% is mathematically impossible.
Forked-model deduplication (planned)
Forked models can produce similar predictions. Correlation-based deduplication is planned and does not run in v1. The proposed approach groups models with highly correlated predictions into one combined vote.
Brier score (the calibration metric)
We use Brier score as our primary calibration metric. For probabilistic predictions:
Brier = (1/N) × Σ (p_i - o_i)²
Where:
p_iis the model's predicted probability for outcomeio_iis the actual outcome (1 if it happened, 0 if not)- N is the number of predictions
Lower is better. A perfect set of predictions scores 0.0. Predicting 0.5 for every binary outcome scores 0.25, the coin-flip baseline. Confident wrong predictions increase the score, up to 1.0 when every prediction is certain and wrong.
These values explain how to read Brier scores. Consensus currently enforces no maximum allowed Brier score for eligibility.
Calibration and accuracy
Brier score measures how closely predicted probabilities match resolved outcomes. Hit rate measures the share of scored picks that were correct. Pushes and unresolved outcomes do not count as scored picks.
Read Brier score and hit rate with the number of scored picks and the evaluation period. Small samples can give an incomplete picture of accuracy. Neither measure automatically changes the assigned tier or its starting weight.
Aggregation formula
For each prediction, the Consensus probability is:
p_consensus = Σ (w_i × p_i) / Σ w_i
Where:
w_iis the assigned-tier starting weight of modeli(Standard by default) after the creator-cap adjustmentp_iis modeli's predicted probability
Dispersion is the weighted standard deviation:
dispersion = sqrt(Σ w_i × (p_i - p_consensus)² / Σ w_i)
Market divergence is:
divergence = p_consensus - p_market
Where p_market is the Vegas-implied probability from the closing line (or current line for pre-game updates).
Free and Pro access
| Plan | Access |
|---|---|
| Free | On-device building and backtesting |
| Pro | Up to two long runs per user per week, subject to shared availability |
Open Pricing to compare Free and Pro access.
What we DO NOT publish
- Identity of contributing creators (unless they've opted in to attribution)
- Individual model probabilities (the inputs to the weighted aggregate)
- Proprietary feature engineering (creators retain that)
Independent verification
We support and encourage independent academic and journalistic verification:
- Researchers can apply for read-only API access (no rate limit, full historical) at research@sharksnip.com
- Anonymized contributing-model data is available upon request for academic research
What can break Consensus
We list these openly because we want users to know the limits:
Sample size during ramp-up. Small contributor pools limit the creator-cap adjustment, as described above. Assigned tiers set starting weights, with Standard as the default. Automatic Brier-based promotion is planned.
Coordinated forking attacks. Many forks can repeat similar predictions. The creator-cap adjustment limits concentration where feasible. Correlation-based deduplication is planned.
Adversarial models. A model can show acceptable overall calibration while it makes systematic errors in a subset of predictions. Compare Brier score and hit rate across evaluation periods and sample sizes.
Changing market prices. Prices can change between prediction updates. Compare the prediction time with the current market before using the analysis. Free and Pro access are described above.
Sport-specific limits. Some sports have fewer resolved predictions. Compare sample sizes before interpreting calibration and hit rate.
We update this list as failure modes emerge.
Methodology version history
- Current v1: Published marketplace models, house models, and public available community models can contribute. Assigned tiers set starting weights, with Standard as the default. The creator-cap adjustment can change final weights.
- Planned: Pack Breaker contribution, automatic Brier-based tier promotion, forked-model correlation deduplication, per-sport sample-size adjustments, and per-market specialization weights.
Questions, suggestions, criticism
We welcome substantive feedback on the methodology:
- Email: methodology@sharksnip.com
- Discord: #methodology channel
If you find a flaw, we'd rather know now than discover it through bad outcomes later.