Prediction Scorecard

How accurate Creek Intelligence's forecasts have been lately. Experimental — self-grading, updated daily.
Updated 2026-07-17 17:25 UTC · scoring window: last 45 days

Every forecast the system makes is recorded and later graded against what the river actually did — this page is that report card. It currently scores 260 physics crest predictions, 2954 empirical likelihood forecasts, and 1461 recession countdowns (145 still maturing). Updated daily; treat it as experimental.

⚠️ Needs attention

Empirical engine over-warns (Big Piney Creek 90.0%, Buffalo at Boxley 95.9%, Cossatot River 89.3%, Hailstone (upper Buffalo) 95.8%, Illinois River (Hwy 16) 100.0%, Mulberry River 79.8%, Richland Creek 98.6% of 'rise likely' calls saw no rise). A lookup_tables.yaml rebuild (ARCH §16-11) is the fix.
Empirical band inversion: the heaviest-rain band is LESS reliable (64.4% no-rise) than the band below it (47.7%) — the rain-to-rise mapping is miscalibrated.
Physics crest predictors off-calibration: Richland Creek 44.0% within ±20% (bias 0.54 ft); Hailstone (upper Buffalo) 28.0% within ±20% (bias -330.78 cfs, but median err just 22.3 cfs — mean skewed by a few outlier events).
Recession big_piney/below_longpool 'low_floatable' timing bias: fires ~24 h early (river falls slower than predicted); MAE 27.8 h, n=41.
Recession buffalo/harriet 'low_floatable' rarely reached (100.0% never-reached, n=286); a recession_baseline may help.
Recession buffalo/pruitt 'too_low' timing bias: fires ~16 h early (river falls slower than predicted); MAE 27.2 h, n=109.
Recession buffalo/st_joe 'low_floatable' rarely reached (68.0% never-reached, n=238); a recession_baseline may help.
Recession mulberry/above_hwy_23 'low_floatable' timing bias: fires ~37 h early (river falls slower than predicted); MAE 41.3 h, n=50.
Recession mulberry/above_hwy_23 'too_low' timing bias: fires ~14 h early (river falls slower than predicted); MAE 30.9 h, n=67.
Recession mulberry/below_hwy_23 'low_floatable' timing bias: fires ~37 h early (river falls slower than predicted); MAE 39.7 h, n=54.

Physics Rise Predictors

Predicts the crest a gauge will reach from rainfall + antecedent moisture. Scored on how close the predicted peak was to the actual peak.

Predictor	Scored	Avg error (MAE)	Bias (mean / median)	Within ±20%
Cossatot River	154	0.67 ft	0.46 / 0.27 ft	71.0%
Richland Creek	59	0.97 ft	0.54 / 0.27 ft	44.0%
Hailstone (upper Buffalo)	47	430.89 cfs	-330.78 / 22.3 cfs	28.0%

Bias is the mean / median signed error (predicted − actual). A large gap between them means a few outlier events — often a single flash-flood onset — are dragging the mean; the median is the typical miss.

Worst recent misses

Cossatot River

2026-07-12T20:20: predicted 11.42, actual 2.98 (err +8.44 ft)
2026-07-15T14:20: predicted 6.86, actual 3.18 (err +3.68 ft)
2026-06-19T18:20: predicted 6.48, actual 2.91 (err +3.57 ft)

Richland Creek

2026-06-14T03:20: predicted 17.39, actual 9.42 (err +7.97 ft)
2026-06-14T04:20: predicted 16.05, actual 9.42 (err +6.63 ft)
2026-06-13T23:20: predicted 2.7, actual 7.21 (err -4.51 ft)

Hailstone (upper Buffalo)

2026-06-22T10:21: predicted 244.0, actual 6420.0 (err -6176.0 cfs)
2026-06-22T11:21: predicted 842.1, actual 6420.0 (err -5577.9 cfs)
2026-06-22T12:21: predicted 4744.3, actual 7300.0 (err -2555.7 cfs)

Empirical Forecast Engine

Predicts the likelihood of a rise to a given level from recent rainfall. Hit-rate = the river rose to at least the called level; no-rise-rate = the forecast rise never happened (lower is better).

Basin	Forecasts	Hit-rate rose to ≥ called level	No-rise rate rise never came	Outcome detail
Big Piney Creek	411	9.7%	90.0%	40 exact · 0 higher · 1 short · 370 none
Buffalo at Boxley	443	4.1%	95.9%	6 exact · 12 higher · 0 short · 425 none
Cossatot River	466	10.3%	89.3%	16 exact · 32 higher · 2 short · 416 none
Hailstone (upper Buffalo)	426	4.2%	95.8%	10 exact · 8 higher · 0 short · 408 none
Illinois River (Hwy 16)	360	0.0%	100.0%	0 exact · 0 higher · 0 short · 360 none
Mulberry River	411	19.2%	79.8%	79 exact · 0 higher · 4 short · 328 none
Richland Creek	437	1.4%	98.6%	0 exact · 6 higher · 0 short · 431 none

By confidence label

Confidence	n	Hit-rate	No-rise rate
high	82	57.3%	42.7%
medium	48	35.4%	64.6%
low	12	50.0%	50.0%

By rainfall percentile band (higher band = heavier rain vs. history)

Band	n	Hit-rate	No-rise rate
below_p25	1997	3.5%	96.4%
p25_to_p50	234	29.9%	70.1%
p50_to_p75	86	52.3%	47.7%
above_p75	59	35.6%	64.4%

A well-calibrated engine would show hit-rate rising with the band. Where it does not, the engine is over-warning — a known selection-bias issue tracked for recalibration.

Recession Countdowns

Predicts how long until a falling river drops to each threshold. HIT% = reached the level near the predicted time; MAE = average timing error (hours). Newly launched — most predictions are still maturing.

Gauge	Target	Graded	HIT%	Never reached	Timing MAE	Bias
big_piney/above_longpool	low_floatable	4	100.0%	0.0%	0.6h	0.5h
big_piney/above_longpool	too_low	44	73.0%	27.0%	13.7h	-5.3h
big_piney/below_longpool	low_floatable	41	93.0%	7.0%	27.8h	-24.3h
big_piney/below_longpool	too_low	114	83.0%	17.0%	22.2h	-1.1h
buffalo/harriet	low_floatable	286	0.0%	100.0%	—	—
buffalo/ponca	low_floatable	104	100.0%	0.0%	10.2h	-1.8h
buffalo/ponca	too_low	73	100.0%	0.0%	10.7h	-0.6h
buffalo/pruitt	low_floatable	81	94.0%	6.0%	13.8h	-7.6h
buffalo/pruitt	too_low	109	92.0%	8.0%	27.2h	-15.5h
buffalo/st_joe	low_floatable	238	32.0%	68.0%	15.8h	-11.5h
cossatot/cossatot	low_floatable	28	100.0%	0.0%	3.9h	-0.3h
cossatot/cossatot	too_low	74	100.0%	0.0%	6.5h	-0.1h
mulberry/above_hwy_23	low_floatable	50	94.0%	6.0%	41.3h	-37.3h
mulberry/above_hwy_23	too_low	67	60.0%	40.0%	30.9h	-13.5h
mulberry/below_hwy_23	low_floatable	54	89.0%	11.0%	39.7h	-36.7h
mulberry/below_hwy_23	too_low	92	45.0%	55.0%	14.9h	1.0h

4 additional gauge/target combinations have fewer than 4 graded outcomes (mostly still-censored or single-sample) and are hidden until they accumulate enough data to be meaningful. Bias is hours predicted − actual; negative means the countdown fires early.

Ponca AI Rainfall Event Analysis

When rain arms it, predicts how big the Ponca gauge will get — a class (Fizzle/Moderate/High/Flood), a typical-peak band, and a flood-risk %. Graded against Ponca's actual crest over the 36 h after each call.

Calls graded	Events	Class exact	Class within 1	Peak in IQR	Peak error (MAE)
420	16	84%	98%	76%	650.0 cfs

Flood-risk calibration (Brier score, lower is better; 382 calls since the raw analog was also logged): override floor 0.09 vs. raw k-NN 0.052 — the raw analog is currently the better-calibrated of the two — the deterministic override floor stays high through the post-crest recession. Small sample, directional only.

Recent events — predicted vs. actual

Event (UTC)	Calls	Predicted class	Max flood-risk	Actual crest	Actual class
2026-07-15T02:02	105	Fizzle	17%	65 cfs	Fizzle
2026-07-11T20:02	82	Fizzle	17%	71 cfs	Fizzle
2026-07-01T20:00	2	Flood	85%	112 cfs	Fizzle
2026-06-29T13:02	1	Flood	85%	224 cfs	Moderate
2026-06-27T13:02	52	Moderate	55%	758 cfs	Moderate
2026-06-24T20:02	50	Moderate	6%	493 cfs	Moderate

Buffalo Rise Engine (per-gauge nowcast)

For each Buffalo mainstem gauge, predicts a coming rise (slight / moderate / large) from local rain + upstream propagation, with a timing window. Graded on whether the gauge actually rose, and within the predicted window. Newly recording — predictions only fire during rain events, so this fills in over time.

Gauge	Graded	Rise happened	On-time	No-rise (false alarm)
boxley	0	—	—	0
ponca	40	20%	75%	32
pruitt	43	33%	57%	29
st_joe	75	20%	7%	60
harriet	49	10%	40%	44

Typical actual rise by predicted category: slight: ~445.0 cfs (n=18), moderate: ~410.0 cfs (n=24). (Categories come from rainfall, so this is how they map to real gauge rises — calibration that accrues over time.)

Recent rise predictions — predicted vs. actual

When (UTC)	Gauge	Predicted	Window	Outcome	Actual rise
2026-07-16T23:09	st_joe	slight	0.0-0.1h	rose	+49.0 cfs @ 17.3h
2026-07-16T22:09	st_joe	slight	0.0-1.1h	rose	+47.0 cfs @ 18.3h
2026-07-16T21:09	st_joe	slight	0.0-2.1h	rose	+47.0 cfs @ 19.3h
2026-07-16T20:09	st_joe	slight	0.0-3.1h	rose	+47.0 cfs @ 20.3h
2026-07-16T19:09	st_joe	slight	0.0-4.1h	rose	+53.0 cfs @ 21.3h
2026-07-16T18:09	st_joe	slight	0.0-5.1h	rose	+49.0 cfs @ 22.3h

Downstream Propagation (Ponca → Pruitt → St. Joe)

When Ponca rises, predicts whether and how big a bump reaches Pruitt and St. Joe, and how many hours after Ponca peaks. Graded on the bump/no-bump call, the crest size, and the timing. Newly recording — only logs during a Ponca rise.

Reach	Graded	Bump call right	Crest size hit	Timing hit
pruitt	4	25%	0%	0%
st_joe	4	25%	0%	0%

Routed Waves (wave tracker)

When an upstream gauge crests, the wave tracker predicts the crest size and arrival window at each downstream gauge. Every confirmed wave is graded after its window closes: did a real rise arrive, was the crest inside the predicted band, and did it arrive inside the window. Waves are rare events — this fills in slowly.

Recording started 2026-07-17 — 0 wave(s) in transit or maturing. Tracked waves only launch on a significant upstream rise, so grades appear storm by storm.

Neural Net Predictions (Ponca & Boxley)

A neural network projects the level hour by hour from 48 h of 1 km radar rainfall (Ponca 6 h ahead, Boxley 4 h). Every hourly issuance is graded against what the gauge actually did: MAE = average error of the median; IN-BAND% = how often the likely range contained the truth (target ~50%); SKILL = error reduction vs assuming the level doesn't change, on hours where it moved.

Ponca (6 h)

Ahead	Graded	MAE (cfs)	In-band	Skill vs no-change
+1 hr	136	1.3	76.5%	-12.4%
+2 hr	136	2.1	52.2%	-17.3%
+3 hr	136	3.1	59.6%	-20.1%
+4 hr	136	5.0	44.9%	-23.5%
+5 hr	136	5.9	37.5%	-21.2%
+6 hr	136	7.2	30.1%	-23.6%

High-water crossings in scored windows: 0 of 136 (model mean probability 4.8%); flood: 0 of 136 (1.0%).

Boxley (4 h)

Ahead	Graded	MAE (cfs)	In-band	Skill vs no-change
+1 hr	132	0.8	29.5%	-4.9%
+2 hr	131	1.2	42.7%	-11.1%
+3 hr	130	1.6	39.2%	-9.6%
+4 hr	129	2.1	47.3%	-8.9%

High-water crossings in scored windows: 0 of 132 (model mean probability 0.9%); flood: 0 of 132 (0.3%).

Neural Net Predictions (Cossatot)

The Cossatot pixel net projects the level hour by hour from 48 h of 1 km radar rainfall (6 h ahead); crossing probabilities come from its companion gradient-boosted ensemble. Graded the same way as the Buffalo net: MAE = average error of the median; IN-BAND% = how often the likely range contained the truth (target ~50%); SKILL = error reduction vs assuming the level doesn't change, on hours where it moved.

Cossatot (6 h)

Ahead	Graded	MAE (cfs)	In-band	Skill vs no-change
+1 hr	65	9.9	23.1%	6.2%
+2 hr	65	18.6	27.7%	10.8%
+3 hr	65	27.6	20.0%	9.9%
+4 hr	65	34.8	21.5%	15.4%
+5 hr	65	45.3	9.2%	13.1%
+6 hr	65	51.3	23.1%	14.4%

Floatable (240 cfs) crossings in scored windows: 6 of 58 (model mean probability 8.5%); high water (1,270 cfs): 0 of 65 (1.2%).

Coverage

What the scorecard grades today, and what is still being wired into the loop:

Predictor family	Status	Notes
Physics rise predictors	graded	Cossatot, Richland, Hailstone — predicted crest height/flow vs. the actual peak.
Empirical forecast engine	graded	6 basins — 'likelihood of rise to tier X' vs. the tier the gauge actually reached.
Recession countdowns	graded	9 gauges — maturing; the longest horizons settle ~7-10 days after they're issued.
Ponca AI Rainfall Event Analysis	graded	Each armed call graded vs Ponca's actual crest over the next 36 h (ponca_analog_eval.py).
Downstream propagation forecast	graded	Ponca -> Pruitt -> St. Joe; predictions now logged + graded on bump/magnitude/timing (propagation_eval.py).
Buffalo per-gauge rise predictions	graded	Per-gauge rise nowcasts recorded + graded vs the gauge's actual rise (buffalo_predictions_archive.py).
Routed waves (wave_router)	graded	F4 tracked waves — each confirmed wave-target's frozen crest band + arrival window graded vs what the target gauge actually did (wave_eval.py).
Buffalo flood_risk labels	not yet logged	Lower-priority companion in buffalo_output; still ungraded.
Neural Net Predictions (pixel-v5)	graded	Ponca 6 h / Boxley 4 h hour-by-hour level tables — every hourly issuance graded against the observed level at each horizon (neural_eval.py).
Neural Net Predictions (Cossatot)	graded	Cossatot 6 h level tables + GBM-ensemble crossing probs — every hourly issuance graded at each horizon (neural_cossatot_eval.py).

Gauges Watersheds Cossatot Intelligence Richland Intelligence Mulberry Intelligence Big Piney Intelligence Illinois River Intel Hailstone Intelligence Buffalo Intelligence Buffalo Study

Guide Changelog Scorecard Page Suggestions and Corrections

Home

♥ Support this project

DISCLAIMER: This site provides creek condition estimates for informational purposes only. Gauge data, radar estimates, and forecasts may be delayed, inaccurate, or unavailable. Always exercise independent judgment. Whitewater kayaking is inherently dangerous — water conditions can change rapidly. This site and its maintainers assume no responsibility for decisions made based on information displayed here.