Chalk Labs

Back to projects

A research project I started in March 2026 to see how well I can predict sports. It pulls market prices and stats from 20+ sources and turns them into outcome probabilities. A model only goes live after it beats a simple baseline on held-out data.

Python FastAPI LightGBM SQLite React Docker

Chalk Labs game page showing a projected final score of 109 to 111, the 80% range of outcomes, and how the game is likely to end — A game page with the projected final score, the 80% range of outcomes, and how the game is likely to end. The score model was tested on 2,571 NBA games.

What it does

Pulls prices from sportsbooks and prediction markets (Kalshi, Polymarket), plus box scores and injury reports, for the NFL, NBA, MLB, NHL, WNBA, college football, and 20+ soccer leagues.
Removes each bookmaker's margin to get a fair probability, then compares venues to see where they disagree.
Everything lands on a dashboard, and every prediction gets graded after the game.

How I test models

I write each test plan before I see any results, then hash it and commit it. The plan names the metric, the baseline to beat, and the bar for passing.

Tests use held-out seasons that were left out of training.
Error bars come from bootstrap resampling, clustered by game or by day.
45 tests so far, and most failed. Only models that pass feed the probabilities.

Results

Predicted vs. actual hit rate, by confidence tier 19,674 live predictions, grouped by how confident the model was and graded after the games. Every tier landed within 1.3 points of its prediction.

Top tier7,155 predictions

66.6%

Middle tier3,922 predictions

58.7%

Lowest tier8,597 predictions

56.1%

50%55%60%65%70%

Show data

Tier	Predictions	Predicted	Actual
Top tier	7,155	67.2%	66.6% (95% CI 64.7–68.4%)
Middle tier	3,922	59.4%	58.7%
Lowest tier	8,597	57.4%	56.1%

Test	Result	Verdict
Top-tier predictions, live, 7,155 predictions over 47 days	Right 66.6% of the time (95% CI 64.7–68.4%) against 67.2% predicted	Pass
NBA minutes model, 2025–26 held-out season	Average error (MAE) of 4.62 minutes vs. 4.79 for a last-10-game average	Pass
MLB Elo ratings vs. the market, 160 held-out games	Brier score (lower is better) of 0.2432 vs. the market's 0.2354	Fail
LightGBM corrections to market prices, 19 sport and market combinations	All 19 lost to the market price they were correcting	Fail

What didn't work

The first version, from March to June 2026, was an automated trading bot on Kalshi. In simulation it lost money on average across about 90,000 decisions, and it failed the kill test I had set in advance. So I shut it down and rebuilt the system around a dashboard I read myself.

The bot also hit a deadlock that stopped all work for weeks while its health checks kept reporting OK. Now the health check asks whether a full cycle finished recently, and a watchdog restarts the service when cycles stop.

Chalk Labs health page showing the engine status and a scheduler heartbeat with a 5-minute cycle and a 15-minute stall threshold — The health page watches the live price feeds and the scheduler. Pricing cycles run every 5 minutes, and a 15-minute gap marks the scheduler as stalled.

How it's built

Python 3.12 with an asyncio scheduler and a FastAPI backend
SQLite with 150+ tables
React and Vite dashboard
LightGBM, Elo, Poisson, and regression models
Docker on a Linux VPS, deployed through GitHub Actions, with offsite backups and tested restores
7,600+ automated tests and strict type checking

The code is private, but I'm happy to walk through it. ibtehaz.utsay@gmail.com