Hackathon judging you can defend.
A self-hosted submission and judging portal that enforces its own rules on the server: deadlines that hold, judges who cannot see each other's scores, a weighted rubric that locks once judging starts, and normalization you can read, reproduce and question.
Take the 5-minute tour
One full event, real actions, synthetic data, about five minutes.
Runs in a private sandbox for this browser session.- Starts with
docker compose upand runs with the network off. - Every write in the interface goes through the same documented REST API.
- Permission checks live in the backend and are tested role by role.
- T1 gallery is public PASS
- T1 project from fixtures shown PASS
- T1 closed event refuses submissions PASS
- T2 judge sees own scores PASS
- T2 judge cannot see peer scores PASS
- T2 participant blocked PASS
- T2 csv export works PASS
run.py has no T3/T4 checks: they are verified by hand with
scripts/verify_tiers.py and docs/TIER-EVIDENCE.md.Every click, live: one event from creation to a verified result.
Recorded on this build: real clicks, real API calls and real terminal runs, with the organizers' fixture and a synthetic calibration event. The narration is a synthetic voice.
The gaps organizers keep hitting, and what we do instead.
The left column summarises gaps named in the DOGFOOD brief about today's hackathon platforms. The right column points to where each answer lives in this codebase.
| Common gap | What VERDICT does | Where to check |
|---|---|---|
| Judging criteria cannot be weighted, or one fixed rubric is shared by every event. | Each event has its own weighted rubric. It locks when the first review is submitted, so weights cannot be tuned after seeing scores. | PUT /events/{slug}/rubric returns 409 scoring_locked after the first review |
| Score normalization is advertised but never documented. | An additive judge-offset model with shrinkage, whose strength is chosen by cross-validation, published with its evidence and limitations. | docs/NORMALIZATION-PROOF.md, regenerated by one script |
| Role checks live in the interface, so a URL typed by hand leaks data. | Every read and write passes a policy layer on the server. Judges see only their own reviews and in-track assignments. | The organizers' checker, plus per-role isolation tests |
| Deadlines are advisory, or enforced only in the form. | Windows are checked on server time after the row is locked, for every write path including images and answers. | Boundary tests at the exact close second |
| No public API, so integrations rely on scrapers and CSV downloads. | The interface itself writes only through /api/v1, described by an OpenAPI schema. | /api/docs and /api/schema |
Try to break it with curl.
These responses were captured from this build against the seeded fixture event, using the demo tokens printed when the portal starts.
Submitting after the deadline
403# as a participant, fixture event closed 2026-03-01 18:00 UTC POST /api/v1/events/sample-hack-2026/projects {"error": {"code": "window_closed", "message": "Submissions for Sample Hack 2026 closed at 2026-03-01T18:00:00Z."}}
The deadline is checked before any validation, so the refusal always says why.
A judge reading another judge's scores
403# as judge B, asking for judge A (jdg_24) GET /api/v1/events/sample-hack-2026/judges/jdg_24/scores {"error": {"code": "forbidden", "message": "Only this judge and the organizers may read these scores."}}
The same request with ?judge=jdg_24 on the own-scores route is refused too.
A judge reading their own scores
200# as judge A GET /api/v1/judge/scores {"judge": {"public_id": "jdg_24", "event": "sample-hack-2026", ...}, "reviews": [ ... 11 reviews, all written by jdg_24 ... ]}
Scoped by the server to the caller; there is no parameter that widens it.
No credentials at all
401# anonymous
GET /api/v1/judge/scores
HTTP/1.1 401 Unauthorized
WWW-Authenticate: Bearer realm="api"
A real 401 with a challenge header, not a redirect to a login page.
One process, one database, one rule book.
A modular Django monolith. Views and API endpoints are thin; every permission and deadline rule sits in a policy and service layer that both share, and every change is written to the audit log in the same transaction.
Browser
Server-rendered pages; forms post JSON to the API.
Scripts and integrations
Bearer tokens created per user, shown once, stored hashed.
Organizers' checker
run.py with the four demo headers.
Auth
Session with CSRF, or bearer token. Login throttling.
Policy and services
Role per event, ownership, track, deadline, locks. The only place rules live.
Judging engine
Pure Python, deterministic: scoring, normalization, Bradley–Terry.
Audit log
Readable sentence per change, in the same transaction.
PostgreSQL 16
Constraints enforce one role per person per event and one active project per team.
Local volume
Uploaded images behind the same visibility policy, plus the generated secret key.
What the normalization does, measured honestly.
From docs/NORMALIZATION-PROOF.md, generated from the organizers' fixtures.json by scripts/normalization_proof.py. Rankings themselves are only shown to organizers until they are published.
Strength chosen by cross-validation
Seeded 5-fold CV picks strong shrinkage on this fixture: error 19.52 against 20.83 for a fixed λ = 2.
predeclared procedure · ties go to the larger λNo detectable judge bias here
Shuffling judge labels within tracks 2,000 times produces offset spreads as large as the real ones.
permutation test · seed 97531Better when judges are biased
In simulations on the fixture's own review design with moderate judge bias, rank correlation with the truth beats raw averaging.
σb = 0.6 · adaptive rule, 20 replicationsThe honest reading: this fixture's judge offsets cannot be told apart from noise, so the method deliberately changes little here, while it corrects strongly when bias is real. That is what a predeclared, evidence-driven rule should do, and why we publish the numbers instead of a promise.
Handled on import, explained to the organizer.
A duplicate submission
"Dry Harbour" arrives twice from the same team. The later entry supersedes the earlier one; the earlier one's reviews are kept for the record and left out of results.
A judge who scores everything the same
One judge gives 4 on every criterion of every project. They are detected and flagged, and the proof shows what excluding them would change.
Review batches nobody finished
Eight projects have two reviews where the rest have three. They are flagged as under-reviewed, and the assignment tool can fill the gaps within each judge's tracks.
Your event, your server, your data.
No cloud account, hosted database or auth provider. Clone the repository, start it, and the fixture event and demo accounts are ready.