rejecthon

You only know what happened to the people you said yes to.
Reject inference is the art of learning from the ones you turned away.

Rejecthon is a hackathon toolkit for one practical credit-scoring question: can you beat an observed-only baseline when outcomes are missing for the applicants who were turned away?

Install

git clone https://github.com/vathymut/rejecthon.git
cd rejecthon
uv sync

Get on the board in three lines

from rejecthon import submit, show

run = submit("first-run")
show()  # color-coded leaderboard

The first call prepares the dataset, seeds the baselines, runs your model on the evaluation set, and stores the result. show() prints where you stand immediately.

Bring your own model

from sklearn.ensemble import ExtraTreesClassifier
from rejecthon import submit, compare, show

run = submit(
    "extra-trees",
    scoring_model=ExtraTreesClassifier(n_estimators=300, random_state=42),
    notes="trying a different algorithm",
)
compare("extra-trees", "hgb_observed")  # did it actually improve?
show()

Use scoring_model= to swap the final classifier. Use inference_model= to change how default probabilities are assigned to rejected applicants.

Classifiers must implement predict_proba(). Wrap any that don't with CalibratedClassifierCV.

Submit for your team

Pass team_name to export a run to S3 as a Submission. Copy .example.env to a git-ignored .env, fill in your AWS credentials, and submit:

run = submit("extra-trees", team_name="Team Alpha")
run["submission_id"]  # "team-alpha-<run_id>"

The export is non-blocking — missing credentials or a failed upload only warn, and your run still stands locally. See Submit a Model.

Documentation

• Quick Reference — all 8 functions on one page
• Getting Started — 30-minute walkthrough to a result you can explain
• The Three Populations — understand the data before you model
• For Business Stakeholders — plain-language explainer, no code required
• Python API