Dataset timetravel_insurance

Feature Scatter

Synthetic two-dimensional classification dataset with a linear boundary and 10% label noise.

Provenance

uv run ./random_classification_data_generator.py --number-of-data-points 200 --feature1-name "TimelineDeviation" --feature1-mean 12 --feature1-stddev 3  --feature2-name "ParadoxCount"  --feature2-mean 5  --feature2-stddev 2  --target-column-name "PolicyClaim"  --primary-key-name "IncidentID"  --table-name "time_travel_incidents"  --splits-table-name "time_travel_splits"  --random-seed 42  --class-deciding-expression "TimelineDeviation + 2 * ParadoxCount > 20"  --false-class-name "Denied"  --true-class-name "Approved" --noise 0.10  --holdout 0.2  --validation 0.5   --output-db-file $(TEMPLATES_DIR)/$(TIMETRAVEL_INSURANCE_DATASET).sqlite  --config-file configs/$(TIMETRAVEL_INSURANCE_DATASET).config.json --use-uuid

Best Validation Round Rank

Test scores by release date

Regression slope: 0.0002, intercept: -3.6697, p-value: 0.05732

Debug

merge info: 31 rows
filter ollama: 30 rows
drop NA release/accuracy: 30 rows
compute KT and sort
added baselines
create figure
scatter models: [(datetime.date(2024, 1, 25), -0.25983377939935925), (datetime.date(2024, 7, 18), -0.09772102067959801), (datetime.date(2024, 10, 22), -0.1256595574906267), (datetime.date(2024, 10, 22), -0.07147167409197236), (datetime.date(2024, 11, 20), -0.1256595574906267), (datetime.date(2024, 11, 20), -0.25983377939935925), (datetime.date(2024, 12, 5), -0.18758554574618913), (datetime.date(2024, 12, 5), -0.09772102067959801), (datetime.date(2025, 1, 30), -0.09772102067959801), (datetime.date(2025, 1, 30), -0.4550069404290902), (datetime.date(2025, 2, 5), -0.09772102067959801), (datetime.date(2025, 2, 19), -0.1555197331793902), (datetime.date(2025, 2, 19), -0.18758554574618913), (datetime.date(2025, 3, 25), -0.1256595574906267), (datetime.date(2025, 4, 14), -0.1555197331793902), (datetime.date(2025, 4, 14), -0.1256595574906267), (datetime.date(2025, 4, 16), -0.1555197331793902), (datetime.date(2025, 4, 16), -0.09772102067959801), (datetime.date(2025, 5, 14), -0.1555197331793902), (datetime.date(2025, 5, 14), -0.1555197331793902), (datetime.date(2025, 5, 14), -0.1256595574906267), (datetime.date(2025, 5, 14), -0.1555197331793902), (datetime.date(2025, 5, 14), -0.09772102067959801), (datetime.date(2025, 5, 14), -0.25983377939935925), (datetime.date(2025, 5, 14), -0.1555197331793902), (datetime.date(2025, 5, 14), -0.18758554574618913), (datetime.date(2025, 5, 14), -0.09772102067959801), (datetime.date(2025, 5, 14), -0.09772102067959801), (datetime.date(2025, 5, 14), -0.09772102067959801), (datetime.date(2025, 5, 14), -2.60422605308447)]
xlim_min 2024-01-01
scatter ensembles: 7
draw ensemble trend line
xlim_max 2025-05-14
axhline corels at -0.207
annotate corels at edge 20222.0
savefig website/dataset/timetravel_insurance/release_scores_timetravel_insurance.png

Model Scores

Train vs Validation

Regression slope: 0.9777, intercept: 0.0670, p-value: 1.2775e-14

Validation vs Train

Regression slope: 0.9040, intercept: 0.0227, p-value: 1.2775e-14

Ensemble Max Scores

Baseline

Baseline Models

Logistic Regression

take the paradoxcount and multiply by -0.559, take the timelinedeviation and multiply by -0.359, add those up and if the number is greater than 6.105 then predict Approved, otherwise predict Denied.

Decision Tree

Dummy

Always predict 0

RuleFit

Bayesian Rule List

CORELS

1. {'col': 'X0', 'index_col': 0, 'cutoff': 0.5, 'val': 0.21875, 'flip': False, 'val_right': 0.625, 'num_pts': 160, 'num_pts_right': 32}
2. {'col': 'X5', 'index_col': 5, 'cutoff': 0.5, 'val': 0.14285714285714285, 'flip': False, 'val_right': 0.5652173913043478, 'num_pts': 128, 'num_pts_right': 23}
3. {'col': 'X7', 'index_col': 7, 'cutoff': 0.5, 'val': 0.0, 'flip': True, 'val_right': 0.1875, 'num_pts': 105, 'num_pts_right': 80}
4. {'val': 0.0, 'num_pts': 25}

EBM