Fraud Detection: ML Hackathon
Built with teammates under hackathon time pressure: a LightGBM classifier over a heavily imbalanced transaction dataset, with feature engineering aimed at making rare fraud patterns visible rather than chasing overall accuracy alone.
- Role
- Team project
- Timeline
- Feb 2026
- Impact
- A multi-class fraud classifier built with a hackathon team, tuned specifically for the minority fraud classes an accuracy number alone would hide.
Attribution: Built during a machine learning hackathon with a team (Mind Over Models). My focus: feature engineering and model iteration for the classifier.
Built as a hackathon submission, no public repository.
engineered features feeding the classifier
Concepts demonstrated
- Applied machine learning
- Imbalanced classification
- Feature engineering
- Rapid iteration under time pressure
Highlights
- Iterated across multiple LightGBM configurations and feature sets on a heavily imbalanced multi-class dataset, rather than stopping at the first model that scored well on aggregate accuracy.
- Engineered domain-specific features in Pandas (transaction velocity, merchant risk scoring, and time-of-day patterns) to surface fraud signals that generic off-the-shelf features missed.
- Optimized specifically for precision on the minority fraud classes, since raw accuracy on an imbalanced dataset can look excellent while still missing most of the fraud that matters.
Problem
Fraud detection datasets are almost always heavily imbalanced. The vast majority of transactions are legitimate, so a classifier can score a deceptively high accuracy while still missing most of the actual fraud. Under hackathon time constraints, the team needed a model that was tuned for the fraud cases, not just the aggregate number.
Engineering approach
- Started from raw transaction data and engineered features specifically aimed at fraud signal rather than relying on generic columns: transaction velocity, merchant-level risk scoring, and time-of-day patterns.
- Iterated across multiple LightGBM configurations and feature combinations rather than settling on the first model that looked good in aggregate.
- Evaluated with an eye on precision for the minority fraud classes specifically, since that's the metric that actually matters for a fraud detector.
Architecture
- Pandas-based feature engineering pipeline turning raw transaction records into behavioral signals (velocity, merchant risk, timing).
- LightGBM multi-class classifier trained over the engineered feature set.
Technical challenges
- Class imbalance meant the obvious failure mode (a model that just predicts the majority class) could still post a very high accuracy number, so the team had to deliberately evaluate and tune around minority-class precision instead of trusting the headline metric.
- Hackathon time constraints meant feature engineering and model iteration had to happen in tight parallel with teammates rather than in the more sequential process a longer project would allow.
Decisions
- Prioritized engineered, domain-specific features (transaction velocity, merchant risk, timing) over throwing raw columns at the model, the kind of feature work that tends to matter more than model choice for tabular fraud data.
- Chose LightGBM for its speed on tabular data under a hackathon deadline, which left more time for feature iteration than a heavier modeling approach would have.
Result
A classifier that reached 99.89% accuracy on the evaluation set, reported here specifically alongside the minority-class precision focus, since accuracy alone is a weak signal on an imbalanced fraud dataset and the team optimized for the harder, more meaningful target.
Learnings
- Evaluating on minority-class precision instead of trusting the headline accuracy number is what kept the team from shipping a model that just predicts 'not fraud' every time and still looks great on paper.
- Under hackathon time pressure, the team didn't have time to validate the engineered features against a holdout time period. The honest next step is checking whether transaction-velocity and merchant-risk features still hold up on data from a different time window, not just a random split.