Building Sadeed at the GASTAT Data Innovation Hackathon
How our three-layer ETL validator reached the top 16 of 132 teams, and what I would change.
Top 16 of 132 teams at the GASTAT Data Innovation Hackathon in partnership with SDAIA. Here's what my team and I built, and what I learned leading us through it.
What we built
Sadeed is a 3-layer ETL validator: Great Expectations for schema checks, a One-Class SVM for numerical anomaly detection, and a locally deployed LLM for semantic analysis. GE and OC-SVM ran in parallel. The LLM ran downstream, called only when OC-SVM flagged a row.
We believed that data sovereignty should not diminish results, which is why the LLM lived locally instead of in the cloud.
What the numbers showed
OC-SVM alone scored an F1 of 0.762 with our validation datasets, which was respectable given how small the training dataset was. Stacking all three layers pushed it to 0.804.
What I would change
The most interesting part wasn't in the numbers. It was in some of the edge cases. A 19-year-old with a master's degree, for example, is a logical contradiction the LLM would catch instantly. But our OC-SVM had to flag the row first, and the use case turned out to be too nuanced for it to catch with a dataset this small. The semantic check was the right tool in the wrong position.
Given another run at it, we'd run two LLMs: a lightweight one in parallel with OC-SVM to catch contradictions upfront, and the heavier local LLM as a second-pass validator to reduce false positives.
The team
- Abdulaziz Aldharrab
- Malik AlNajjar
- Shahad Khuzayyim
- Mayasah Al-Harbi
- Salman Rajab