AI engineering · 2026
IDX Stock Scanner
A daily decision-support agent for Indonesian equities: rules filter, an XGBoost model ranks, and an LLM explains. It now runs itself — scheduled scans, signal tracking against the index, and a gated challenger-model workflow.
- layers — rules → ML → LLM
- 3
- GitHub Actions workflows
- 8
- test functions
- 470+
Architecture
Select a stage to explore it.
01 · Data fetcher
Daily OHLCV for IDX-listed stocks behind an abstract fetcher contract, with incremental updates persisted to Parquet — providers can change without touching the pipeline.
Visuals

01 · Problem
Screening hundreds of IDX tickers daily by hand doesn't scale, but neither does trusting a single opaque model with a ranked list. The design problem is trust: every recommendation needs to be filterable, rankable, and explainable.
02 · Solution
A three-layer pipeline where each layer does what it's structurally best at: deterministic rules provide guardrails, gradient-boosted ranking provides prioritization, and an LLM provides per-ticker narrative explanation. No layer is asked to do another's job.
03 · Engineering decisions
Rules run before ML deliberately — guardrails should be auditable, and a rejected stock should have a stateable reason. The LLM sits last and only explains; it never picks. And the whole system is framed as decision support with that caveat written into the README, because overstating a financial tool's promise is the fastest way to make it harmful.
04 · What changed since launch
The scanner moved from a script I ran to a system that runs on a schedule. GitHub Actions now run the pre-market scan, a post-close movers job, and an evening job that records how each signal actually performed, benchmarked against the IHSG index in the dashboard.
Model updates follow a challenger workflow: a new model is trained from the signal database, compared against explicit promotion criteria, and only promoted — with a rollback path — when it passes. Drift monitoring raises alerts rather than retraining silently. An audit pass removed a silent mock-data fallback in the broker-data integration so missing data now fails visibly.
05 · Results
A daily scanner producing ranked, explained candidates from IDX market data, refreshed automatically every trading day — the most heavily iterated project in the portfolio. The model's ranking power is modest, which is exactly why the rules and the performance tracking around it matter.
06 · Lessons learned
Hybrid systems beat monoliths when each layer's failure mode is different: rules fail loudly, models fail statistically, LLMs fail plausibly. Keeping them separate keeps each failure diagnosable.