../projects
Reinforcement learning exit agent
A C++ market recorder and simulator plus a Python reinforcement learning agent that learns when to sell out of short-lived BTC prediction markets.
- Type
- Machine learning
- Date
- Built with
- Reinforcement learning, C++20, Python, SQLite
What it is
A research project on Polymarket's five-minute "will BTC go up?" markets. Instead of guessing the direction, the agent starts each window holding both sides and learns when to sell which side, and how much. It is research only: there is no live trading path.
What I built
- A recorder in C++20 that follows each rotating market on its own, stores every order book update and the Chainlink BTC price feed with exact receive times, and compresses about 28 GB a day of raw data down to about 3 GB.
- A fill simulator that replays the recorded order books with real measured network latency and fees. Checked against real trades, it reproduced 97.5% of sells exactly.
- A dataset builder that records, every second, what the agent could see (both order books, BTC momentum and volatility) and the exact result of every possible sale at that moment, with no look-ahead.
- The agent in Python: a win-probability model (ridge logistic regression, pretrained on thousands of historical markets), then fitted Q-iteration over hold / sell UP / sell DOWN, with a risk penalty in the reward. A risk formula sizes each sale so the position is pulled back towards balanced.
How it's tested
The agent learns on earlier windows and is scored only on later, unseen ones, against simple baselines like holding to the end and fixed time exits. Every baseline sees the same entries and the same latency, so the comparisons are fair.
Want to talk about this project, or something like it for your team?
Email me about it
