Architecture
ARCHITECTURE.mdKalshiTrader Architecture
The system map. For history and decision context see PLAN.md; for the regulatory posture see analysis_l.md.
Data flow
Kalshi REST ──┬─ FastDiscovery (20s, min_open_ts) ─┐
├─ MarketSync (2m open /10m unopened)├─ Markets.sync_markets ─ diff → market_events + "markets:new"
└─ MarketQuotes (per-series, on tick)┘ │
write-through marks + throttled market_snapshots
Kalshi WS ────┬─ trades ─ TradeBuffer ─ batched → trades table + "market:trade:T"
├─ orderbook ─ OrderbookTracker (books for maker fills)
└─ fills/positions → Trading (user account state)
Gemini WS ────── GeminiSocket → GeminiQuotes ETS (bookTicker + depth5, move tracking)
Polymarket WS ── PolymarketSocket → PolyQuotes ETS (CLOB books for linked markets)
Venue REST ───── VenueSync (Polymarket gamma + Gemini events) → markets (PM:/GM: tickers)
Coinbase/Pyth ── CandleSync/Backfill → candles (1h × 18 symbols, 15m liquid five;
Coinbase-first behind a circuit breaker since the Pyth API lockout)
NOAA/ESPN ────── WeatherClient / SportsData; EconCalendar (FOMC/NFP/claims/CPI)
Cross-venue: Venues.Equivalence links same-event markets (deterministic ticker parsing for crypto ladders, fuzzy titles elsewhere) → market_links
venue_spreads;Workers.ArbPaperlocks crossed books on paper with
venue-true fees, records leg quote ages/depth, and is gated by Research.ArbEvidence family verdicts (a settlement divergence = NO-GO). Workers.VenueSettlement fetches real Gemini/Polymarket resolutions.
ARM-NOW alerting (issue #205)
ArbPaper fires a CRITICAL "ARM AND EXECUTE" alert on :ops whenever all of the following are true during a paper-arb tick:
config :kalshi_trader, :arb_executor_live_enabledistrueArbExecutor.flight_check/0returns zero:failitems (:warnitems are allowed — operators know to review but they don't block)- The executor is not already armed for the current ISO week
- The spread clears the same gates as the #136 "EXECUTABLE ARB" alert: net ≥
:arb_alert_net_cents(default 4¢), depth ≥:arb_alert_min_size_cents(default $5), and link is confident - The current UTC hour is outside quiet hours (
config :kalshi_trader, :arb_arm_now_quiet_hours, {22, 6}— suppresses 22:00–05:59 UTC; set tonilto disable)
A per-link 30-minute cooldown prevents repeat spam on sticky spreads (independent of the existing EXECUTABLE ARB 30-minute suppress).
The alert is intentionally distinct in wording from "EXECUTABLE ARB" (#136) so operators can tell them apart at a glance. It includes tickers, net edge, depth, and a deep link to /arb (built from KalshiTraderWeb.Endpoint.url/0, which resolves to the full production URL via PHX_HOST, falling back to the relative /arb path if the endpoint is unavailable).
Scope: quiet hours apply only to ARM-NOW alerts. Other critical ops alerts (missed legs, auto-pause, auto-halt) are unaffected.
Strategies consume PubSub topics + periodic {:tick, now}; signals persist and broadcast; the execution engine turns them into orders.
Layers and key modules
| Layer | Modules |
|---|---|
| Ingestion | Ingestion.{FastDiscovery, SocketManager, TradeBuffer, OrderbookTracker}, Workers.{MarketSync, CandleSync, CandleBackfill, SeriesSync, SettlementSync} |
| Domain contexts | Markets (sync/diff/lifecycle), MarketData (trades/candles), Trading (signals/orders/fills/performance/audit), Weather, Exposure, Crypto (asset registry), Models, ApiTokens |
| Strategies | Strategies.Strategy behaviour + Runner + Strategies (seeding/supervision); implementations in Strategies.*; statistics in Strategies.{CandleStats, MarketQuotes}, ML.Logistic |
| Execution | Execution.{Engine, RestingOrders, PositionManager, RiskManager, FillStats}, Fees |
| Venues | Venues.{Venue, Polymarket, Gemini, Equivalence, Meta, Fees, FeeVerification}, Ingestion.{GeminiSocket, GeminiQuotes, PolymarketSocket, PolyQuotes}, Workers.{VenueSync, ArbPaper, VenueSettlement, BookSnapshot} — FeeVerification extracts actual fee fields from order payloads and compares against Venues.Fees (issue #183; see VENUES.md § "Fee model & verification") |
| Research | Backtest (PIT replay + exit sim + maker blending), Research.{ArbEvidence, SpreadStudy, VenueLeadLag, VenueCalibration, BookStudy, FillRealism, Correlation, AllocationAdvisor, MonteCarlo, Stress, EventStudy}, mix tasks kalshi.{backtest,sweep,event_study,seasonality,archive,archive.verify,token,webhook,export,fees.verify} |
| Reporting/ops | Workers.{MorningReport, WeeklyReport, ResearchReport, PilotDigest, AutoTune, SeasonalityTune, Lifecycle, Watchdog, DataQuality, FillQuality, WeatherCalibration, ModelFit, EquitySnapshot, LiveRecon, WatchlistAlerts, Backup, RestoreCheck, Prune, LiveOrderSync}, Alerts.{Notifier, Push} (Discord + web push, per-device all/critical filters), Webhooks, ErrorLog (in-memory ring + persisted errors table), Reports |
| Web | Six-group nav (Tracker · Markets · Crypto · Trading · Venues · Portfolio); LiveViews incl. /positions, /event/:ticker, /search, /reports, public /performance, /report, /developers, /terms, /disclaimer; PWA (manifest + service worker + push); ApiController (/api/signals, /api/kill, /api/health) |
The strategy contract
init(params) → {:ok, state}; subscriptions(state) → [topic]; handle_event(event, state) → {:signals, [map], state} | {:noop, state}. Events are PubSub messages plus {:tick, DateTime} every tick_ms. Signal maps: market_ticker (required), side, action, price_cents (the ask), bid_cents (enables maker posting), count (pre-Kelly), confidence (0..1 — feeds Kelly sizing and calibration), rationale.
Strategies must be point-in-time safe: inside backtests the process key :backtest_as_of is set; MarketQuotes then serves historical snapshots, CandleStats truncates history, and anything touching live feeds must check the key (see MomentumFifteen.live_spot).
Execution pipeline
signal → Kelly sizing (confidence × virtual bankroll, quarter-Kelly, opt-out via sizing: "fixed") → RiskManager.check (kill switch; order cost; per-market, per-underlying, total exposure caps; mode-specific limits with live micro-caps) → entry:
- paper maker (default): rest at
bid_cents,RestingOrdersfills off the trade feed with queue-position modeling (size-ahead from the tracked book), taker fallback atmaker_timeout_min; - paper taker (
entry: "taker"): immediate fill at ask + slippage + fee; - live: requires
live_approved; resting orders reconciled byLiveOrderSync.
Exits: PositionManager (TP/SL/settle-freeze, per-strategy exits params). Trading-hours filters (trading_hours/exclude_hours params) gate ticks in the live Runner AND in backtests. Lifecycle circuit-breaks on trailing-7d drawdown, auto-culls deep losers, auto-demotes live strategies on realized loss, and announces first-time live-gate passers (all audited). Live extras: LiveGate (readiness + one-click pilot at $0.25/order), live maker entries with timeout cancels, LiveOps.cancel_all!, nightly LiveRecon balance reconciliation, intraday drawdown + unexplained balance-move alerts.
Money conventions
All prices/PnL in integer cents; strikes in float dollars. Fees: Fees.taker_fee_cents = ⌈0.07·C·P·(1−P)⌉ in exact integer math; maker = 0. Position accounting nets buys/sells per (strategy, ticker, side); settled positions pay 100¢ × remaining contracts on a win. Invariant tests: test/kalshi_trader/money_invariants_test.exs.
Audit rules
Every strategy-config mutation goes through Trading.upsert_strategy_config(attrs, actor) and writes audit_log (before → after, actor user:<email> | system:<source>). Automated actors: system:{seed, auto_tune, seasonality, circuit_breaker, lifecycle, lifecycle_cull, live_demotion, live_gate, arb_evidence}. AutoTune applies changes only walk-forward-validated, opt-in, one grid step/night. Statistical inputs (candle stats, fitted models, measured weather sigma, maker rate) refresh continuously and are not audited — they're data, not configuration.
Worker schedule (Oban cron; watchdog auto-verifies coverage)
See the crontab in config/config.exs — it is the source of truth. Rough map: market sync 2m/10m, candles 2m, watchdog 5m, live-order sync 5m, settlement + series hourly, equity snapshots hourly, model refit + autotune nightly, morning report + lifecycle daily, weather calibration daily, fill quality + weekly report Mondays, backup nightly (+ offsite copy), restore check monthly, prune daily.
Gotchas (hard-won)
- 2026 Kalshi API: prices arrive as
<base>_dollarsstrings and counts as<base>_fp; status query paramunopened↔ object statusinitialized; markets universe ≫ the 60k sync cap (hence FastDiscovery + quote-driven snapshots). - Novelty: every ladder window is a "new event";
data.new_seriesis the news signal,new_eventis diluted (millions/week). - Timestamps in Postgres are naive UTC; raw SQL params must be cast
::timestampwith naive values, never::timestamptz. config/*.exschanges (including git branch switches touching them) wedge the dev code reloader — restart the server.- Research CLI tasks set
KALSHI_MINIMAL=1(Repo+PubSub only boot); never run a full second node against the same DB. - Kill processes by the port-4001 owner pid, never by image name (
erl.exematching also catches unrelated processes). - Backups: nightly dump + OneDrive offsite copy + monthly restore proof; the local archive warehouse (
mix kalshi.archive, nightly Task Scheduler job) is the cloud-loss recovery path —kalshi.archive.verifyalerts if it lags (a restore drill caught it silently stale once; seedocs/RESTORE_DRILL.md). - Work tracking: GitHub issues labeled
roadmap, closed with evidence comments (commit, tests, measured results) +evidence-closedlabel.
Settlement and oracle divergence coverage
There is no OracleMonitor module in this codebase — the claim in an earlier planning digest (managed\_next20 item) that an orphan stub existed was incorrect. No such module was ever committed or removed; there is nothing to replace or delete.
Proactive settlement/oracle risk is covered by three layers:
| Layer | Module(s) | What it catches |
|---|---|---|
| Rules drift (pre-settlement) | Research.TermsSentinel (cron daily 06:30) | Venue edits settlement-terms text after a link is graded — source, measurement time, inclusivity, early-settlement clause changes. CRITICAL alert on confident (traded) links. |
| Post-settlement trueness | Workers.DataQuality → Research.ArbTruth + Research.DivergenceForensics | Arb settlement trueness audit; divergence forensics on resolved markets. |
| Resolution sync | Workers.VenueSettlement + Workers.SettlementSync | Fetches real Gemini/Polymarket resolutions; blocks execution on divergence verdicts. |
If a future live oracle-feed monitor is wanted (e.g. real-time oracle price vs expected settlement price during the settlement window), that would be a brand-new feature added as its own worker/module — not a replacement for any stub.
Live arb analysis pack (#207)
Research.LiveArbAnalysisPack and mix kalshi.arb.analysis_pack are the #207 evidence path — a read-only post-trade analysis tool that fires on the first real two-leg executions. The pack is NOT a trading surface; it does NOT arm, enable live orders, or place any orders. Calling it or the mix task can never cause capital to move.
When arb_executions has no mode = "live" rows the pack returns a labelled template stub (:no_live_trades). When rows exist the pack covers:
- Realized vs quoted edge per execution
- Fees actual vs
Venues.Feesmodel (noting if FeeVerification #183 is present on the branch) - Latency at each hop (from
audit_logentityarb_executor) - Slip (far fill vs quote; Kalshi fill assumed at quote per executor design)
- GeminiRecon reconciliation status at report time
- Status breakdown (filled_both / stub / unfilled / failed)
Do not confuse this module with any live-enabling path. The arm/1 flow, live_gates, and arb_executor_live_enabled config are unrelated to this report.