Nova

Low-latency C++ quantitative market making infrastructure. Mirrors portfolio allocation predictions ("Pawn state") from Valkey into a lock-free in-memory store, providing the foundation for automated trading.

Phase 1 — Pawn State Module COMPLETE
Active Task
Data API Header
27
Tests Passing
9
Source Files
2
ADRs Recorded
7
CE Learnings
Tech Stack
Language: C++20State Store: Valkey (Redis-compatible, AWS ElastiCache cluster)Build System: CMakeConcurrency: Wait-free reads via `std::atomic` + ring buffer (see ADR-001)Valkey Client: hiredisTesting: GoogleTest
Performance
250ns
Snapshot Read p50
49us
Snapshot Read p99
1.2M/s
Read Throughput (8 threads)
215K/s
Write Throughput
File Tree
src/core/
config.h .h
Build key patterns from config
logger.h .h
Optional custom sink. If not set, logs to stderr.
src/pawn_state/
pawn_state.h .h
Key: "timeframe:base:quote" e.g. "1min:XAG:GBP"
state_store.cpp .cpp
Initialize with an empty map in slot 0
state_store.h .h
Lock-free state store using atomic pointer swap + deferred deletion.
valkey_connection.cpp .cpp
Values are JSON strings, use GET not HGETALL
valkey_connection.h .h
Scan all state keys matching the configured pattern using SCAN.
valkey_sync.cpp .cpp
Check if connection died mid-fetch
valkey_sync.h .h
Start the sync loop in a background thread.
PRDs
#TitleStatus
PRD-001Pawn Trading EngineActive
Epics
#TitlePRDStatus
EPIC-001Pawn State ModulePRD-001Done
EPIC-001Concurrency Model (wait-free atomic pointer + ring buffer)PRD-2026-03-16Accepted (revised)
EPIC-002Production Data Format Alignment (JSON strings, multi-timeframe)PRD-2026-03-15Accepted
Tests
+
14 unit tests — PawnState, StateMap, StateStore
+
9 integration tests — Valkey connection, SCAN, sync, dynamic detection
+
4 concurrency tests — torn read detection, snapshot immutability, stress
Architecture Decisions
ADR-001 Accepted (revised)

ADR-001: Concurrency Model for State Store

2026-03-15 (updated 2026-03-16)

Replaced with a truly lock-free design: ```cpp std::atomic current_; std::array, 16> history_; ``` - **Reader:** `current_.load(std::memory_order_acquire)` — a single atomic pointer load. No lock, no CAS, no retry loop. This is **wait-free** (the st

ADR-002 Accepted

ADR-002: Production Data Format Alignment

2026-03-15

Updated all code to match production format: - `GET` instead of `HGETALL` for state values - Minimal JSON parser (no library dependency) for the fixed 6-field format - Local map key: `timeframe:base:quote` (e.g., `1min:XAG:GBP`) - Config exposes `state_key()`, `pairs_key()`, `last_ts_key()` builders

Compound Engineering Learnings

2026-03-15 - atomic not available on Apple Clang

Context: Building lock-free state store for Pawn State Module

Learning: `std::atomic>` (C++20 P0718R2) is not implemented in Apple Clang 17 / libc++. It compiles on GCC 12+ with libstdc++ but fails with "requires trivially copyable type" on macOS.

Pattern: Use `std::atomic` + a ring buffer of `unique_ptr` for deferred deletion. Reader does a single wait-free atomic load. Writer publishes via `atomic::store(release)` and keeps old maps alive in the ring buffer. No locks, no shared_ptr, no CAS.

2026-03-15 - Concurrency test invariant must hold for ALL entries

Context: Writing torn-read detection test for StateStore

Learning: When encoding a consistency invariant (e.g., `base + quote == 1000000`) across map entries in a concurrency test, the invariant must hold for EVERY entry in the map, not just the first one. Initially wrote `BTCUSD: {base * 0.001, quote}` where quote was `1000000 - base` — this broke the invariant for BTCUSD.

Pattern: If using a sum-invariant for torn read detection, compute both fields from the same source value for each entry independently.

2026-03-15 - Validate production data format before building

Context: Built Pawn State Module against PRD assumptions, then discovered production Valkey uses JSON strings (not hashes), different field names, negative positions, and multi-timeframe keys.

Learning: Always ask for a sample of real production data (SCAN + GET/TYPE) before implementing the data layer. PRD assumptions about storage format were wrong on 4 counts: value type, field names, sign semantics, and key structure.

Pattern: Before implementing any Valkey/Redis data access, run `TYPE key`, `GET`/`HGETALL` on a real key, and `SCAN` to see actual key patterns. Build the parser against real sample data.

2026-03-16 - Full rebuild each sync cycle prevents stale entries

Context: Reviewing whether delisted symbols persist in the state map

Learning: If a symbol is removed from Valkey (brain stops publishing, delisted pair), a naive incremental update would retain the stale entry forever. Building a fresh map from SCAN results each cycle and atomically swapping it guarantees that only currently-published keys exist in the local state.

Pattern: For state mirrors, prefer full rebuild + atomic swap over incremental merge. It's naturally self-cleaning — stale keys disappear when they stop appearing in SCAN. The cost is O(N) per cycle, but for <1000 keys at 100ms intervals this is negligible.

2026-03-16 - Macro parameter names must not collide with method names

Context: Building logger macros for Nova trading engine

Learning: If a macro parameter is named `level` and the macro body calls `_l.level()`, the compiler substitutes the parameter into the method call, producing `_l.::nova::LogLevel::Error()`. Use unique parameter names like `lvl_` in macros to avoid collisions with member function names.

Pattern: Always suffix macro parameter names with `_` (e.g., `lvl_`, `mod_`) to prevent collisions with any method or variable in the expansion context.

2026-03-16 - SCAN must use full cursor loop

Context: Reviewing Valkey key scanning in Pawn State Module

Learning: A single `SCAN 0 MATCH pattern COUNT 100` only returns a partial result. Redis/Valkey SCAN is cursor-based — you must loop `do { SCAN cursor } while (cursor != 0)` to guarantee all keys are visited. COUNT is a hint, not a limit.

Pattern: Always implement SCAN as a do-while loop on the cursor. Never assume a single call returns all keys.

2026-03-16 - shared_mutex is not lock-free, atomic pointer load is

Context: Replacing shared_mutex-based state store with truly lock-free reads

Learning: `std::shared_mutex` with `shared_lock` is contention-minimal but NOT lock-free — a reader can block while waiting for a writer's exclusive lock. True lock-free (actually wait-free) reads require `std::atomic::load(acquire)` — a single atomic pointer load with zero synchronization.

Pattern: Use `atomic` + a ring buffer of `unique_ptr` for deferred deletion. Reader does one atomic load. Writer stores new map in the ring buffer, then atomically publishes the pointer. Ring buffer size * sync interval = safe reader window.

Phase 1 Requirements
+
FR-1: Connect to Valkey
+
FR-2: Continuously synchronize Pawn keys
+
FR-3: Detect new symbols dynamically
+
FR-4: Maintain local in-memory state
+
FR-5: Instant read access to other modules
+
FR-6: Never block readers (lock-free reads)
+
FR-7: No network calls in read path
+
FR-8: Multi-thread safe
+
FR-9: Production-ready and tested
+
Structured logging with observability