plan synthetic and smart-flow phases

This commit is contained in:
dirtydishes 2026-06-16 13:46:08 -04:00
parent d1fac6c7ec
commit eaa22de302
19 changed files with 1198 additions and 1 deletions

View file

@ -1,3 +1,13 @@
{"_type":"issue","id":"islandflow-zxh.4","title":"Smart-flow phase 04: replay evaluation and golden tests","description":"Make replay evaluation and golden tests the acceptance gate for smart-flow changes as described in docs/implementation/smart-money/04-replay-evaluation-golden-tests.md. The phase validates derived evidence/hypothesis outputs from deterministic raw streams.","acceptance_criteria":"Replay recomputes derived outputs from raw fixtures; golden signatures cover positive, abstain, false-positive, and noisy scenarios; tests are deterministic and infra-free by default; optional service-container tests are clearly gated.","spec_id":"docs/implementation/smart-money/04-replay-evaluation-golden-tests.md","status":"open","priority":1,"issue_type":"feature","owner":"dishes@dpdrm.com","created_at":"2026-06-16T17:38:44Z","created_by":"dirtydishes","updated_at":"2026-06-16T17:38:44Z","labels":["phase","planning","smart-flow","smart-money"],"dependencies":[{"issue_id":"islandflow-zxh.4","depends_on_id":"islandflow-259.4","type":"blocks","created_at":"2026-06-16T13:39:09Z","created_by":"dirtydishes","metadata":"{}"},{"issue_id":"islandflow-zxh.4","depends_on_id":"islandflow-zxh","type":"parent-child","created_at":"2026-06-16T13:38:44Z","created_by":"dirtydishes","metadata":"{}"}],"dependency_count":1,"dependent_count":1,"comment_count":0}
{"_type":"issue","id":"islandflow-259.4","title":"Synthetic market-data phase 04: replay integration","description":"Integrate synthetic runs into replay as described in docs/implementation/synthetic-market-data/04-replay-integration.md. Replay should consume generated fixtures or materialized rows with stable ordering and source/run selectors.","acceptance_criteria":"Replay can select synthetic source/run IDs; ordering uses event time, ingest time, sequence, and stable event ID; derived output signatures can be compared to manifests; infra-backed replay checks are optional and not required for the fast Bun test path.","spec_id":"docs/implementation/synthetic-market-data/04-replay-integration.md","status":"open","priority":1,"issue_type":"feature","owner":"dishes@dpdrm.com","created_at":"2026-06-16T17:38:43Z","created_by":"dirtydishes","updated_at":"2026-06-16T17:38:43Z","labels":["phase","planning","synthetic-market-data"],"dependencies":[{"issue_id":"islandflow-259.4","depends_on_id":"islandflow-259","type":"parent-child","created_at":"2026-06-16T13:38:42Z","created_by":"dirtydishes","metadata":"{}"},{"issue_id":"islandflow-259.4","depends_on_id":"islandflow-zxh.3","type":"blocks","created_at":"2026-06-16T13:39:08Z","created_by":"dirtydishes","metadata":"{}"}],"dependency_count":1,"dependent_count":1,"comment_count":0}
{"_type":"issue","id":"islandflow-zxh.3","title":"Smart-flow phase 03: hypothesis scoring and abstention","description":"Implement hypothesis scoring, alternative explanations, confidence, and abstention behavior as described in docs/implementation/smart-money/03-hypothesis-scoring-abstention.md. The phase turns evidence clusters into cautious hypotheses rather than certainty claims.","acceptance_criteria":"Scores separate evidence strength, confidence, conviction, and penalties; abstention is a first-class output with reasons; alternatives and negative evidence are preserved; compatibility projections do not become the canonical domain model.","spec_id":"docs/implementation/smart-money/03-hypothesis-scoring-abstention.md","status":"open","priority":1,"issue_type":"feature","owner":"dishes@dpdrm.com","created_at":"2026-06-16T17:38:41Z","created_by":"dirtydishes","updated_at":"2026-06-16T17:38:41Z","labels":["phase","planning","smart-flow","smart-money"],"dependencies":[{"issue_id":"islandflow-zxh.3","depends_on_id":"islandflow-259.3","type":"blocks","created_at":"2026-06-16T13:39:07Z","created_by":"dirtydishes","metadata":"{}"},{"issue_id":"islandflow-zxh.3","depends_on_id":"islandflow-zxh","type":"parent-child","created_at":"2026-06-16T13:38:41Z","created_by":"dirtydishes","metadata":"{}"}],"dependency_count":1,"dependent_count":1,"comment_count":0}
{"_type":"issue","id":"islandflow-259.3","title":"Synthetic market-data phase 03: scenarios, labels, and expected outputs","description":"Implement deterministic scenario authoring, separate ground-truth labels, and expected-output manifests as described in docs/implementation/synthetic-market-data/03-scenarios-labels-expected-outputs.md. This phase is intentionally split into smaller child issues for reviewable PRs.","acceptance_criteria":"Scenario catalog covers representative institutional, retail-attention, event-noise, volatility, hedge, and negative/no-alert conditions; labels remain separate from emitted events; expected outputs include required/forbidden evidence and false-positive penalties; generated outputs are deterministic and reviewable.","spec_id":"docs/implementation/synthetic-market-data/03-scenarios-labels-expected-outputs.md","status":"open","priority":1,"issue_type":"feature","owner":"dishes@dpdrm.com","created_at":"2026-06-16T17:38:40Z","created_by":"dirtydishes","updated_at":"2026-06-16T17:38:40Z","labels":["phase","planning","synthetic-market-data"],"dependencies":[{"issue_id":"islandflow-259.3","depends_on_id":"islandflow-259","type":"parent-child","created_at":"2026-06-16T13:38:40Z","created_by":"dirtydishes","metadata":"{}"},{"issue_id":"islandflow-259.3","depends_on_id":"islandflow-zxh.2","type":"blocks","created_at":"2026-06-16T13:39:06Z","created_by":"dirtydishes","metadata":"{}"}],"dependency_count":1,"dependent_count":2,"comment_count":0}
{"_type":"issue","id":"islandflow-zxh.2","title":"Smart-flow phase 02: evidence clustering and features","description":"Implement evidence extraction, eligibility, clustering, and feature construction as described in docs/implementation/smart-money/02-evidence-clustering-features.md. The phase moves toward evidence clusters without overconfident participant claims.","acceptance_criteria":"Eligibility decisions, quote joins, evidence quality, clustering keys, and feature values are represented explicitly; ingest remains normalization-first; features preserve traceable evidence refs; stale/wide/noisy inputs can be rejected or down-weighted with reasons.","spec_id":"docs/implementation/smart-money/02-evidence-clustering-features.md","status":"open","priority":1,"issue_type":"feature","owner":"dishes@dpdrm.com","created_at":"2026-06-16T17:38:39Z","created_by":"dirtydishes","updated_at":"2026-06-16T17:38:39Z","labels":["phase","planning","smart-flow","smart-money"],"dependencies":[{"issue_id":"islandflow-zxh.2","depends_on_id":"islandflow-259.2","type":"blocks","created_at":"2026-06-16T13:39:05Z","created_by":"dirtydishes","metadata":"{}"},{"issue_id":"islandflow-zxh.2","depends_on_id":"islandflow-zxh","type":"parent-child","created_at":"2026-06-16T13:38:38Z","created_by":"dirtydishes","metadata":"{}"}],"dependency_count":1,"dependent_count":2,"comment_count":0}
{"_type":"issue","id":"islandflow-259.2","title":"Synthetic market-data phase 02: manifests, fixtures, and CLI","description":"Implement manifest, fixture, and CLI support for deterministic synthetic runs as described in docs/implementation/synthetic-market-data/02-manifests-fixtures-cli.md. The phase turns the deterministic engine into reusable test/demo artifacts.","acceptance_criteria":"CLI can generate fixtures and expected-output-ready manifests from seed bundles and profiles; manifests pin generator version, seed, parameter hash, event hashes, and replay ordering; fixture helpers support infra-free Bun tests; generated artifacts avoid embedding hidden labels in market events.","spec_id":"docs/implementation/synthetic-market-data/02-manifests-fixtures-cli.md","status":"open","priority":1,"issue_type":"feature","owner":"dishes@dpdrm.com","created_at":"2026-06-16T17:38:37Z","created_by":"dirtydishes","updated_at":"2026-06-16T17:38:37Z","labels":["phase","planning","synthetic-market-data"],"dependencies":[{"issue_id":"islandflow-259.2","depends_on_id":"islandflow-259","type":"parent-child","created_at":"2026-06-16T13:38:37Z","created_by":"dirtydishes","metadata":"{}"},{"issue_id":"islandflow-259.2","depends_on_id":"islandflow-zxh.1","type":"blocks","created_at":"2026-06-16T13:39:04Z","created_by":"dirtydishes","metadata":"{}"}],"dependency_count":1,"dependent_count":2,"comment_count":0}
{"_type":"issue","id":"islandflow-zxh.1","title":"Smart-flow phase 01: contracts and vocabulary","description":"Implement the contract and naming foundation described in docs/implementation/smart-money/01-contracts-vocabulary.md. The phase separates facts, evidence, hypotheses, confidence, abstention, and user-facing insight projections before classifier behavior is expanded.","acceptance_criteria":"Canonical contracts distinguish observations, evidence clusters, hypotheses, confidence vectors, abstention reasons, and insight projections; legacy smart-money naming is compatibility-only where needed; version fields are present; migration risks and aliases are documented.","spec_id":"docs/implementation/smart-money/01-contracts-vocabulary.md","status":"open","priority":1,"issue_type":"feature","owner":"dishes@dpdrm.com","created_at":"2026-06-16T17:38:36Z","created_by":"dirtydishes","updated_at":"2026-06-16T17:38:36Z","labels":["phase","planning","smart-flow","smart-money"],"dependencies":[{"issue_id":"islandflow-zxh.1","depends_on_id":"islandflow-259.1","type":"blocks","created_at":"2026-06-16T13:39:03Z","created_by":"dirtydishes","metadata":"{}"},{"issue_id":"islandflow-zxh.1","depends_on_id":"islandflow-zxh","type":"parent-child","created_at":"2026-06-16T13:38:35Z","created_by":"dirtydishes","metadata":"{}"}],"dependency_count":1,"dependent_count":1,"comment_count":0}
{"_type":"issue","id":"islandflow-259.1","title":"Synthetic market-data phase 01: deterministic spine","description":"Implement the deterministic synthetic market-data foundation described in docs/implementation/synthetic-market-data/01-deterministic-spine.md. The phase extracts generation into a seeded package/API that emits canonical market events with provenance while keeping labels separate.","acceptance_criteria":"Seeded generation is byte/hash stable for fixed inputs; emitted events use canonical OptionPrint, OptionNBBO, EquityPrint, and EquityQuote contracts; provenance metadata includes run/seed/parameter context; hidden labels are not embedded in market events; early tests run without Docker, ClickHouse, NATS, or Redis.","spec_id":"docs/implementation/synthetic-market-data/01-deterministic-spine.md","status":"open","priority":1,"issue_type":"feature","owner":"dishes@dpdrm.com","created_at":"2026-06-16T17:38:34Z","created_by":"dirtydishes","updated_at":"2026-06-16T17:38:34Z","labels":["phase","planning","synthetic-market-data"],"dependencies":[{"issue_id":"islandflow-259.1","depends_on_id":"islandflow-259","type":"parent-child","created_at":"2026-06-16T13:38:33Z","created_by":"dirtydishes","metadata":"{}"}],"dependency_count":0,"dependent_count":1,"comment_count":0}
{"_type":"issue","id":"islandflow-zxh","title":"Plan smart-money to smart-flow implementation phases","description":"Track the phase-by-phase implementation plan split from docs/plans/smart-flow-architecture-review.md. Durable roadmap: docs/implementation/smart-money/00-roadmap.md. This epic covers evidence-backed smart-flow contracts, clustering, hypothesis scoring, replay evaluation, API/UI explainability, and future calibration.","acceptance_criteria":"Phase docs exist under docs/implementation/smart-money; child phase issues link to their docs; blocker dependencies reflect the planned implementation order; no application code is implemented as part of this planning epic.","spec_id":"docs/implementation/smart-money/00-roadmap.md","status":"open","priority":1,"issue_type":"epic","owner":"dishes@dpdrm.com","created_at":"2026-06-16T17:38:32Z","created_by":"dirtydishes","updated_at":"2026-06-16T17:38:32Z","labels":["planning","smart-flow","smart-money"],"dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"islandflow-259","title":"Plan synthetic market-data implementation phases","description":"Track the phase-by-phase implementation plan split from docs/plans/synthetic-market-data-architecture-review.md. Durable roadmap: docs/implementation/synthetic-market-data/00-roadmap.md. This epic covers deterministic synthetic event generation with canonical market event types, separate labels/manifests, replay integration, demo/load profiles, and future historical calibration.","acceptance_criteria":"Phase docs exist under docs/implementation/synthetic-market-data; child phase issues link to their docs; blocker dependencies reflect the planned implementation order; no application code is implemented as part of this planning epic.","spec_id":"docs/implementation/synthetic-market-data/00-roadmap.md","status":"open","priority":1,"issue_type":"epic","owner":"dishes@dpdrm.com","created_at":"2026-06-16T17:38:30Z","created_by":"dirtydishes","updated_at":"2026-06-16T17:38:30Z","labels":["planning","synthetic-market-data"],"dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"islandflow-0e3","title":"Fix PR 23 CI failures","description":"PR 23 is failing the Forgejo CI Validate workflow. Reproduce the failing gates locally, fix the underlying formatting/lint/typecheck/test/build issues, update the PR branch, and confirm the remote check passes.","status":"closed","priority":1,"issue_type":"bug","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-06-14T19:35:07Z","created_by":"dirtydishes","updated_at":"2026-06-14T19:37:01Z","started_at":"2026-06-14T19:35:12Z","closed_at":"2026-06-14T19:37:01Z","close_reason":"Local Validate workflow passes after applying formatter output and syncing the Docker workspace snapshot.","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"islandflow-0e3","title":"Fix PR 23 CI failures","description":"PR 23 is failing the Forgejo CI Validate workflow. Reproduce the failing gates locally, fix the underlying formatting/lint/typecheck/test/build issues, update the PR branch, and confirm the remote check passes.","status":"closed","priority":1,"issue_type":"bug","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-06-14T19:35:07Z","created_by":"dirtydishes","updated_at":"2026-06-14T19:37:01Z","started_at":"2026-06-14T19:35:12Z","closed_at":"2026-06-14T19:37:01Z","close_reason":"Local Validate workflow passes after applying formatter output and syncing the Docker workspace snapshot.","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"islandflow-9w7","title":"Allow local dev origins on hosted API","description":"Local bun run dev:web and desktop-local point at the hosted API, but browser requests from http://127.0.0.1:3000 are blocked because the API omits CORS headers and returns 404 for OPTIONS preflight. Add API-side CORS handling, validate local web/desktop browser access, and deploy the API fix.","acceptance_criteria":"API responses include Access-Control-Allow-Origin for allowed local/dev origins; OPTIONS preflight succeeds; bun run dev:web reaches hosted REST/WS endpoints from a browser; bun run dev:desktop local mode reaches the backend through the local web UI; tests/build pass; fix is deployed to api.flow.deltaisland.io.","status":"closed","priority":1,"issue_type":"bug","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-06-13T15:04:19Z","created_by":"dirtydishes","updated_at":"2026-06-13T15:29:42Z","started_at":"2026-06-13T15:04:26Z","closed_at":"2026-06-13T15:29:42Z","close_reason":"Hosted API now reflects allowed local dev origins and handles OPTIONS preflight; local web and desktop dev runners both reach https://api.flow.deltaisland.io; API tests, typecheck, and web build passed.","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"islandflow-9w7","title":"Allow local dev origins on hosted API","description":"Local bun run dev:web and desktop-local point at the hosted API, but browser requests from http://127.0.0.1:3000 are blocked because the API omits CORS headers and returns 404 for OPTIONS preflight. Add API-side CORS handling, validate local web/desktop browser access, and deploy the API fix.","acceptance_criteria":"API responses include Access-Control-Allow-Origin for allowed local/dev origins; OPTIONS preflight succeeds; bun run dev:web reaches hosted REST/WS endpoints from a browser; bun run dev:desktop local mode reaches the backend through the local web UI; tests/build pass; fix is deployed to api.flow.deltaisland.io.","status":"closed","priority":1,"issue_type":"bug","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-06-13T15:04:19Z","created_by":"dirtydishes","updated_at":"2026-06-13T15:29:42Z","started_at":"2026-06-13T15:04:26Z","closed_at":"2026-06-13T15:29:42Z","close_reason":"Hosted API now reflects allowed local dev origins and handles OPTIONS preflight; local web and desktop dev runners both reach https://api.flow.deltaisland.io; API tests, typecheck, and web build passed.","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"islandflow-xkq","title":"Rebuild production dashboard options news around mock9 aesthetic","description":"Reconstruct the production web UI for Dashboard, Options, and News around the mock9 through mock12 dense terminal aesthetic while preserving production data subscriptions, drawers, virtualization, route helpers, redirects, and validation.","status":"closed","priority":1,"issue_type":"feature","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-06-13T14:07:34Z","created_by":"dirtydishes","updated_at":"2026-06-13T14:26:46Z","started_at":"2026-06-13T14:07:53Z","closed_at":"2026-06-13T14:26:46Z","close_reason":"Rebuilt Dashboard, Options, and News around the dense mock9 to mock12 production aesthetic; tests and build passed, and Browser visual inspection was documented as blocked by the unavailable in-app browser backend.","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"islandflow-xkq","title":"Rebuild production dashboard options news around mock9 aesthetic","description":"Reconstruct the production web UI for Dashboard, Options, and News around the mock9 through mock12 dense terminal aesthetic while preserving production data subscriptions, drawers, virtualization, route helpers, redirects, and validation.","status":"closed","priority":1,"issue_type":"feature","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-06-13T14:07:34Z","created_by":"dirtydishes","updated_at":"2026-06-13T14:26:46Z","started_at":"2026-06-13T14:07:53Z","closed_at":"2026-06-13T14:26:46Z","close_reason":"Rebuilt Dashboard, Options, and News around the dense mock9 to mock12 production aesthetic; tests and build passed, and Browser visual inspection was documented as blocked by the unavailable in-app browser backend.","dependency_count":0,"dependent_count":0,"comment_count":0}
@ -23,7 +33,7 @@
{"_type":"issue","id":"islandflow-k4f","title":"Gate deploy script on docker workspace snapshot sync","description":"Prevent frozen-lockfile build failures during deploy by adding a local preflight in scripts/deploy.ts that runs bun run check:docker-workspace and aborts with a clear sync+commit remediation message when stale.","status":"closed","priority":1,"issue_type":"task","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-05-15T23:01:44Z","created_by":"dirtydishes","updated_at":"2026-05-15T23:04:11Z","started_at":"2026-05-15T23:01:48Z","closed_at":"2026-05-15T23:04:11Z","close_reason":"Closed","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"islandflow-k4f","title":"Gate deploy script on docker workspace snapshot sync","description":"Prevent frozen-lockfile build failures during deploy by adding a local preflight in scripts/deploy.ts that runs bun run check:docker-workspace and aborts with a clear sync+commit remediation message when stale.","status":"closed","priority":1,"issue_type":"task","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-05-15T23:01:44Z","created_by":"dirtydishes","updated_at":"2026-05-15T23:04:11Z","started_at":"2026-05-15T23:01:48Z","closed_at":"2026-05-15T23:04:11Z","close_reason":"Closed","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"islandflow-xll","title":"Fix bun.lock drift causing frozen-lockfile Docker build failures","description":"Docker image builds fail in multiple targets (candles, web, ingest services) because bun install --frozen-lockfile detects lockfile changes. Update workspace lockfile to match manifests and verify frozen install succeeds.","status":"closed","priority":1,"issue_type":"bug","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-05-15T22:52:38Z","created_by":"dirtydishes","updated_at":"2026-05-15T22:55:23Z","started_at":"2026-05-15T22:52:40Z","closed_at":"2026-05-15T22:55:23Z","close_reason":"Closed","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"islandflow-xll","title":"Fix bun.lock drift causing frozen-lockfile Docker build failures","description":"Docker image builds fail in multiple targets (candles, web, ingest services) because bun install --frozen-lockfile detects lockfile changes. Update workspace lockfile to match manifests and verify frozen install succeeds.","status":"closed","priority":1,"issue_type":"bug","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-05-15T22:52:38Z","created_by":"dirtydishes","updated_at":"2026-05-15T22:55:23Z","started_at":"2026-05-15T22:52:40Z","closed_at":"2026-05-15T22:55:23Z","close_reason":"Closed","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"islandflow-9nd","title":"Hosted synthetic tape redesign with internal control surface","description":"Implement hosted synthetic market redesign with shared deterministic regime engine, internal JetStream KV control plane, ingest coupling across options and equities, and an internal bottom-right synthetic-control drawer with Next proxy routes. Preserve the six public smart-money categories while adding hidden subtype families, soft coverage accounting, and backend-only admin endpoints.\n","status":"closed","priority":1,"issue_type":"feature","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-05-14T01:25:02Z","created_by":"dirtydishes","updated_at":"2026-05-14T02:10:03Z","started_at":"2026-05-14T01:25:09Z","closed_at":"2026-05-14T02:10:03Z","close_reason":"Completed","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"islandflow-9nd","title":"Hosted synthetic tape redesign with internal control surface","description":"Implement hosted synthetic market redesign with shared deterministic regime engine, internal JetStream KV control plane, ingest coupling across options and equities, and an internal bottom-right synthetic-control drawer with Next proxy routes. Preserve the six public smart-money categories while adding hidden subtype families, soft coverage accounting, and backend-only admin endpoints.\n","status":"closed","priority":1,"issue_type":"feature","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-05-14T01:25:02Z","created_by":"dirtydishes","updated_at":"2026-05-14T02:10:03Z","started_at":"2026-05-14T01:25:09Z","closed_at":"2026-05-14T02:10:03Z","close_reason":"Completed","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"islandflow-9dz","title":"Tune synthetic smart-money scenario coverage","description":"Redesign synthetic smart-money option prints so the emitted scenarios trigger each classifier category more consistently while staying directionally plausible. Focus on scenario mix, DTE/moneyness, price placement, and event/structure context so the Electron demo reliably shows institutional directional, retail whale, event-driven, vol seller, arbitrage, and hedge reactive hits.\n","status":"in_progress","priority":1,"issue_type":"task","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-05-13T21:36:37Z","created_by":"dirtydishes","updated_at":"2026-05-13T21:36:41Z","started_at":"2026-05-13T21:36:41Z","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"islandflow-9dz","title":"Tune synthetic smart-money scenario coverage","description":"Redesign synthetic smart-money option prints so the emitted scenarios trigger each classifier category more consistently while staying directionally plausible. Focus on scenario mix, DTE/moneyness, price placement, and event/structure context so the Electron demo reliably shows institutional directional, retail whale, event-driven, vol seller, arbitrage, and hedge reactive hits.\n","notes":"2026-06-16 planning split: canonical phase tracking now lives under islandflow-259 and islandflow-zxh, with scenario/label work centered on docs/implementation/synthetic-market-data/03-scenarios-labels-expected-outputs.md and demo/load work on docs/implementation/synthetic-market-data/05-demo-load-profiles.md. Re-scope or close this issue after deterministic foundations are in place so it does not bypass the new phase dependency graph.","status":"in_progress","priority":1,"issue_type":"task","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-05-13T21:36:37Z","created_by":"dirtydishes","updated_at":"2026-06-16T17:43:59Z","started_at":"2026-05-13T21:36:41Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"islandflow-zuf","title":"Fix Home to Tape tab navigation freeze","description":"Home-to-Tape navigation becomes unresponsive because TerminalAppShell enters a live-mode rerender loop. The pinned-evidence prune effect writes new Map instances even when contents are unchanged, which can retrigger state updates indefinitely on the Home route where alert evidence prefetch is active. Make pruning idempotent and add regression coverage.\n","status":"closed","priority":1,"issue_type":"bug","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-05-13T15:05:56Z","created_by":"dirtydishes","updated_at":"2026-05-13T15:08:01Z","started_at":"2026-05-13T15:06:06Z","closed_at":"2026-05-13T15:08:01Z","close_reason":"Completed","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"islandflow-zuf","title":"Fix Home to Tape tab navigation freeze","description":"Home-to-Tape navigation becomes unresponsive because TerminalAppShell enters a live-mode rerender loop. The pinned-evidence prune effect writes new Map instances even when contents are unchanged, which can retrigger state updates indefinitely on the Home route where alert evidence prefetch is active. Make pruning idempotent and add regression coverage.\n","status":"closed","priority":1,"issue_type":"bug","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-05-13T15:05:56Z","created_by":"dirtydishes","updated_at":"2026-05-13T15:08:01Z","started_at":"2026-05-13T15:06:06Z","closed_at":"2026-05-13T15:08:01Z","close_reason":"Completed","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"islandflow-9ug","title":"Electron desktop shell for hosted Islandflow","description":"Build a macOS-first Electron desktop shell workspace that loads hosted Islandflow in a locked-down BrowserWindow, adds Bun-first dev/package scripts, documents the workflow, and preserves the existing remote API/WS contract.\n","status":"closed","priority":1,"issue_type":"feature","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-05-13T13:11:40Z","created_by":"dirtydishes","updated_at":"2026-05-13T13:20:57Z","started_at":"2026-05-13T13:12:03Z","closed_at":"2026-05-13T13:20:57Z","close_reason":"Completed","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"islandflow-9ug","title":"Electron desktop shell for hosted Islandflow","description":"Build a macOS-first Electron desktop shell workspace that loads hosted Islandflow in a locked-down BrowserWindow, adds Bun-first dev/package scripts, documents the workflow, and preserves the existing remote API/WS contract.\n","status":"closed","priority":1,"issue_type":"feature","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-05-13T13:11:40Z","created_by":"dirtydishes","updated_at":"2026-05-13T13:20:57Z","started_at":"2026-05-13T13:12:03Z","closed_at":"2026-05-13T13:20:57Z","close_reason":"Completed","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"islandflow-sh1","title":"Fix live websocket stale lag and reconnect loop","description":"Investigate and fix API live consumer lag causing stale timestamps, feed-behind status, and reconnect loops. Optimize live cache persistence path, add lag telemetry/alerts, and validate in runtime.","status":"closed","priority":1,"issue_type":"bug","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-05-04T17:04:34Z","created_by":"dirtydishes","updated_at":"2026-05-04T17:09:44Z","started_at":"2026-05-04T17:04:38Z","closed_at":"2026-05-04T17:09:44Z","close_reason":"Completed: optimized live cache persistence path, added lag telemetry, deployed api via docker compose on di, verified ws freshness and low hotFeedLagMs","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"islandflow-sh1","title":"Fix live websocket stale lag and reconnect loop","description":"Investigate and fix API live consumer lag causing stale timestamps, feed-behind status, and reconnect loops. Optimize live cache persistence path, add lag telemetry/alerts, and validate in runtime.","status":"closed","priority":1,"issue_type":"bug","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-05-04T17:04:34Z","created_by":"dirtydishes","updated_at":"2026-05-04T17:09:44Z","started_at":"2026-05-04T17:04:38Z","closed_at":"2026-05-04T17:09:44Z","close_reason":"Completed: optimized live cache persistence path, added lag telemetry, deployed api via docker compose on di, verified ws freshness and low hotFeedLagMs","dependency_count":0,"dependent_count":0,"comment_count":0}
@ -32,6 +42,16 @@
{"_type":"issue","id":"islandflow-ayo","title":"Drop stale backlog events from live fanout","description":"Follow-up to live freshness rollout: /ws/live was still fanning out stale backlog events for freshness-gated channels, which kept tape panes in Live feed behind despite active synthetic ingest. Gate fanout and cache ingest by freshness for options/nbbo/equities/flow.","status":"closed","priority":1,"issue_type":"bug","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-04-28T21:26:39Z","created_by":"dirtydishes","updated_at":"2026-04-28T21:26:44Z","started_at":"2026-04-28T21:26:44Z","closed_at":"2026-04-28T21:26:44Z","close_reason":"Completed","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"islandflow-ayo","title":"Drop stale backlog events from live fanout","description":"Follow-up to live freshness rollout: /ws/live was still fanning out stale backlog events for freshness-gated channels, which kept tape panes in Live feed behind despite active synthetic ingest. Gate fanout and cache ingest by freshness for options/nbbo/equities/flow.","status":"closed","priority":1,"issue_type":"bug","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-04-28T21:26:39Z","created_by":"dirtydishes","updated_at":"2026-04-28T21:26:44Z","started_at":"2026-04-28T21:26:44Z","closed_at":"2026-04-28T21:26:44Z","close_reason":"Completed","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"islandflow-0v6","title":"Fix tape freshness, NBBO coverage, pause controls, and filter popup","description":"Implement the tape fixes requested for synthetic options notional sizing, strict live freshness, live-mode pause/resume behavior, stronger NBBO snapshot coverage, and moving flow filters behind a popup. Includes server-side live cache changes, web terminal state/UI changes, and tests for synthetic pricing, live snapshot freshness/NBBO retention, and live pause/filter interactions.","status":"closed","priority":1,"issue_type":"task","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-04-28T21:02:52Z","created_by":"dirtydishes","updated_at":"2026-04-28T21:13:38Z","started_at":"2026-04-28T21:02:57Z","closed_at":"2026-04-28T21:13:38Z","close_reason":"Completed","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"islandflow-0v6","title":"Fix tape freshness, NBBO coverage, pause controls, and filter popup","description":"Implement the tape fixes requested for synthetic options notional sizing, strict live freshness, live-mode pause/resume behavior, stronger NBBO snapshot coverage, and moving flow filters behind a popup. Includes server-side live cache changes, web terminal state/UI changes, and tests for synthetic pricing, live snapshot freshness/NBBO retention, and live pause/filter interactions.","status":"closed","priority":1,"issue_type":"task","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-04-28T21:02:52Z","created_by":"dirtydishes","updated_at":"2026-04-28T21:13:38Z","started_at":"2026-04-28T21:02:57Z","closed_at":"2026-04-28T21:13:38Z","close_reason":"Completed","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"islandflow-e4r","title":"Implement smart-money flow filtering and synthetic firehose modes","description":"Implement the approved multi-surface plan for named synthetic market profiles, options raw-vs-signal filtering, live/API filter contracts, Tape page client-side flow filters, firehose-readiness improvements, tests, and README updates.","status":"closed","priority":1,"issue_type":"feature","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-04-28T20:10:49Z","created_by":"dirtydishes","updated_at":"2026-04-28T20:29:29Z","started_at":"2026-04-28T20:10:53Z","closed_at":"2026-04-28T20:29:29Z","close_reason":"Implemented synthetic market profiles, options signal-path filtering, signal-aware API/replay contracts, Tape page filters, tests, and README updates. Follow-up tracked in islandflow-biq.","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"islandflow-e4r","title":"Implement smart-money flow filtering and synthetic firehose modes","description":"Implement the approved multi-surface plan for named synthetic market profiles, options raw-vs-signal filtering, live/API filter contracts, Tape page client-side flow filters, firehose-readiness improvements, tests, and README updates.","status":"closed","priority":1,"issue_type":"feature","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-04-28T20:10:49Z","created_by":"dirtydishes","updated_at":"2026-04-28T20:29:29Z","started_at":"2026-04-28T20:10:53Z","closed_at":"2026-04-28T20:29:29Z","close_reason":"Implemented synthetic market profiles, options signal-path filtering, signal-aware API/replay contracts, Tape page filters, tests, and README updates. Follow-up tracked in islandflow-biq.","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"islandflow-zxh.5.2","title":"Split smart-flow phase 05b: UI explainability surfaces","description":"PR-sized child of smart-flow phase 05. Rework UI surfaces around evidence quality, confidence versus conviction, alternatives, abstention, and why-not context.","acceptance_criteria":"UI explains why a hypothesis exists, why it may be weak, and why the system abstained; evidence refs and alternatives are inspectable; copy avoids overconfident smart-money claims.","spec_id":"docs/implementation/smart-money/05-api-ui-explainability.md","status":"open","priority":2,"issue_type":"task","owner":"dishes@dpdrm.com","created_at":"2026-06-16T17:39:02Z","created_by":"dirtydishes","updated_at":"2026-06-16T17:39:02Z","labels":["phase","phase-split","planning","smart-flow","smart-money"],"dependencies":[{"issue_id":"islandflow-zxh.5.2","depends_on_id":"islandflow-zxh.5","type":"parent-child","created_at":"2026-06-16T13:39:01Z","created_by":"dirtydishes","metadata":"{}"},{"issue_id":"islandflow-zxh.5.2","depends_on_id":"islandflow-zxh.5.1","type":"blocks","created_at":"2026-06-16T13:39:20Z","created_by":"dirtydishes","metadata":"{}"}],"dependency_count":1,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"islandflow-zxh.5.1","title":"Split smart-flow phase 05a: evidence API and websocket surfaces","description":"PR-sized child of smart-flow phase 05. Expose evidence, hypotheses, insights, alternatives, and abstention through API and websocket surfaces.","acceptance_criteria":"New or aliased endpoints expose typed evidence/hypothesis/insight payloads; legacy smart-money surfaces are compatibility aliases; responses include version, refs, abstention, and alternative explanation fields.","spec_id":"docs/implementation/smart-money/05-api-ui-explainability.md","status":"open","priority":2,"issue_type":"task","owner":"dishes@dpdrm.com","created_at":"2026-06-16T17:39:00Z","created_by":"dirtydishes","updated_at":"2026-06-16T17:39:00Z","labels":["phase","phase-split","planning","smart-flow","smart-money"],"dependencies":[{"issue_id":"islandflow-zxh.5.1","depends_on_id":"islandflow-259.5","type":"blocks","created_at":"2026-06-16T13:39:19Z","created_by":"dirtydishes","metadata":"{}"},{"issue_id":"islandflow-zxh.5.1","depends_on_id":"islandflow-zxh.5","type":"parent-child","created_at":"2026-06-16T13:39:00Z","created_by":"dirtydishes","metadata":"{}"}],"dependency_count":1,"dependent_count":1,"comment_count":0}
{"_type":"issue","id":"islandflow-zxh.3.2","title":"Split smart-flow phase 03b: abstention and insight projection","description":"PR-sized child of smart-flow phase 03. Add abstention, alternatives, and compatibility insight projections without making projections canonical.","acceptance_criteria":"Abstention reasons are first-class; alternative explanations are exposed; SmartFlowInsight projection is derived from hypotheses and remains separate from canonical evidence/hypothesis events.","spec_id":"docs/implementation/smart-money/03-hypothesis-scoring-abstention.md","status":"open","priority":2,"issue_type":"task","owner":"dishes@dpdrm.com","created_at":"2026-06-16T17:38:59Z","created_by":"dirtydishes","updated_at":"2026-06-16T17:38:59Z","labels":["phase","phase-split","planning","smart-flow","smart-money"],"dependencies":[{"issue_id":"islandflow-zxh.3.2","depends_on_id":"islandflow-zxh.3","type":"parent-child","created_at":"2026-06-16T13:38:58Z","created_by":"dirtydishes","metadata":"{}"},{"issue_id":"islandflow-zxh.3.2","depends_on_id":"islandflow-zxh.3.1","type":"blocks","created_at":"2026-06-16T13:39:18Z","created_by":"dirtydishes","metadata":"{}"}],"dependency_count":1,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"islandflow-zxh.3.1","title":"Split smart-flow phase 03a: hypothesis score vectors","description":"PR-sized child of smart-flow phase 03. Convert evidence clusters into typed hypothesis score vectors with policy/model versions.","acceptance_criteria":"Hypothesis score vectors include evidence strength, direction, confidence, penalties, and policy/model version; scoring avoids participant-identity certainty and preserves negative evidence.","spec_id":"docs/implementation/smart-money/03-hypothesis-scoring-abstention.md","status":"open","priority":2,"issue_type":"task","owner":"dishes@dpdrm.com","created_at":"2026-06-16T17:38:57Z","created_by":"dirtydishes","updated_at":"2026-06-16T17:38:57Z","labels":["phase","phase-split","planning","smart-flow","smart-money"],"dependencies":[{"issue_id":"islandflow-zxh.3.1","depends_on_id":"islandflow-259.3","type":"blocks","created_at":"2026-06-16T13:39:18Z","created_by":"dirtydishes","metadata":"{}"},{"issue_id":"islandflow-zxh.3.1","depends_on_id":"islandflow-zxh.3","type":"parent-child","created_at":"2026-06-16T13:38:57Z","created_by":"dirtydishes","metadata":"{}"}],"dependency_count":1,"dependent_count":1,"comment_count":0}
{"_type":"issue","id":"islandflow-zxh.2.2","title":"Split smart-flow phase 02b: clustering and feature vectors","description":"PR-sized child of smart-flow phase 02. Build evidence clusters and feature vectors from explicit facts while preserving lineage.","acceptance_criteria":"Clusters have deterministic keys and windows; features distinguish measured facts from inferred structure; every feature value can trace back to source evidence refs.","spec_id":"docs/implementation/smart-money/02-evidence-clustering-features.md","status":"open","priority":2,"issue_type":"task","owner":"dishes@dpdrm.com","created_at":"2026-06-16T17:38:56Z","created_by":"dirtydishes","updated_at":"2026-06-16T17:38:56Z","labels":["phase","phase-split","planning","smart-flow","smart-money"],"dependencies":[{"issue_id":"islandflow-zxh.2.2","depends_on_id":"islandflow-zxh.2","type":"parent-child","created_at":"2026-06-16T13:38:55Z","created_by":"dirtydishes","metadata":"{}"},{"issue_id":"islandflow-zxh.2.2","depends_on_id":"islandflow-zxh.2.1","type":"blocks","created_at":"2026-06-16T13:39:17Z","created_by":"dirtydishes","metadata":"{}"}],"dependency_count":1,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"islandflow-zxh.2.1","title":"Split smart-flow phase 02a: eligibility and evidence facts","description":"PR-sized child of smart-flow phase 02. Extract direct observations, quote joins, execution context, and eligibility decisions as explicit evidence facts.","acceptance_criteria":"Evidence facts preserve raw refs and quote context; eligibility decisions carry accept/reject/down-weight reasons; stale quotes, wide spreads, and noisy conditions are represented without hypothesis language.","spec_id":"docs/implementation/smart-money/02-evidence-clustering-features.md","status":"open","priority":2,"issue_type":"task","owner":"dishes@dpdrm.com","created_at":"2026-06-16T17:38:54Z","created_by":"dirtydishes","updated_at":"2026-06-16T17:38:54Z","labels":["phase","phase-split","planning","smart-flow","smart-money"],"dependencies":[{"issue_id":"islandflow-zxh.2.1","depends_on_id":"islandflow-259.2","type":"blocks","created_at":"2026-06-16T13:39:16Z","created_by":"dirtydishes","metadata":"{}"},{"issue_id":"islandflow-zxh.2.1","depends_on_id":"islandflow-zxh.2","type":"parent-child","created_at":"2026-06-16T13:38:54Z","created_by":"dirtydishes","metadata":"{}"}],"dependency_count":1,"dependent_count":1,"comment_count":0}
{"_type":"issue","id":"islandflow-259.3.2","title":"Split synthetic phase 03b: expected-output manifests","description":"PR-sized child of synthetic phase 03. Add expected-output manifest entries and comparison hooks for generated scenarios.","acceptance_criteria":"Expected-output manifests capture derived event expectations, alert/no-alert outcomes, evidence requirements, and false-positive penalties; comparison output is deterministic and focused for review.","spec_id":"docs/implementation/synthetic-market-data/03-scenarios-labels-expected-outputs.md","status":"open","priority":2,"issue_type":"task","owner":"dishes@dpdrm.com","created_at":"2026-06-16T17:38:53Z","created_by":"dirtydishes","updated_at":"2026-06-16T17:38:53Z","labels":["phase","phase-split","planning","synthetic-market-data"],"dependencies":[{"issue_id":"islandflow-259.3.2","depends_on_id":"islandflow-259.3","type":"parent-child","created_at":"2026-06-16T13:38:52Z","created_by":"dirtydishes","metadata":"{}"},{"issue_id":"islandflow-259.3.2","depends_on_id":"islandflow-259.3.1","type":"blocks","created_at":"2026-06-16T13:39:15Z","created_by":"dirtydishes","metadata":"{}"}],"dependency_count":1,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"islandflow-259.3.1","title":"Split synthetic phase 03a: scenario catalog and labels","description":"PR-sized child of synthetic phase 03. Define the deterministic scenario catalog and separate label records without changing emitted market event contracts.","acceptance_criteria":"Scenario definitions are deterministic and named; labels are stored separately from market events; each label records run/scenario/event refs, expected class/direction/confidence band, and required or forbidden evidence.","spec_id":"docs/implementation/synthetic-market-data/03-scenarios-labels-expected-outputs.md","status":"open","priority":2,"issue_type":"task","owner":"dishes@dpdrm.com","created_at":"2026-06-16T17:38:51Z","created_by":"dirtydishes","updated_at":"2026-06-16T17:38:51Z","labels":["phase","phase-split","planning","synthetic-market-data"],"dependencies":[{"issue_id":"islandflow-259.3.1","depends_on_id":"islandflow-259.3","type":"parent-child","created_at":"2026-06-16T13:38:51Z","created_by":"dirtydishes","metadata":"{}"},{"issue_id":"islandflow-259.3.1","depends_on_id":"islandflow-zxh.2","type":"blocks","created_at":"2026-06-16T13:39:14Z","created_by":"dirtydishes","metadata":"{}"}],"dependency_count":1,"dependent_count":1,"comment_count":0}
{"_type":"issue","id":"islandflow-zxh.5","title":"Smart-flow phase 05: API/UI explainability","description":"Expose evidence-backed smart-flow results through API, websocket, and UI explainability surfaces as described in docs/implementation/smart-money/05-api-ui-explainability.md. This comes after replay-validated contracts and scoring behavior exist.","acceptance_criteria":"API/WS surfaces expose evidence, hypotheses, insights, alternatives, and abstention clearly; UI distinguishes evidence quality, confidence, conviction, and why-not signals; legacy endpoints remain aliases only where needed; replay/golden checks support changed projections.","spec_id":"docs/implementation/smart-money/05-api-ui-explainability.md","status":"open","priority":2,"issue_type":"feature","owner":"dishes@dpdrm.com","created_at":"2026-06-16T17:38:47Z","created_by":"dirtydishes","updated_at":"2026-06-16T17:38:47Z","labels":["phase","planning","smart-flow","smart-money"],"dependencies":[{"issue_id":"islandflow-zxh.5","depends_on_id":"islandflow-259.5","type":"blocks","created_at":"2026-06-16T13:39:11Z","created_by":"dirtydishes","metadata":"{}"},{"issue_id":"islandflow-zxh.5","depends_on_id":"islandflow-zxh","type":"parent-child","created_at":"2026-06-16T13:38:47Z","created_by":"dirtydishes","metadata":"{}"}],"dependency_count":1,"dependent_count":1,"comment_count":0}
{"_type":"issue","id":"islandflow-259.5","title":"Synthetic market-data phase 05: demo and load profiles","description":"Implement named deterministic demo and load profiles as described in docs/implementation/synthetic-market-data/05-demo-load-profiles.md. This comes after foundations, manifests, scenarios, and replay are in place.","acceptance_criteria":"Demo profiles select named deterministic runs/scenarios instead of ambient randomness; load profiles scale rates without changing event semantics; live/demo emitters remain thin consumers of the synthetic package; UI/demo controls do not precede deterministic foundations.","spec_id":"docs/implementation/synthetic-market-data/05-demo-load-profiles.md","status":"open","priority":2,"issue_type":"feature","owner":"dishes@dpdrm.com","created_at":"2026-06-16T17:38:46Z","created_by":"dirtydishes","updated_at":"2026-06-16T17:38:46Z","labels":["phase","planning","synthetic-market-data"],"dependencies":[{"issue_id":"islandflow-259.5","depends_on_id":"islandflow-259","type":"parent-child","created_at":"2026-06-16T13:38:45Z","created_by":"dirtydishes","metadata":"{}"},{"issue_id":"islandflow-259.5","depends_on_id":"islandflow-zxh.4","type":"blocks","created_at":"2026-06-16T13:39:10Z","created_by":"dirtydishes","metadata":"{}"}],"dependency_count":1,"dependent_count":3,"comment_count":0}
{"_type":"issue","id":"islandflow-v6f","title":"Virtualize dashboard priority board","description":"Improve the root dashboard Priority Board readability and scrolling. Remove the redundant packet column, show packet IDs as secondary evidence metadata without the flowpacket prefix, rename the confusing Decision column, and use TanStack virtual scrolling for the row list.","status":"closed","priority":2,"issue_type":"task","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-06-15T20:58:29Z","created_by":"dirtydishes","updated_at":"2026-06-15T21:01:37Z","started_at":"2026-06-15T20:58:34Z","closed_at":"2026-06-15T21:01:37Z","close_reason":"Completed Priority Board virtualization, copy cleanup, and validation.","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"islandflow-v6f","title":"Virtualize dashboard priority board","description":"Improve the root dashboard Priority Board readability and scrolling. Remove the redundant packet column, show packet IDs as secondary evidence metadata without the flowpacket prefix, rename the confusing Decision column, and use TanStack virtual scrolling for the row list.","status":"closed","priority":2,"issue_type":"task","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-06-15T20:58:29Z","created_by":"dirtydishes","updated_at":"2026-06-15T21:01:37Z","started_at":"2026-06-15T20:58:34Z","closed_at":"2026-06-15T21:01:37Z","close_reason":"Completed Priority Board virtualization, copy cleanup, and validation.","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"islandflow-h0k","title":"Polish dashboard route","description":"Final polish pass for the root dashboard route. Align the command surface to the existing Islandflow terminal design system, tighten visual and copy details, and validate responsive behavior and build quality.","status":"closed","priority":2,"issue_type":"task","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-06-15T20:04:12Z","created_by":"dirtydishes","updated_at":"2026-06-15T20:09:14Z","started_at":"2026-06-15T20:04:13Z","closed_at":"2026-06-15T20:09:14Z","close_reason":"Dashboard route polish shipped on dedicated branch: route title hierarchy cleaned up, command rail isolated from legacy header styles, mobile overflow/touch behavior added, and build/tests passed.","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"islandflow-h0k","title":"Polish dashboard route","description":"Final polish pass for the root dashboard route. Align the command surface to the existing Islandflow terminal design system, tighten visual and copy details, and validate responsive behavior and build quality.","status":"closed","priority":2,"issue_type":"task","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-06-15T20:04:12Z","created_by":"dirtydishes","updated_at":"2026-06-15T20:09:14Z","started_at":"2026-06-15T20:04:13Z","closed_at":"2026-06-15T20:09:14Z","close_reason":"Dashboard route polish shipped on dedicated branch: route title hierarchy cleaned up, command rail isolated from legacy header styles, mobile overflow/touch behavior added, and build/tests passed.","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"islandflow-j30","title":"Polish dashboard command header","description":"Live design polish for the dashboard command header. Simplify the page header copy to Dashboard, keep status and scope in one compact line, and replace the vague empty-filter copy with a clearer all-flow state.","status":"closed","priority":2,"issue_type":"task","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-06-15T19:29:23Z","created_by":"dirtydishes","updated_at":"2026-06-15T19:31:00Z","started_at":"2026-06-15T19:29:28Z","closed_at":"2026-06-15T19:31:00Z","close_reason":"Dashboard command header polish shipped: simplified title, clarified scope/filter language, compacted status rail, and validated with web build plus focused terminal tests.","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"islandflow-j30","title":"Polish dashboard command header","description":"Live design polish for the dashboard command header. Simplify the page header copy to Dashboard, keep status and scope in one compact line, and replace the vague empty-filter copy with a clearer all-flow state.","status":"closed","priority":2,"issue_type":"task","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-06-15T19:29:23Z","created_by":"dirtydishes","updated_at":"2026-06-15T19:31:00Z","started_at":"2026-06-15T19:29:28Z","closed_at":"2026-06-15T19:31:00Z","close_reason":"Dashboard command header polish shipped: simplified title, clarified scope/filter language, compacted status rail, and validated with web build plus focused terminal tests.","dependency_count":0,"dependent_count":0,"comment_count":0}
@ -150,6 +170,8 @@
{"_type":"issue","id":"islandflow-zsy","title":"Expose Forgejo SSH on a direct DNS hostname","description":"git.deltaisland.io currently resolves through Cloudflare's proxy, so SSH on port 2222 does not complete even though the Forgejo container is listening on the host. If SSH-based git/beads workflows are desired, add a DNS-only hostname (or adjust the existing record) that points directly at the server for Forgejo SSH.","status":"open","priority":3,"issue_type":"task","created_at":"2026-05-17T10:34:06Z","created_by":"delta","updated_at":"2026-05-17T10:34:06Z","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"islandflow-zsy","title":"Expose Forgejo SSH on a direct DNS hostname","description":"git.deltaisland.io currently resolves through Cloudflare's proxy, so SSH on port 2222 does not complete even though the Forgejo container is listening on the host. If SSH-based git/beads workflows are desired, add a DNS-only hostname (or adjust the existing record) that points directly at the server for Forgejo SSH.","status":"open","priority":3,"issue_type":"task","created_at":"2026-05-17T10:34:06Z","created_by":"delta","updated_at":"2026-05-17T10:34:06Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"islandflow-38p","title":"Add native deployment unit templates and rollback helpers","description":"The deploy helper now supports --runtime native, but the repo still relies on operator-managed systemd units and manual rollback. Add checked-in native deployment templates or provisioning guidance for the expected units, and consider lightweight rollback/smoke-test helpers once the host-native path is exercised on the real VPS.","status":"open","priority":3,"issue_type":"task","owner":"dishes@dpdrm.com","created_at":"2026-05-15T23:46:42Z","created_by":"dirtydishes","updated_at":"2026-05-15T23:46:42Z","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"islandflow-38p","title":"Add native deployment unit templates and rollback helpers","description":"The deploy helper now supports --runtime native, but the repo still relies on operator-managed systemd units and manual rollback. Add checked-in native deployment templates or provisioning guidance for the expected units, and consider lightweight rollback/smoke-test helpers once the host-native path is exercised on the real VPS.","status":"open","priority":3,"issue_type":"task","owner":"dishes@dpdrm.com","created_at":"2026-05-15T23:46:42Z","created_by":"dirtydishes","updated_at":"2026-05-15T23:46:42Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"islandflow-575","title":"Document smart-money event calendar env","description":"Document smart-money event-calendar environment configuration in env examples and README.\n","status":"closed","priority":3,"issue_type":"task","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-05-05T06:57:14Z","created_by":"dirtydishes","updated_at":"2026-05-05T06:57:57Z","started_at":"2026-05-05T06:57:17Z","closed_at":"2026-05-05T06:57:57Z","close_reason":"Documented event-calendar env variables","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"islandflow-575","title":"Document smart-money event calendar env","description":"Document smart-money event-calendar environment configuration in env examples and README.\n","status":"closed","priority":3,"issue_type":"task","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-05-05T06:57:14Z","created_by":"dirtydishes","updated_at":"2026-05-05T06:57:57Z","started_at":"2026-05-05T06:57:17Z","closed_at":"2026-05-05T06:57:57Z","close_reason":"Documented event-calendar env variables","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"islandflow-zxh.6","title":"Future smart-flow phase 99: calibration","description":"Track future smart-flow calibration and research-grade validation as described in docs/implementation/smart-money/99-future-calibration.md. This is explicitly not an MVP dependency.","acceptance_criteria":"Future calibration is separated from MVP scoring; calibration datasets, policy versioning, and evaluation metrics are specified before implementation; abstention and confidence semantics remain auditable.","spec_id":"docs/implementation/smart-money/99-future-calibration.md","status":"open","priority":4,"issue_type":"task","owner":"dishes@dpdrm.com","created_at":"2026-06-16T17:38:50Z","created_by":"dirtydishes","updated_at":"2026-06-16T17:38:50Z","labels":["future","planning","smart-flow","smart-money"],"dependencies":[{"issue_id":"islandflow-zxh.6","depends_on_id":"islandflow-259.6","type":"blocks","created_at":"2026-06-16T13:39:13Z","created_by":"dirtydishes","metadata":"{}"},{"issue_id":"islandflow-zxh.6","depends_on_id":"islandflow-zxh","type":"parent-child","created_at":"2026-06-16T13:38:50Z","created_by":"dirtydishes","metadata":"{}"},{"issue_id":"islandflow-zxh.6","depends_on_id":"islandflow-zxh.5","type":"blocks","created_at":"2026-06-16T13:39:12Z","created_by":"dirtydishes","metadata":"{}"}],"dependency_count":2,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"islandflow-259.6","title":"Future synthetic market-data phase 99: historical calibration","description":"Track future historical calibration for the synthetic generator as described in docs/implementation/synthetic-market-data/99-future-historical-calibration.md. This is explicitly not an MVP dependency.","acceptance_criteria":"Historical calibration remains outside the MVP blocker chain; calibration inputs, privacy/market-data constraints, and parameter fitting strategy are documented before implementation; deterministic synthetic foundations continue to work without historical data.","spec_id":"docs/implementation/synthetic-market-data/99-future-historical-calibration.md","status":"open","priority":4,"issue_type":"task","owner":"dishes@dpdrm.com","created_at":"2026-06-16T17:38:49Z","created_by":"dirtydishes","updated_at":"2026-06-16T17:38:49Z","labels":["future","planning","synthetic-market-data"],"dependencies":[{"issue_id":"islandflow-259.6","depends_on_id":"islandflow-259","type":"parent-child","created_at":"2026-06-16T13:38:48Z","created_by":"dirtydishes","metadata":"{}"},{"issue_id":"islandflow-259.6","depends_on_id":"islandflow-259.5","type":"blocks","created_at":"2026-06-16T13:39:12Z","created_by":"dirtydishes","metadata":"{}"}],"dependency_count":1,"dependent_count":1,"comment_count":0}
{"_type":"issue","id":"islandflow-iwg","title":"Summarize June 9 git activity for standup","description":"Create the daily standup summary for git activity on 2026-06-09, grounded in commits and touched files, and save the report in docs/general.","status":"closed","priority":4,"issue_type":"task","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-06-10T13:02:19Z","created_by":"dirtydishes","updated_at":"2026-06-10T13:04:44Z","started_at":"2026-06-10T13:02:30Z","closed_at":"2026-06-10T13:04:44Z","close_reason":"Created docs/general/2026-06-10-0902-standup-summary-2026-06-09.html with a git-grounded summary of June 9 activity.","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"islandflow-iwg","title":"Summarize June 9 git activity for standup","description":"Create the daily standup summary for git activity on 2026-06-09, grounded in commits and touched files, and save the report in docs/general.","status":"closed","priority":4,"issue_type":"task","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-06-10T13:02:19Z","created_by":"dirtydishes","updated_at":"2026-06-10T13:04:44Z","started_at":"2026-06-10T13:02:30Z","closed_at":"2026-06-10T13:04:44Z","close_reason":"Created docs/general/2026-06-10-0902-standup-summary-2026-06-09.html with a git-grounded summary of June 9 activity.","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"islandflow-19f","title":"Summarize June 7 git activity for standup","description":"Create the daily standup summary for yesterday's git activity, grounded in commits and changed files, and save the report in docs/general.","status":"closed","priority":4,"issue_type":"task","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-06-08T13:01:02Z","created_by":"dirtydishes","updated_at":"2026-06-08T13:03:00Z","started_at":"2026-06-08T13:01:15Z","closed_at":"2026-06-08T13:03:00Z","close_reason":"Standup summary for 2026-06-07 created in docs/general with grounded git-history validation.","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"islandflow-19f","title":"Summarize June 7 git activity for standup","description":"Create the daily standup summary for yesterday's git activity, grounded in commits and changed files, and save the report in docs/general.","status":"closed","priority":4,"issue_type":"task","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-06-08T13:01:02Z","created_by":"dirtydishes","updated_at":"2026-06-08T13:03:00Z","started_at":"2026-06-08T13:01:15Z","closed_at":"2026-06-08T13:03:00Z","close_reason":"Standup summary for 2026-06-07 created in docs/general with grounded git-history validation.","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"islandflow-a1m","title":"Publish June 3 standup summary","description":"Why this issue exists and what needs to be done:\\n- Produce the daily standup summary for git activity on 2026-06-03.\\n- Ground every statement in commits and touched files only.\\n- Save the HTML artifact under docs/general and complete the automation handoff workflow.","status":"closed","priority":4,"issue_type":"task","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-06-04T13:02:04Z","created_by":"dirtydishes","updated_at":"2026-06-04T13:03:43Z","started_at":"2026-06-04T13:03:34Z","closed_at":"2026-06-04T13:03:43Z","close_reason":"Created docs/general/2026-06-04-standup-summary-2026-06-03.html with a commit-grounded summary of June 3 git activity.","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"islandflow-a1m","title":"Publish June 3 standup summary","description":"Why this issue exists and what needs to be done:\\n- Produce the daily standup summary for git activity on 2026-06-03.\\n- Ground every statement in commits and touched files only.\\n- Save the HTML artifact under docs/general and complete the automation handoff workflow.","status":"closed","priority":4,"issue_type":"task","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-06-04T13:02:04Z","created_by":"dirtydishes","updated_at":"2026-06-04T13:03:43Z","started_at":"2026-06-04T13:03:34Z","closed_at":"2026-06-04T13:03:43Z","close_reason":"Created docs/general/2026-06-04-standup-summary-2026-06-03.html with a commit-grounded summary of June 3 git activity.","dependency_count":0,"dependent_count":0,"comment_count":0}

View file

@ -69,6 +69,10 @@ Working style that avoids common problems here:
- Keep `.env` aligned with `.env.example`; adapters default to synthetic modes for local development. - Keep `.env` aligned with `.env.example`; adapters default to synthetic modes for local development.
- Dev runners persist child PID state in `.tmp/`; if a previous run crashed, restart via the standard `bun run dev*` commands so stale processes are cleaned up. - Dev runners persist child PID state in `.tmp/`; if a previous run crashed, restart via the standard `bun run dev*` commands so stale processes are cleaned up.
## Implementation Phase Plans
Before beginning synthetic market-data or smart-money/smart-flow implementation work, read `docs/implementation/README.md` and the specific phase file linked from the active Beads issue. Treat `docs/plans/*architecture-review.md` files as background guidance only; the current phase document and Beads issue define the active scope. Keep PRs phase-bounded and reviewable, split smaller Beads child issues when a phase is too large, and add/update Beads dependencies when one phase blocks another.
## Forgejo Is Canonical ## Forgejo Is Canonical
This repository's primary home is Forgejo: This repository's primary home is Forgejo:

View file

@ -0,0 +1,58 @@
# Implementation Phase Plans
This directory is the active planning layer for the synthetic market-data and smart-money/smart-flow architecture work.
The architecture reviews in `docs/plans/` are background guidance. Future implementation work should use the current phase document and matching Beads issue as the active scope. If a phase document and an older architecture review disagree, pause and update the phase document or Beads issue before writing code.
## Source Plans
- `docs/plans/synthetic-market-data-architecture-review.md`
- `docs/plans/smart-flow-architecture-review.md`
## Planning Rules
- Prefer small, reviewable PRs.
- Do not implement an entire architecture plan at once.
- Use Beads issues for execution tracking and dependency management.
- Keep durable architecture and phase detail in these docs, not in long Beads descriptions.
- Synthetic data must emit canonical market event types, not synthetic-only pipeline event types.
- Synthetic labels must remain separate from emitted market events.
- Smart-flow logic must distinguish facts, evidence, hypotheses, confidence, and abstention.
- Historical calibration is future work, not an MVP dependency.
- Early synthetic tests must not require Docker, ClickHouse, NATS, or Redis.
- Synthetic foundations should come before demos, UI controls, or live service work.
## Beads Map
| Stream | Epic | Roadmap |
| --- | --- | --- |
| Synthetic market data | `islandflow-259` - Plan synthetic market-data implementation phases | `docs/implementation/synthetic-market-data/00-roadmap.md` |
| Smart money / smart flow | `islandflow-zxh` - Plan smart-money to smart-flow implementation phases | `docs/implementation/smart-money/00-roadmap.md` |
## Dependency Order
This is the intended MVP ordering. Future calibration phases sit after the MVP chain and should not block it.
| Order | Phase | Beads issue | Blocks next because |
| ---: | --- | --- | --- |
| 1 | Synthetic deterministic spine | `islandflow-259.1` | The smart-flow vocabulary needs stable raw event/provenance assumptions. |
| 2 | Smart-flow contracts and vocabulary | `islandflow-zxh.1` | Synthetic manifests should target the eventual evidence/hypothesis language. |
| 3 | Synthetic manifests, fixtures, and CLI | `islandflow-259.2` | Evidence clustering needs deterministic fixtures before broad behavior changes. |
| 4 | Smart-flow evidence clustering and features | `islandflow-zxh.2` | Scenario labels need the evidence vocabulary they are expected to exercise. |
| 5 | Synthetic scenarios, labels, and expected outputs | `islandflow-259.3` | Hypothesis scoring needs labeled positive, negative, and abstention cases. |
| 6 | Smart-flow hypothesis scoring and abstention | `islandflow-zxh.3` | Synthetic replay integration should validate the derived hypothesis pipeline. |
| 7 | Synthetic replay integration | `islandflow-259.4` | Smart-flow golden tests need replayable synthetic runs. |
| 8 | Smart-flow replay evaluation and golden tests | `islandflow-zxh.4` | Demos should wait until replay proves the semantics. |
| 9 | Synthetic demo and load profiles | `islandflow-259.5` | API/UI explainability should show stable, named, deterministic runs. |
| 10 | Smart-flow API/UI explainability | `islandflow-zxh.5` | This is the final MVP presentation layer after the evidence pipeline is validated. |
## Future Work
| Future phase | Beads issue | Notes |
| --- | --- | --- |
| Synthetic historical calibration | `islandflow-259.6` | Depends on synthetic phase 05, but is not required for MVP. |
| Smart-flow calibration | `islandflow-zxh.6` | Depends on smart-flow phase 05 and synthetic future calibration, but is not required for MVP. |
## Existing Related Issue
`islandflow-9dz` already tracks tuning synthetic smart-money scenario coverage. It is narrower than these phase plans and was already in progress before this split. Treat it as related context for `docs/implementation/synthetic-market-data/03-scenarios-labels-expected-outputs.md`, not as the phase-level tracker.

View file

@ -0,0 +1,40 @@
# Smart Money / Smart Flow Roadmap
This roadmap breaks `docs/plans/smart-flow-architecture-review.md` into implementation-sized phases. The recommended direction is Option B: keep the working stack, but rebuild the domain pipeline around observations, evidence clusters, cautious hypotheses, confidence, alternatives, abstention, replay evaluation, and user-facing insight projections.
## Core Constraints
- Do not treat "smart money" as a canonical fact emitted by the system.
- Distinguish direct facts, evidence, hypotheses, confidence, alternatives, and abstention.
- Preserve evidence and uncertainty in storage, API, websocket, and UI surfaces.
- Keep Redis as hot cache only, not hidden baseline truth.
- Make replay evaluation the acceptance gate before expanding UI confidence.
- Keep historical or research-grade calibration as future work, not an MVP dependency.
## Phase Sequence
| Phase | Beads issue | Depends on | Purpose |
| --- | --- | --- | --- |
| 01 - Contracts and vocabulary | `islandflow-zxh.1` | `islandflow-259.1` | Define evidence/hypothesis/insight contracts and retire canonical overconfidence. |
| 02 - Evidence clustering and features | `islandflow-zxh.2` | `islandflow-259.2` | Extract eligibility, evidence facts, clusters, and traceable features. |
| 03 - Hypothesis scoring and abstention | `islandflow-zxh.3` | `islandflow-259.3` | Score cautious hypotheses and represent abstention/alternatives. |
| 04 - Replay evaluation and golden tests | `islandflow-zxh.4` | `islandflow-259.4` | Validate derived outputs through deterministic replay and golden fixtures. |
| 05 - API/UI explainability | `islandflow-zxh.5` | `islandflow-259.5` | Expose evidence-backed insights and uncertainty to API, WS, and UI. |
| 99 - Future calibration | `islandflow-zxh.6` | `islandflow-zxh.5`, `islandflow-259.6` | Calibrate confidence and policy behavior later with richer datasets. |
## PR Split Notes
Several phases are broad enough to split before implementation:
- `islandflow-zxh.2.1` - Split smart-flow phase 02a: eligibility and evidence facts
- `islandflow-zxh.2.2` - Split smart-flow phase 02b: clustering and feature vectors
- `islandflow-zxh.3.1` - Split smart-flow phase 03a: hypothesis score vectors
- `islandflow-zxh.3.2` - Split smart-flow phase 03b: abstention and insight projection
- `islandflow-zxh.5.1` - Split smart-flow phase 05a: evidence API and websocket surfaces
- `islandflow-zxh.5.2` - Split smart-flow phase 05b: UI explainability surfaces
If an implementation PR crosses contracts, compute, storage, API, and UI in one change, stop and split it.
## Matching Beads Epic
- `islandflow-zxh` - Plan smart-money to smart-flow implementation phases

View file

@ -0,0 +1,66 @@
# Smart-Flow Phase 01: Contracts and Vocabulary
## Purpose
Introduce the domain vocabulary and contracts that distinguish observations, evidence clusters, hypotheses, confidence, abstention, and user-facing insight projections.
## Why this phase comes now
The current system has useful infrastructure but overconfident domain names. Before changing classifier behavior, the codebase needs the language to express what is observed, what is inferred, what is uncertain, and when the system should abstain.
## Dependencies on earlier phases
- `islandflow-259.1` - Synthetic deterministic spine, so contract work can align with canonical raw event and provenance assumptions.
## Likely files/modules touched
- `packages/types/src/events.ts`
- Shared type exports in `packages/types/`
- Compatibility type aliases where legacy names are still needed
- Storage schema planning docs or migration notes
- Tests for schema parsing or event compatibility
## In-scope work
- Define or prepare contracts for `FlowEvidenceCluster`, `FlowCandidate`, `FlowHypothesisEvent`, `SmartFlowInsight`, `EvidenceQuality`, `BaselineSnapshot`, and version fields.
- Mark legacy "smart money" naming as compatibility or projection language, not canonical truth.
- Define how facts, evidence, hypotheses, scores, confidence, and abstention differ.
- Preserve compatibility aliases for existing API/UI paths where necessary.
- Add concise migration notes for future phases.
## Explicitly out-of-scope work
- Rewriting classifier scoring.
- Moving ingest policy.
- Adding new API endpoints or UI drawers.
- Building replay golden suites.
- Historical calibration or research-grade model fitting.
## Acceptance criteria
- Contracts distinguish observations, evidence, hypotheses, insight projections, confidence, alternatives, and abstention.
- Legacy naming remains only where compatibility requires it.
- Version fields are included for policy/model evolution.
- Future phases can refer to these contracts without redefining the vocabulary.
- Migration risk and compatibility aliases are documented.
## Test strategy
Use type-level checks and schema/serialization tests where practical. Add compatibility tests only for public contracts that must remain stable. Avoid broad behavior tests until evidence extraction and scoring phases exist.
## Risks / design traps
- Renaming everything without compatibility will break consumers.
- Keeping "smart money" as canonical language will preserve the old overconfidence.
- Mixing facts and hypotheses in one event shape will make replay evaluation weaker.
- Adding too many future fields can make contracts noisy before behavior exists.
## Suggested future Codex implementation prompt
```text
Implement docs/implementation/smart-money/01-contracts-vocabulary.md for Beads issue islandflow-zxh.1. Focus on contracts, vocabulary, version fields, and compatibility aliases only. Do not rewrite scoring, API/UI explainability, replay tests, or calibration.
```
## Matching Beads issue title/id
- `islandflow-zxh.1` - Smart-flow phase 01: contracts and vocabulary

View file

@ -0,0 +1,69 @@
# Smart-Flow Phase 02: Evidence Clustering and Features
## Purpose
Make evidence extraction, eligibility, quote/context joins, clustering, and feature construction explicit and traceable before hypothesis scoring changes.
## Why this phase comes now
Contracts alone do not change behavior. This phase gives the system a clean evidence layer so later scoring can reason from auditable facts instead of a generic feature bag or overconfident classifier labels.
## Dependencies on earlier phases
- `islandflow-zxh.1` - Smart-flow contracts and vocabulary
- `islandflow-259.2` - Synthetic manifests, fixtures, and CLI
## Likely files/modules touched
- `services/compute/src/`
- `packages/types/src/events.ts`
- `packages/storage/src/` for typed evidence storage planning or implementation
- Tests under `services/compute/tests/`
- Fixture helpers from the synthetic package
## In-scope work
- Represent direct observations, quote joins, execution context, and eligibility decisions as evidence facts.
- Build deterministic evidence clusters with traceable source refs.
- Compute feature vectors from evidence while preserving whether a value is observed, derived, or inferred.
- Carry evidence quality, stale quote, wide spread, odd lot, complex spread, and noisy context signals.
- Move toward ingest-as-normalization, not ingest-as-signal-policy.
## Explicitly out-of-scope work
- Final hypothesis score policy.
- API and UI explainability.
- Historical calibration.
- Claiming participant identity.
- Replacing all storage tables in the same PR.
## Acceptance criteria
- Eligibility decisions have explicit accept, reject, or down-weight reasons.
- Evidence clusters have deterministic keys/windows and preserve raw refs.
- Feature values trace back to evidence refs.
- Stale, wide, noisy, or ambiguous conditions can be represented without pretending to know intent.
- The phase is split into PR-sized children when implementation starts.
## Test strategy
Use deterministic fixtures from synthetic phase 02 where available. Add focused tests for quote joining, eligibility rejection, cluster key stability, feature derivation, and trace refs. Keep tests infra-free unless a later optional storage integration explicitly needs services.
## Risks / design traps
- Recreating the old `FlowPacket` as a renamed generic feature bag.
- Letting ingest services make signal-policy decisions.
- Losing evidence refs during aggregation.
- Treating cluster features as hypotheses before the scoring phase.
## Suggested future Codex implementation prompt
```text
Implement docs/implementation/smart-money/02-evidence-clustering-features.md for Beads issue islandflow-zxh.2. Use split issues islandflow-zxh.2.1 and islandflow-zxh.2.2 for PR-sized work. Focus on evidence facts, eligibility, clustering, and traceable features. Do not implement final scoring, API/UI explainability, or calibration.
```
## Matching Beads issue title/id
- `islandflow-zxh.2` - Smart-flow phase 02: evidence clustering and features
- PR split: `islandflow-zxh.2.1` - Split smart-flow phase 02a: eligibility and evidence facts
- PR split: `islandflow-zxh.2.2` - Split smart-flow phase 02b: clustering and feature vectors

View file

@ -0,0 +1,70 @@
# Smart-Flow Phase 03: Hypothesis Scoring and Abstention
## Purpose
Convert evidence clusters into cautious flow hypotheses with explicit score vectors, alternatives, penalties, confidence, conviction, and abstention reasons.
## Why this phase comes now
Scoring should wait until the system can represent evidence clearly and synthetic scenarios can describe expected positive, negative, and abstention cases. This phase is where the product stops acting like every signal is a confident "smart money" claim.
## Dependencies on earlier phases
- `islandflow-zxh.1` - Smart-flow contracts and vocabulary
- `islandflow-zxh.2` - Evidence clustering and features
- `islandflow-259.3` - Synthetic scenarios, labels, and expected outputs
## Likely files/modules touched
- `services/compute/src/`
- `packages/types/src/events.ts`
- `packages/storage/src/smart-money-events.ts` or successor storage modules
- Compute tests and fixture/golden comparison helpers
- Compatibility projection code for legacy alerts or classifier hits
## In-scope work
- Define score vectors for hypothesis type, direction, evidence strength, confidence, conviction, and penalties.
- Preserve alternative explanations and negative evidence.
- Make abstention a first-class output with reasons.
- Add policy/model version fields.
- Derive compatibility `SmartFlowInsight` or legacy projections from canonical hypothesis events.
## Explicitly out-of-scope work
- UI presentation overhaul.
- API endpoint expansion.
- Historical calibration.
- Participant identity claims.
- Tuning all thresholds against live historical data.
## Acceptance criteria
- Hypothesis scores separate evidence strength, confidence, conviction, and penalties.
- Abstention outputs include machine-readable and user-readable reasons.
- Alternative explanations are preserved.
- Compatibility projections do not become the canonical domain model.
- Score policy changes are deterministic against synthetic fixtures.
## Test strategy
Use synthetic scenario fixtures and expected-output manifests. Cover positive hypotheses, abstentions, false-positive suppressions, alternative explanations, and noisy scenarios. Keep output comparisons stable and focused on score signatures rather than brittle full payload dumps.
## Risks / design traps
- Rebranding old classifier hits as hypotheses without changing semantics.
- Treating confidence as probability when it is only policy confidence.
- Hiding abstention in logs instead of output events.
- Letting compatibility alert projections dictate canonical scoring design.
## Suggested future Codex implementation prompt
```text
Implement docs/implementation/smart-money/03-hypothesis-scoring-abstention.md for Beads issue islandflow-zxh.3. Use split issues islandflow-zxh.3.1 and islandflow-zxh.3.2 for PR-sized work. Build cautious hypothesis scoring, alternatives, and abstention from evidence clusters. Do not add API/UI explainability or historical calibration.
```
## Matching Beads issue title/id
- `islandflow-zxh.3` - Smart-flow phase 03: hypothesis scoring and abstention
- PR split: `islandflow-zxh.3.1` - Split smart-flow phase 03a: hypothesis score vectors
- PR split: `islandflow-zxh.3.2` - Split smart-flow phase 03b: abstention and insight projection

View file

@ -0,0 +1,69 @@
# Smart-Flow Phase 04: Replay Evaluation and Golden Tests
## Purpose
Make deterministic replay and golden output comparison the acceptance gate for smart-flow behavior changes.
## Why this phase comes now
Replay evaluation should come after synthetic replay can select stable runs and after hypothesis scoring has outputs worth validating. This phase turns architecture discipline into a repeatable test path.
## Dependencies on earlier phases
- `islandflow-zxh.1` - Smart-flow contracts and vocabulary
- `islandflow-zxh.2` - Evidence clustering and features
- `islandflow-zxh.3` - Hypothesis scoring and abstention
- `islandflow-259.4` - Synthetic replay integration
## Likely files/modules touched
- `services/replay/src/`
- `services/compute/tests/`
- Synthetic fixture and manifest comparison helpers
- Golden fixture directories
- Optional service-container integration config if added later
## In-scope work
- Recompute derived evidence/hypothesis outputs from raw synthetic streams.
- Compare stable output signatures with expected manifests.
- Include positive, abstention, false-positive, and noisy scenarios.
- Make replay/golden tests deterministic and infra-free by default.
- Gate optional ClickHouse/NATS/Redis tests outside the default path.
## Explicitly out-of-scope work
- New scoring policy beyond fixes needed for deterministic evaluation.
- UI explainability.
- Historical calibration.
- Large generated fixture dumps.
- Making Docker-backed tests mandatory.
## Acceptance criteria
- Replay recomputes derived smart-flow outputs from raw fixtures.
- Golden signatures cover positive, abstain, false-positive, and noisy scenarios.
- Default tests are deterministic and infra-free.
- Optional service-backed tests are clearly gated.
- Failures show concise, reviewable diffs or signature mismatches.
## Test strategy
Use fixture-backed replay and compact golden signatures first. Add a small number of representative scenarios rather than broad generated dumps. If service-backed tests are added, mark them optional and document their dependencies.
## Risks / design traps
- Golden files that are too large will become rubber-stamped.
- Full payload comparisons may break on harmless metadata changes.
- Optional infra tests can accidentally become required in CI.
- Replay that starts from derived events instead of raw fixtures will miss pipeline regressions.
## Suggested future Codex implementation prompt
```text
Implement docs/implementation/smart-money/04-replay-evaluation-golden-tests.md for Beads issue islandflow-zxh.4. Build deterministic replay/golden validation from raw synthetic fixtures. Keep default tests infra-free, compare stable signatures, and do not add UI explainability or historical calibration.
```
## Matching Beads issue title/id
- `islandflow-zxh.4` - Smart-flow phase 04: replay evaluation and golden tests

View file

@ -0,0 +1,72 @@
# Smart-Flow Phase 05: API/UI Explainability
## Purpose
Expose evidence-backed smart-flow outputs through API, websocket, and UI surfaces that make evidence quality, confidence, conviction, alternatives, and abstention understandable.
## Why this phase comes now
The presentation layer should wait until contracts, evidence, scoring, and replay evaluation are stable. Otherwise the UI will harden old overconfident language or teach users to trust unvalidated outputs.
## Dependencies on earlier phases
- `islandflow-zxh.1` - Smart-flow contracts and vocabulary
- `islandflow-zxh.2` - Evidence clustering and features
- `islandflow-zxh.3` - Hypothesis scoring and abstention
- `islandflow-zxh.4` - Replay evaluation and golden tests
- `islandflow-259.5` - Synthetic demo and load profiles
## Likely files/modules touched
- `services/api/src/`
- Websocket payload types and channel names
- `apps/web/`
- Shared UI/domain types in `packages/types/`
- API and UI tests
## In-scope work
- Add or alias API/WS surfaces for evidence, hypotheses, insights, alternatives, and abstention.
- Keep legacy smart-money endpoints as aliases where needed, not canonical contracts.
- Rework UI surfaces around evidence quality, confidence versus conviction, alternatives, abstention, and why-not context.
- Ensure named deterministic demos can display stable explainability examples.
- Keep replay/golden validation tied to changed projections.
## Explicitly out-of-scope work
- Rewriting scoring policy.
- Adding new synthetic foundations.
- Historical calibration.
- Claiming participant identity.
- UI copy that implies certainty where the model only has evidence-backed hypotheses.
## Acceptance criteria
- API/WS payloads expose evidence refs, hypotheses, insights, alternatives, abstention reasons, and version fields.
- UI distinguishes evidence quality, confidence, conviction, and why-not signals.
- Legacy smart-money surfaces remain compatibility aliases where required.
- Replay/golden checks support changed projection behavior.
- Explainability copy avoids overconfident certainty claims.
## Test strategy
Use API contract tests, websocket payload tests, and focused UI tests for evidence/abstention rendering. Validate with deterministic demo runs from synthetic phase 05. Manual visual review should supplement, not replace, replay/golden validation.
## Risks / design traps
- UI can accidentally reintroduce "smart money" certainty.
- API aliases can become de facto canonical if not documented.
- Too many fields without hierarchy will make explainability harder to scan.
- Building UI before replay validation can make demos persuasive but untrustworthy.
## Suggested future Codex implementation prompt
```text
Implement docs/implementation/smart-money/05-api-ui-explainability.md for Beads issue islandflow-zxh.5. Use split issues islandflow-zxh.5.1 and islandflow-zxh.5.2 for PR-sized work. Expose evidence-backed API/WS/UI explainability after replay/golden validation. Do not change core scoring or add calibration.
```
## Matching Beads issue title/id
- `islandflow-zxh.5` - Smart-flow phase 05: API/UI explainability
- PR split: `islandflow-zxh.5.1` - Split smart-flow phase 05a: evidence API and websocket surfaces
- PR split: `islandflow-zxh.5.2` - Split smart-flow phase 05b: UI explainability surfaces

View file

@ -0,0 +1,65 @@
# Smart-Flow Phase 99: Future Calibration
## Purpose
Plan future calibration of smart-flow confidence, policy thresholds, penalties, and abstention behavior after the MVP evidence/hypothesis pipeline is working and replay-validated.
## Why this phase comes now
The architecture should leave room for calibration, but calibration should not block the MVP. The system first needs clean facts, evidence, hypotheses, and replayable evaluation before tuning can be meaningful.
## Dependencies on earlier phases
- `islandflow-zxh.5` - Smart-flow API/UI explainability
- `islandflow-259.6` - Future synthetic historical calibration
## Likely files/modules touched
- Future calibration tooling in `services/compute/` or a research package
- Policy/model version registry
- Evaluation reports or benchmark datasets
- Storage/query helpers for historical derived outputs
- Documentation for metrics and calibration governance
## In-scope work
- Define calibration datasets and evaluation metrics.
- Specify how confidence, conviction, penalties, abstention, and alternatives are tuned.
- Preserve policy/model versioning and replayability.
- Document what makes a calibration dataset acceptable.
- Keep user-facing confidence semantics auditable.
## Explicitly out-of-scope work
- MVP contracts and scoring foundations.
- API/UI explainability for the initial pipeline.
- Treating historical calibration as proof of participant identity.
- Using private or licensed data in committed fixtures without approval.
## Acceptance criteria
- Calibration remains outside the MVP blocker chain.
- Dataset provenance, metrics, and policy versioning are documented before implementation.
- Confidence and abstention semantics remain explainable after tuning.
- Replay can compare calibrated policy versions without losing auditability.
## Test strategy
When implemented, use replayed benchmark datasets with versioned policy outputs. Track false positives, abstentions, precision-like metrics, and scenario-specific regressions. Keep calibration tests separate from the early deterministic fixture tests.
## Risks / design traps
- Treating calibrated confidence as objective truth.
- Tuning to demos instead of representative market regimes.
- Losing policy version lineage.
- Committing restricted data or large generated benchmark artifacts.
## Suggested future Codex implementation prompt
```text
Implement docs/implementation/smart-money/99-future-calibration.md for Beads issue islandflow-zxh.6 only after the MVP smart-flow phases are complete. Define calibration datasets, metrics, policy versioning, and replay comparison. Do not make calibration a prerequisite for earlier evidence, scoring, or UI work.
```
## Matching Beads issue title/id
- `islandflow-zxh.6` - Future smart-flow phase 99: calibration

View file

@ -0,0 +1,36 @@
# Synthetic Market-Data Roadmap
This roadmap breaks `docs/plans/synthetic-market-data-architecture-review.md` into implementation-sized phases. The recommended direction is still Option B: extract deterministic synthetic generation into a first-class reusable engine while keeping the useful NATS, ClickHouse, compute, API, replay, and web stack.
## Core Constraints
- Emit canonical market event types: `OptionPrint`, `OptionNBBO`, `EquityPrint`, and `EquityQuote`.
- Do not create synthetic-only market event types for the main pipeline.
- Keep hidden ground-truth labels separate from emitted market events.
- Keep early quality gates infra-free: `bun test` should not require Docker, ClickHouse, NATS, or Redis.
- Build deterministic foundations before demos, UI controls, or live synthetic service behavior.
- Treat historical calibration as future work, not as a dependency for the MVP synthetic generator.
## Phase Sequence
| Phase | Beads issue | Depends on | Purpose |
| --- | --- | --- | --- |
| 01 - Deterministic spine | `islandflow-259.1` | None | Create the seeded generation foundation and canonical event output contract. |
| 02 - Manifests, fixtures, CLI | `islandflow-259.2` | `islandflow-zxh.1` | Turn deterministic generation into durable fixtures and manifests. |
| 03 - Scenarios, labels, expected outputs | `islandflow-259.3` | `islandflow-zxh.2` | Author named scenarios, separate labels, and expected derived outputs. |
| 04 - Replay integration | `islandflow-259.4` | `islandflow-zxh.3` | Make replay consume synthetic runs with stable ordering and output comparison. |
| 05 - Demo and load profiles | `islandflow-259.5` | `islandflow-zxh.4` | Expose named deterministic demo/load profiles after replay validation. |
| 99 - Future historical calibration | `islandflow-259.6` | `islandflow-259.5` | Calibrate parameters from historical data later, after the MVP is stable. |
## PR Split Notes
Most phases are intended to fit in one focused PR. Phase 03 is already split into PR-sized Beads children because scenario authoring and expected-output comparison can grow quickly:
- `islandflow-259.3.1` - Split synthetic phase 03a: scenario catalog and labels
- `islandflow-259.3.2` - Split synthetic phase 03b: expected-output manifests
If any other phase starts touching unrelated service, API, UI, and storage behavior in one PR, split it before implementation continues.
## Matching Beads Epic
- `islandflow-259` - Plan synthetic market-data implementation phases

View file

@ -0,0 +1,68 @@
# Synthetic Market-Data Phase 01: Deterministic Spine
## Purpose
Create the reusable deterministic foundation for synthetic market data. This phase should define the package/API shape for seeded generation, stable run identity, profile inputs, canonical event outputs, and provenance metadata.
## Why this phase comes now
Everything else depends on reproducible raw events. Manifests, labels, replay, demos, and smart-flow tests are only trustworthy if the same seed/profile bundle produces the same canonical market event stream every time.
## Dependencies on earlier phases
None. This is the first synthetic phase.
## Likely files/modules touched
- Future `packages/synthetic-market/` workspace or equivalent package boundary
- `packages/types/src/events.ts`
- Synthetic logic currently embedded in `services/ingest-options/` and `services/ingest-equities/`
- Shared package manifests such as `package.json`, `bunfig.toml`, or workspace config if a new package is added
- Infra-free unit tests under the new package or nearby package test folders
## In-scope work
- Define `SyntheticRun`, `SeedBundle`, `ParameterSnapshot`, `SymbolProfile`, `LiquidityProfile`, `VolatilityRegime`, `OptionChainProfile`, and `GeneratedEventBatch` shapes.
- Pick and wrap a deterministic PRNG so fixed inputs produce stable output.
- Emit canonical `OptionPrint`, `OptionNBBO`, `EquityPrint`, and `EquityQuote` events.
- Attach provenance such as `source_kind`, `run_id`, `parameter_snapshot_hash`, and optional `scenario_id`.
- Preserve compatibility with the existing pipeline's raw market event contracts.
- Add fast deterministic tests that run in plain `bun test`.
## Explicitly out-of-scope work
- Scenario catalogs and ground-truth label records.
- Manifest generation and CLI workflows.
- Replay service integration.
- Hosted demo controls or live synthetic emitters.
- Historical calibration from real market data.
- Docker, ClickHouse, NATS, or Redis integration tests.
## Acceptance criteria
- A fixed seed/profile bundle produces byte-stable or hash-stable event output.
- Generated events use canonical market event contracts, not synthetic-only pipeline event types.
- Hidden labels are not embedded in emitted market events.
- Provenance metadata is available for downstream filtering and auditing.
- Tests cover determinism, tick validity, quote/trade invariants, and basic profile normalization without requiring infrastructure.
## Test strategy
Use infra-free Bun tests. Cover PRNG repeatability, profile parsing, event ordering within generated batches, option quote/print validity, equity quote/print validity, and provenance field stability. Avoid any test that needs Docker, ClickHouse, NATS, or Redis.
## Risks / design traps
- Hiding wall-clock timers or random calls inside the generator will break determinism.
- Creating synthetic-only market event types will fork the pipeline contract.
- Embedding labels directly on market events will leak ground truth into production-like paths.
- Over-designing a full market simulator now will slow down the MVP.
## Suggested future Codex implementation prompt
```text
Implement docs/implementation/synthetic-market-data/01-deterministic-spine.md for Beads issue islandflow-259.1. Stay inside the deterministic synthetic market-data foundation only. Do not add scenario labels, manifests, replay integration, demos, or historical calibration. Emit canonical market event types and keep early tests infra-free.
```
## Matching Beads issue title/id
- `islandflow-259.1` - Synthetic market-data phase 01: deterministic spine

View file

@ -0,0 +1,68 @@
# Synthetic Market-Data Phase 02: Manifests, Fixtures, and CLI
## Purpose
Turn the deterministic generator into reusable artifacts: fixture files, run manifests, and a CLI that can produce repeatable synthetic runs for tests, replay, demos, and later evaluation.
## Why this phase comes now
The deterministic spine gives the repo stable raw events. The next step is to make those events durable and addressable so downstream phases can reference exact generated runs instead of recreating ad hoc local randomness.
## Dependencies on earlier phases
- `islandflow-259.1` - Synthetic deterministic spine
- `islandflow-zxh.1` - Smart-flow contracts and vocabulary, so manifest expectations can align with the emerging evidence/hypothesis language
## Likely files/modules touched
- Future `packages/synthetic-market/` CLI entrypoints
- Fixture directories under a package or service test area
- Manifest schemas, likely JSON or YAML
- `package.json` scripts if a repo command is added
- Tests for manifest parsing and fixture generation
## In-scope work
- Define `ExpectedOutputManifest`, `ReplayPlan`, and generated fixture artifact layout.
- Add a CLI command that accepts seed bundle, profile, scenario/run name, output directory, and deterministic generation options.
- Write manifests that pin generator version, seed bundle, parameter snapshot hash, generated event hashes, replay ordering, and run metadata.
- Add fixture helpers for tests to load generated batches without infrastructure.
- Keep labels as separate records or future manifest sections, not market-event fields.
## Explicitly out-of-scope work
- Full scenario catalog authoring.
- Smart-flow expected output comparisons.
- Replay service source selection.
- ClickHouse fixture materialization.
- UI demo selection.
- Historical calibration.
## Acceptance criteria
- A CLI can generate repeatable fixtures and manifests from fixed inputs.
- Manifests include generator version, seed/profile identity, parameter hash, event hashes, and replay ordering.
- Fixture helpers can load generated event batches in infra-free tests.
- Generated artifacts do not embed hidden labels into canonical market events.
- Re-running generation with the same inputs produces stable manifests or an intentional diff.
## Test strategy
Use plain Bun tests for CLI argument parsing, manifest schema parsing, deterministic fixture output, and fixture-loader helpers. Golden files should be small and intentionally reviewed. Do not require Docker, ClickHouse, NATS, or Redis.
## Risks / design traps
- Manifests that omit generator version or parameter hashes will become hard to audit.
- Large generated fixtures can create noisy reviews; keep early fixtures tiny.
- A CLI that silently uses defaults will make tests look deterministic while hiding input drift.
- Mixing expected smart-flow outputs too early can couple this phase to unfinished classifier changes.
## Suggested future Codex implementation prompt
```text
Implement docs/implementation/synthetic-market-data/02-manifests-fixtures-cli.md for Beads issue islandflow-259.2. Build manifest, fixture, and CLI support on top of the deterministic spine. Keep tests infra-free and do not implement scenario labels, replay integration, demo profiles, or historical calibration.
```
## Matching Beads issue title/id
- `islandflow-259.2` - Synthetic market-data phase 02: manifests, fixtures, and CLI

View file

@ -0,0 +1,71 @@
# Synthetic Market-Data Phase 03: Scenarios, Labels, and Expected Outputs
## Purpose
Author named deterministic scenarios, separate ground-truth labels, and expected-output manifests that downstream smart-flow logic can use for positive, negative, abstention, and false-positive validation.
## Why this phase comes now
The generator and manifest layers should exist before scenario authoring. Smart-flow evidence clustering should also define enough vocabulary for expected outputs to describe evidence requirements without leaking labels into emitted market events.
## Dependencies on earlier phases
- `islandflow-259.1` - Synthetic deterministic spine
- `islandflow-zxh.1` - Smart-flow contracts and vocabulary
- `islandflow-259.2` - Manifests, fixtures, and CLI
- `islandflow-zxh.2` - Evidence clustering and features
## Likely files/modules touched
- Future scenario catalog files under `packages/synthetic-market/`
- Label schema definitions
- Manifest expected-output sections
- Fixture generation tests
- Smart-flow fixture expectations in compute test areas, once available
## In-scope work
- Define `ScenarioInjection` and `GroundTruthLabel` records.
- Add named scenario profiles for institutional directional flow, retail-attention flow, event/noise flow, volatility-seller behavior, hedge-reactive flow, arbitrage-like structure, and no-alert negatives.
- Keep labels keyed by `run_id`, `scenario_id`, event IDs or trace IDs, expected class, expected direction, confidence band, required evidence, forbidden evidence, and false-positive penalties.
- Extend manifests with expected derived events, alert/no-alert expectations, and evidence requirements.
- Make generated scenario outputs reviewable and deterministic.
## Explicitly out-of-scope work
- Emitting labels on market events.
- Building a live synthetic service.
- Adding UI scenario controls.
- Implementing historical calibration.
- Rewriting smart-flow scoring behavior beyond what is needed to express expected outputs.
## Acceptance criteria
- Scenario fixtures are named, deterministic, and small enough for review.
- Labels remain separate from emitted market events.
- Expected-output manifests include positive expectations, no-alert expectations, evidence requirements, forbidden evidence, and false-positive penalties.
- The phase can test both "should detect" and "should abstain or suppress" cases.
- Existing issue `islandflow-9dz` is treated as related scenario-tuning context, not as the broad phase tracker.
## Test strategy
Use fixture-generation and manifest-validation tests first. Add focused golden comparisons only where the smart-flow contract is ready. Keep the default test path infra-free. Optional service-backed scenario loading can wait for a later integration phase.
## Risks / design traps
- Labels leaking into canonical event payloads will invalidate evaluation.
- Only authoring positive scenarios will make the classifier overfit demos.
- Broad scenario catalogs can become too large for one PR.
- Expected outputs that name legacy "smart money" certainty can undermine the new evidence/hypothesis model.
## Suggested future Codex implementation prompt
```text
Implement docs/implementation/synthetic-market-data/03-scenarios-labels-expected-outputs.md for Beads issue islandflow-259.3. Split the work using islandflow-259.3.1 and islandflow-259.3.2 if needed. Keep labels separate from emitted events, include negative/no-alert expectations, and avoid demos or live service work.
```
## Matching Beads issue title/id
- `islandflow-259.3` - Synthetic market-data phase 03: scenarios, labels, and expected outputs
- PR split: `islandflow-259.3.1` - Split synthetic phase 03a: scenario catalog and labels
- PR split: `islandflow-259.3.2` - Split synthetic phase 03b: expected-output manifests

View file

@ -0,0 +1,69 @@
# Synthetic Market-Data Phase 04: Replay Integration
## Purpose
Make replay consume synthetic runs deterministically, either directly from generated fixtures or from materialized storage rows, while preserving the same ordering semantics the real replay path uses.
## Why this phase comes now
Replay should not be wired to synthetic data until the generator, manifests, labels, and smart-flow hypothesis pipeline have stable semantics. At this point, replay can become a serious acceptance gate instead of a demo convenience.
## Dependencies on earlier phases
- `islandflow-259.1` - Synthetic deterministic spine
- `islandflow-259.2` - Manifests, fixtures, and CLI
- `islandflow-259.3` - Scenarios, labels, and expected outputs
- `islandflow-zxh.3` - Hypothesis scoring and abstention
## Likely files/modules touched
- `services/replay/src/`
- API replay routes in `services/api/`
- Replay-related shared types in `packages/types/`
- Optional fixture materialization helpers in `packages/storage/`
- Replay tests or golden comparison helpers
## In-scope work
- Add replay source/run selectors for synthetic runs.
- Support fixture-backed replay without infrastructure where practical.
- Preserve ordering by event time, ingest time, sequence, and stable event ID.
- Compare replayed derived outputs against manifest signatures or expected-output sections.
- Keep optional ClickHouse/NATS materialized replay tests behind non-default gates.
## Explicitly out-of-scope work
- Building new scenario labels.
- Reworking smart-flow scoring policy.
- Demo profile controls.
- Load testing.
- Historical calibration.
## Acceptance criteria
- Replay can select a synthetic source and `run_id`.
- Fixture-backed replay respects manifest ordering.
- Derived output signatures can be compared with expected manifests.
- Fast replay tests remain infra-free by default.
- Optional infra-backed tests are clearly named and gated.
## Test strategy
Start with fixture-backed replay ordering tests and manifest-signature comparisons. Add optional service-container or ClickHouse materialization tests only after the fast path is stable, and do not make those tests part of the default `bun test` requirement.
## Risks / design traps
- Creating a synthetic-only replay path with different ordering will hide bugs.
- Letting optional infra tests become default will slow or destabilize CI.
- Comparing full raw payloads everywhere may make tests brittle; use stable signatures where better.
- Replay selectors that are not run-scoped can mix synthetic and live data.
## Suggested future Codex implementation prompt
```text
Implement docs/implementation/synthetic-market-data/04-replay-integration.md for Beads issue islandflow-259.4. Add synthetic source/run replay support with stable ordering and manifest comparison. Do not add demo controls, load profiles, or historical calibration, and keep the fast test path infra-free.
```
## Matching Beads issue title/id
- `islandflow-259.4` - Synthetic market-data phase 04: replay integration

View file

@ -0,0 +1,70 @@
# Synthetic Market-Data Phase 05: Demo and Load Profiles
## Purpose
Expose deterministic synthetic runs as named demo and load profiles after the generation, manifest, scenario, and replay foundations are in place.
## Why this phase comes now
Demos are useful only after the underlying data can be trusted. This phase deliberately waits until replay and golden evaluation prove the event semantics, so hosted controls do not become a front door to ambient randomness.
## Dependencies on earlier phases
- `islandflow-259.1` - Synthetic deterministic spine
- `islandflow-259.2` - Manifests, fixtures, and CLI
- `islandflow-259.3` - Scenarios, labels, and expected outputs
- `islandflow-259.4` - Replay integration
- `islandflow-zxh.4` - Smart-flow replay evaluation and golden tests
## Likely files/modules touched
- Thin synthetic emitters in `services/ingest-options/` and `services/ingest-equities/`
- Demo/run selection API surfaces in `services/api/`
- Web demo controls in `apps/web/`
- Load profile definitions in the synthetic package
- Tests for profile selection and rate scaling
## In-scope work
- Add named `DemoProfile` and `LoadProfile` definitions.
- Make live/demo emitters thin consumers of deterministic synthetic runs.
- Let demo controls select named runs/scenarios rather than changing hidden random behavior.
- Ensure load profiles scale event rates without changing event semantics.
- Document local demo usage once implemented.
## Explicitly out-of-scope work
- Foundation generator work.
- New smart-flow scoring policy.
- Replacing replay evaluation with UI-only checks.
- Historical calibration.
- Production provider configuration decisions.
## Acceptance criteria
- Demo profiles are deterministic and named.
- Load profiles scale rate or volume without mutating scenario semantics.
- Hosted or local controls select known runs/scenarios.
- Live/demo emitters remain thin and do not own generator policy.
- The UI does not expose synthetic controls before the backing deterministic runs exist.
## Test strategy
Use unit tests for profile parsing, profile selection, and rate-scaling semantics. Add replay-driven smoke checks for named demo runs. Manual UI validation is appropriate only after automated replay/golden checks pass.
## Risks / design traps
- Demo controls can pressure the codebase back into wall-clock randomness.
- Load profiles may accidentally change business semantics while changing only rate was intended.
- UI-first implementation can hide missing run provenance.
- Reusing production config for synthetic demos can make operator behavior ambiguous.
## Suggested future Codex implementation prompt
```text
Implement docs/implementation/synthetic-market-data/05-demo-load-profiles.md for Beads issue islandflow-259.5. Add named deterministic demo/load profiles and thin emitter/control integration only after replay validation exists. Do not implement historical calibration or change production provider policy.
```
## Matching Beads issue title/id
- `islandflow-259.5` - Synthetic market-data phase 05: demo and load profiles

View file

@ -0,0 +1,64 @@
# Synthetic Market-Data Phase 99: Future Historical Calibration
## Purpose
Plan future calibration of synthetic generator parameters from historical market data without making historical data a dependency for the MVP generator.
## Why this phase comes now
It is useful to name the future work now so early designs keep calibration hooks in mind. It should not come before deterministic generation, manifests, scenarios, replay, or demo profiles.
## Dependencies on earlier phases
- `islandflow-259.5` - Synthetic demo and load profiles
## Likely files/modules touched
- Future calibration tools under the synthetic package
- Historical data import or sampling utilities
- Parameter fitting scripts
- Documentation for data provenance and licensing constraints
- Optional research notebooks or reports if the repo later adopts them
## In-scope work
- Define calibration datasets and constraints.
- Specify how historical distributions map to `ParameterSnapshot`, liquidity, volatility, and option-chain profiles.
- Preserve deterministic replay from calibrated parameters.
- Document privacy, licensing, and provenance requirements for historical data.
## Explicitly out-of-scope work
- MVP synthetic generator requirements.
- Early tests and fixture generation.
- Live synthetic demos.
- Smart-flow scoring changes.
- Any assumption that historical data is needed to start implementation.
## Acceptance criteria
- Historical calibration remains outside the MVP blocker chain.
- Calibration inputs and ownership constraints are documented before implementation.
- Fitted parameters can still be pinned into deterministic seed/profile bundles.
- Calibration does not require emitted synthetic events to diverge from canonical market event contracts.
## Test strategy
When this future phase is implemented, use small public or licensed calibration samples with deterministic parameter fitting tests. Add regression checks that calibrated profiles still produce stable manifests. Do not retrofit historical data into earlier infra-free tests.
## Risks / design traps
- Treating calibration as necessary for MVP will delay foundational work.
- Historical data licensing can constrain what can be committed or shared.
- Overfitting synthetic profiles to a tiny period can produce misleading demos.
- Calibration tools can accidentally leak proprietary or sensitive data into fixtures.
## Suggested future Codex implementation prompt
```text
Implement docs/implementation/synthetic-market-data/99-future-historical-calibration.md for Beads issue islandflow-259.6 only after MVP synthetic phases are complete. Keep calibration optional, documented, and deterministic. Do not make historical data a dependency for earlier synthetic tests or demos.
```
## Matching Beads issue title/id
- `islandflow-259.6` - Future synthetic market-data phase 99: historical calibration

View file

@ -0,0 +1,135 @@
# Architecture Review: Evidence-Backed Smart-Flow Detection
## Summary
No source code was modified. The current architecture is **not suitable as-is**, but it is **close enough to refactor, not rewrite**. The stack is right; the domain language and pipeline shape are not.
Research direction: direct observation → inference → hypothesis, with preserved evidence and visible uncertainty. See [smart-flow-market-mechanics.md](/Users/kell/dev/islandflow/docs/research-docs/smart-flow-market-mechanics.md:7).
Key code evidence: `FlowPacket` is a generic feature bag in [events.ts](/Users/kell/dev/islandflow/packages/types/src/events.ts:193), `SmartMoneyEvent` already has useful score/abstention fields in [events.ts](/Users/kell/dev/islandflow/packages/types/src/events.ts:283), compute emits smart-money events then compatibility hits/alerts in [index.ts](/Users/kell/dev/islandflow/services/compute/src/index.ts:1086), storage keeps core hypothesis detail as JSON in [smart-money-events.ts](/Users/kell/dev/islandflow/packages/storage/src/smart-money-events.ts:24), and replay currently replays raw market streams rather than validating the whole derived pipeline in [replay/index.ts](/Users/kell/dev/islandflow/services/replay/src/index.ts:69).
## Area Classification
| Area | Call | Architecture Review |
|---|---:|---|
| Domain model | **refactor** | Good bones, wrong center. Make evidence, hypotheses, scores, and alternatives first-class. |
| Event taxonomy | **refactor** | Raw/derived split is good; `smart_money`, `dark.inferred`, and `classifier_hits` leak overconfident product language. |
| Service boundaries | **refactor** | Ingest does too much signal policy; compute is too broad. Split pipeline stages before adding more intelligence. |
| `FlowPacket` | **refactor** | Keep concept, rename/reframe as `FlowEvidenceCluster` or `FlowCandidate`. Not a product domain object. |
| `SmartMoneyEvent` | **redesign** | Replace canonical object with `FlowHypothesisEvent`; use `SmartFlowInsight` only as UI/API projection. |
| Classifier pipeline | **redesign** | Current rules mix evidence extraction, hypothesis scoring, narrative labels, and alerting. Needs staged outputs. |
| ClickHouse/storage | **refactor** | Right datastore; raw tables are decent, derived evidence/hypotheses need typed/queryable columns plus JSON sidecars. |
| Redis baselines/cache | **refactor** | Right hot-state role; wrong as hidden baseline truth. Baselines need replayable snapshots/versioning. |
| NATS/JetStream subjects | **refactor** | Right bus; subjects should express stage/version: observations, evidence, hypotheses, insights. |
| Replay determinism | **redesign** | Present but not central enough. Replay must be the acceptance gate for derived outputs. |
| API/WebSocket | **refactor** | Mechanics are good; public surface should expose evidence bundles and hypotheses, not internal legacy names. |
| UI evidence model | **refactor** | Directionally good, but still foregrounds “profile/probability” over evidence quality, alternatives, and uncertainty. |
| Test strategy | **redesign** | Unit tests are solid scaffolding; needs fixture replay, false-positive suites, calibration, and end-to-end determinism. |
## Direct Answers
1. **Current suitability:** no. Useful infrastructure, but not yet an evidence-backed smart-flow architecture.
2. **`SmartMoneyEvent`:** not a good canonical domain object. Use **`FlowHypothesisEvent`**. `ParticipantHypothesisEvent` implies participant identity too strongly. `SmartFlowInsight` should be a user-facing projection.
3. **`FlowPacket`:** not as named. Keep the abstraction as an internal evidence cluster, rename to `FlowEvidenceCluster` or `FlowCandidate`.
4. **Service boundaries:** not right. Ingest should normalize only; evidence quality, eligibility, clustering, hypothesis scoring, and insight projection should be separate stages.
5. **ClickHouse/Redis/NATS roles:** yes broadly. ClickHouse = authoritative event/audit store. Redis = hot cache only. NATS = transport, not truth. All three need cleaner contracts.
6. **Replay central enough:** no. It should be how every detection change proves itself.
7. **UI uncertainty:** partially. It shows evidence refs, profile ladders, abstention, and suppression, but needs confidence vs conviction, alternative explanations, evidence quality, and “why not” signals.
8. **First-class domain objects:** raw observations, execution context, quote join, eligibility decision, evidence cluster, structure hypothesis, evidence quality score, baseline snapshot, hypothesis score vector, false-positive penalty, catalyst context, flow hypothesis event, smart-flow insight, replay run.
9. **Implementation details:** Redis list layout, durable consumer names, current classifier thresholds, ClickHouse batch writer, adapter internals, legacy `ClassifierHitEvent`, alert severity math, UI cache mechanics.
10. **Delete/defer:** canonical “smart money” naming, real-time dark-pool certainty, standalone whale-premium alerts, trade-level open/close claims, participant identity claims, simplistic premium alert score, ingest-time signal filtering, `retail_whale` as a canonical profile unless reframed as attention/lottery flow.
## Option A — Conservative
Summary: keep current objects and services; add evidence-quality fields, UI copy fixes, and replay tests.
Pros: fastest, lowest migration risk, preserves current endpoints and UI.
Cons: leaves misleading canonical names; makes future research harder; keeps inference tangled inside current compute flow.
Complexity: low. Migration risk: low.
Better: less overconfidence, more visible suppression, quicker validation.
Worse: domain debt remains; `SmartMoneyEvent` becomes harder to undo later.
Likely kept: most code in `services/compute`, `packages/types`, `packages/storage`, API routes, UI panes.
Likely rewritten: alert scoring, UI labels, some profile fields.
Likely deleted: almost nothing.
PR sequence:
1. Rename UI copy from “Smart money” to “Smart flow candidate.”
2. Add evidence-quality and alternative-explanation fields to existing event.
3. Add replay consistency tests around current outputs.
4. Add typed ClickHouse columns for high-value JSON fields.
5. Deprecate, but do not remove, legacy classifier hit display.
## Option B — Refactor
Summary: keep Bun/TS, NATS, ClickHouse, Redis, API/WS, and the terminal UI, but rebuild the domain pipeline around evidence clusters and hypothesis events.
Pros: fixes the products epistemic spine without wasting useful infrastructure; best fit for pre-alpha.
Cons: breaking contract migration; touches types, storage, compute, API, UI, and tests.
Complexity: medium-high. Migration risk: medium.
Better: replayability, auditability, naming, evidence display, calibration, and future research velocity.
Worse: more short-term churn; old demos and endpoints need compatibility aliases.
Likely kept: raw market schemas, adapters, NATS/ClickHouse/Redis clients, live socket mechanics, virtualized UI, replay service skeleton, many feature calculations.
Likely rewritten: `SmartMoneyEvent`, `FlowPacket`, classifier pipeline, alert projection, ClickHouse derived schemas, API channel names, UI evidence drawers.
Likely deleted: canonical `smart_money` naming, ingest signal policy, premium-heavy alert scoring, `ClassifierHitEvent` as primary domain surface.
PR sequence:
1. Introduce `FlowEvidenceCluster`, `FlowHypothesisEvent`, `SmartFlowInsight`, `EvidenceQuality`, and version fields; keep aliases for compatibility.
2. Move signal eligibility out of ingest; ingest publishes normalized observations plus execution context only.
3. Split compute internally into evidence join → cluster/structure → hypothesis scoring → insight/alert projection.
4. Replace derived JSON-only storage with typed query columns for evidence quality, hypothesis scores, model version, policy version, and refs.
5. Add replay-run harness that recomputes derived outputs from raw streams and compares signatures.
6. Add `/flow/evidence`, `/flow/hypotheses`, `/flow/insights` plus WS equivalents; keep legacy endpoints as aliases.
7. Rework UI drawers/tables around evidence quality, confidence vs conviction, alternatives, abstention, and catalyst/noise context.
8. Add fixture suites for stale quotes, complex spreads, 0DTE/event noise, deep ITM, wide spreads, and off-exchange ambiguity.
## Option C — Redesign
Summary: if starting over, build an event-sourced evidence engine with raw observations as the only source of truth and every derived artifact generated by versioned, replayable policies.
Pros: cleanest long-term architecture; strongest research discipline; easiest calibration/backtesting story.
Cons: slowest; overkill before product fit; discards too much working terminal and streaming infrastructure.
Complexity: very high. Migration risk: high.
Better: clean contracts, model versioning, deterministic replay, research-grade evidence lineage.
Worse: delivery speed, continuity, and working UI velocity.
Likely kept: market adapters, some schemas, ClickHouse client, NATS helpers, UI visual direction, selected tests.
Likely rewritten: almost all compute, storage schemas, API contracts, replay, UI data model.
Likely deleted: `FlowPacket`, `SmartMoneyEvent`, `ClassifierHitEvent`, `AlertEvent` as currently shaped, current subject hierarchy, current derived tables.
PR sequence:
1. Define new canonical event taxonomy and versioned policy registry.
2. Build raw observation lake and deterministic replay runner first.
3. Build evidence extraction and quote/condition eligibility services.
4. Build cluster and structure hypothesis services.
5. Build hypothesis scoring and calibration services.
6. Build insight projection API.
7. Rebuild terminal against new evidence/hypothesis contracts.
8. Backfill or discard old derived data.
## Recommendation
Choose **Option B**.
Bluntly: Option A is too timid for a pre-alpha product whose current names already fight the research. Option C is intellectually clean but wastes too much working infrastructure. Option B keeps the stack and terminal momentum while fixing the core mistake: treating “smart money” as a thing the system emits, instead of treating smart flow as a cautious, evidence-backed hypothesis with alternatives.
The first implementation move should be the contract/naming PR: introduce `FlowHypothesisEvent` and `FlowEvidenceCluster` with compatibility aliases, then make replay the gate before touching more classifier logic.

View file

@ -0,0 +1,81 @@
# Synthetic Market-Data Architecture Review
## Summary
- Target file: `docs/plans/synthetic-market-data-architecture-review.md`. No files were changed in this Plan Mode pass.
- Recommendation: **Option B — Refactor**. Conservative work would trap determinism inside ingest adapters; full redesign is premature. Refactor makes synthetic generation first-class while keeping the useful NATS, ClickHouse, compute, API, and web stack.
- Core direction: build a no-history, seeded, manifest-driven synthetic event engine with canonical real event types, separate labels/manifests, deterministic replay, fixture generation, load profiles, and demo scenarios.
## Direct Answers
1. Synthetic generation should be a **combination**: a reusable `@islandflow/synthetic-market` package, a CLI for fixture/run generation, replay-source integration, test fixture helpers, and demo presets. A service should be only a thin live/demo emitter.
2. Synthetic events should map to existing canonical event types: `OptionPrint`, `OptionNBBO`, `EquityPrint`, and `EquityQuote`. Do not create parallel synthetic-only market event types for the main pipeline.
3. Use **metadata plus isolation**, not permanent separate business schemas. Add provenance such as `source_kind`, `run_id`, `parameter_snapshot_hash`, and optional `scenario_id`; use run-scoped subjects/databases for tests and load runs when isolation matters.
4. Ground-truth labels should be separate label records keyed by `run_id`, `scenario_id`, event IDs/trace IDs, expected class, expected direction, confidence band, required/forbidden evidence, and false-positive penalties. Do not expose hidden labels on emitted market events.
5. Expected-output manifests should be versioned JSON/YAML artifacts produced by the CLI. They should pin seed bundle, generator version, parameter snapshot hash, generated event hashes, replay ordering, expected derived events, alert/no-alert expectations, and evidence requirements.
6. Deterministic replay should consume either generated fixture files directly or materialized ClickHouse rows through the same replay ordering: event time, ingest time, seq, stable event ID. Replay should support a `synthetic` source/run selector.
7. Tests should use synthetic data at three levels: pure package invariants, small golden manifests through compute batch logic, and optional infra-backed NATS/ClickHouse integration tests. `bun test` should not require Docker.
8. Demos should use named demo runs/scenarios, not ambient live randomness. Keep the hosted synthetic control drawer for live demo tuning, but add deterministic demo run selection/replay.
9. First-class domain objects: `SyntheticRun`, `SeedBundle`, `ParameterSnapshot`, `SymbolProfile`, `LiquidityProfile`, `VolatilityRegime`, `OptionChainProfile`, `ScenarioInjection`, `GroundTruthLabel`, `ExpectedOutputManifest`, `GeneratedEventBatch`, `ReplayPlan`, `LoadProfile`, and `DemoProfile`.
10. Implementation details: PRNG algorithm internals, sampling formulas, placement heuristics, adapter timers, NATS consumer names, Redis rolling windows, ClickHouse loader mechanics, UI labels, and cache policy.
## Area Classification
- Existing replay architecture: **refactor**. Keep event-time merge and stream publishing; add generated-stream sources, run IDs, manifests, and deterministic output comparison.
- Event schemas: **refactor**. Keep canonical raw/derived event shapes; add provenance metadata and separate label/manifest schemas.
- Service boundaries: **refactor**. Move generator logic out of ingest adapters into a package; adapters become thin emitters.
- Test structure: **redesign**. Current tests are unit-heavy and adapter-local; add fixture manifests, golden outputs, and batch replay checks.
- ClickHouse fixture strategy: **refactor**. Keep storage helpers; add run-scoped fixture loaders and optional run metadata, not permanent synthetic clone tables.
- NATS/JetStream: **keep/refactor**. Keep canonical subjects for production behavior; support isolated subject prefixes or disposable streams for tests/load.
- Redis baseline interaction: **refactor**. Keep Redis for live rolling state; golden tests should use in-memory/resettable baselines.
- UI/demo needs: **refactor**. Keep replay UI and synthetic admin rail; add named deterministic demo modes and scenario selectors.
- CI feasibility: **keep/refactor**. Keep fast Bun CI; make synthetic package/golden tests infra-free and defer Docker integration to a separate job.
## Option A — Conservative
- Summary: wrap the current synthetic ingest adapters with minimal metadata, a small fixture CLI, and a few golden tests.
- Pros: fastest, least migration, preserves current demos.
- Cons: determinism remains mixed with wall-clock timers and live adapter behavior; labels/manifests stay bolted on.
- Complexity: low to medium. Migration risk: low.
- Better: quick smoke fixtures, basic provenance, modest replay demos.
- Worse: long-term generator quality, test reliability, scenario authoring.
- Kept: current ingest adapters, bus/storage/API/web mostly unchanged.
- Rewritten: small parts of synthetic adapters and tests.
- Deleted/deferred: deep replay refactor, new package boundary, batch harness.
- PR sequence: add metadata schemas; add CLI wrapper; add fixture files; add basic replay filters; add initial golden tests.
## Option B — Refactor
- Summary: create `@islandflow/synthetic-market` as the deterministic engine; make adapters, CLI, replay, tests, and demos consume it.
- Pros: deterministic by design, reusable, testable, demo-friendly, preserves the working stack.
- Cons: more up-front movement; current adapter logic must be untangled.
- Complexity: medium. Migration risk: medium-low.
- Better: seeded runs, profiles, labels, manifests, replay, golden tests, load profiles.
- Worse: short-term churn and some duplicated paths during migration.
- Kept: canonical event schemas, NATS subjects, ClickHouse helpers, compute classifiers, API replay endpoints, web replay shell.
- Rewritten: synthetic options/equities adapters, synthetic control state, replay source abstraction, tests around synthetic scenarios.
- Deleted/deferred: adapter-local scenario catalog after migration; full LOB/agent/ML simulation.
- PR sequence: add package and schemas; move current generators behind deterministic API; add CLI manifest generation; refactor adapters to consume package; add replay synthetic source/run filters; add golden fixture tests; add demo selector.
## Option C — Redesign
- Summary: rebuild around a unified deterministic event-log architecture where generation, replay, live demo, storage, and tests all consume run-partitioned event logs.
- Pros: cleanest long-term model; excellent determinism, provenance, and replay semantics.
- Cons: too much rebuild for pre-alpha; delays product learning.
- Complexity: high. Migration risk: high.
- Better: architecture purity, reproducible environments, run isolation.
- Worse: delivery speed, disruption, operational risk.
- Kept: some compute/classifier/domain logic and UI concepts.
- Rewritten: replay, ingest, storage partitioning, bus topology, fixture/test harness.
- Deleted/deferred: current synthetic adapters, current replay service shape, much of current live/demo plumbing.
- PR sequence: define event log/envelope; implement generator; rebuild replay; rebuild storage materialization; port compute; port API/UI; retire old ingest paths.
## Recommendation
Choose **Option B**. Bluntly: Option A is a patch, and it will keep producing impressive-looking but untrustworthy demos. Option C is architecture vanity for a pre-alpha product. Option B is the grown-up move: extract the generator into a deterministic package, keep the useful event pipeline, and make replay/tests/demos consume the same generated runs.
## Test Plan
- Unit: PRNG determinism, profile normalization, tick validity, quote/trade invariants, option chain sparsity, label/manifest schema parsing.
- Golden: fixed seed plus manifest produces byte/hash-stable raw events and stable smart-money/alert signatures.
- Replay: synthetic source ordering matches manifest; derived outputs match expected-output manifest.
- Integration: optional NATS/ClickHouse run-scoped fixture test behind a non-default CI job.
- Demo/load: named demo profiles render in replay UI; load profile scales rates without changing event semantics.
## Assumptions
- MVP remains no-history-first.
- Canonical real event schemas remain the pipeline contract.
- Hidden labels are never embedded directly in market events.
- Infra-backed tests are useful, but the first synthetic quality gate must pass in plain `bun test`.