dirtydishes/islandflow
1
0
Fork 0
Code Issues 2 Pull requests 4 Projects Releases Packages Wiki Activity Actions 1

stabilize live api memory and add an options pipeline explainer #8

Open
dirtydishes wants to merge 6 commits from stabilize-live-api-memory into main
pull from: stabilize-live-api-memory
merge into: dirtydishes:main
Conversation 0 Commits 6 Files changed 12 +1711 -52
4 changed files with 1711 additions and 52 deletions
Download patch file Download diff file Expand all files Collapse all files
Showing only changes of commit db7370052f - Show all commits

3
.beads/issues.jsonl
View file

@ -1,4 +1,4 @@
{"_type":"issue","id":"islandflow-thp","title":"stabilize live api memory and reduce internal cache churn","description":"The native VPS deployment is repeatedly OOM-killing islandflow-api.service during live operation. The API live cache is retaining oversized channel histories and rewriting large Redis lists on every flush, which drives multi-GB Bun RSS and heavy loopback traffic between the API, Redis, NATS, and ClickHouse. Implement an emergency VPS mitigation plus repo hardening so unsafe env values, reconnect snapshots, and Redis persistence patterns cannot push the live API back into OOM.","acceptance_criteria":"1. VPS live cache env values are reduced to safe defaults and live redis state is cleared before restart. 2. services/api/src/live.ts enforces server-side live cache caps and clamps snapshot_limit accordingly. 3. Hot generic feed Redis persistence no longer rewrites entire lists on every flush. 4. Metrics/logging expose subscription counts, snapshot sizes, redis flush volume, and API memory trend. 5. Relevant tests pass and the deployment is restarted successfully.","notes":"Implemented and deployed the live-state hardening to the VPS. Final validation after restart showed the API around 120 MB RSS with capped live cache depths and clean systemd restarts.","status":"in_progress","priority":1,"issue_type":"bug","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-05-23T01:30:43Z","created_by":"dirtydishes","updated_at":"2026-05-23T01:50:29Z","started_at":"2026-05-23T01:30:52Z","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"islandflow-thp","title":"stabilize live api memory and reduce internal cache churn","description":"The native VPS deployment is repeatedly OOM-killing islandflow-api.service during live operation. The API live cache is retaining oversized channel histories and rewriting large Redis lists on every flush, which drives multi-GB Bun RSS and heavy loopback traffic between the API, Redis, NATS, and ClickHouse. Implement an emergency VPS mitigation plus repo hardening so unsafe env values, reconnect snapshots, and Redis persistence patterns cannot push the live API back into OOM.","acceptance_criteria":"1. VPS live cache env values are reduced to safe defaults and live redis state is cleared before restart. 2. services/api/src/live.ts enforces server-side live cache caps and clamps snapshot_limit accordingly. 3. Hot generic feed Redis persistence no longer rewrites entire lists on every flush. 4. Metrics/logging expose subscription counts, snapshot sizes, redis flush volume, and API memory trend. 5. Relevant tests pass and the deployment is restarted successfully.","notes":"Implemented and deployed the live-state hardening to the VPS. Final validation after restart showed the API around 120 MB RSS with capped live cache depths and clean systemd restarts.","status":"closed","priority":1,"issue_type":"bug","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-05-23T01:30:43Z","created_by":"dirtydishes","updated_at":"2026-05-23T01:50:41Z","started_at":"2026-05-23T01:30:52Z","closed_at":"2026-05-23T01:50:41Z","close_reason":"Closed","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"islandflow-sc6","title":"fix electron codex bridge preload loading","description":"Electron settings showed the browser-only Desktop Required fallback because the renderer did not see the native islandflowDesktop preload bridge or an Electron user-agent marker. Fix the desktop launch path so ChatGPT/Codex subscription controls are available inside Islandflow Desktop again.","notes":"Reopened after live Electron still showed the browser-only fallback. Follow-up fix adds an explicit preload runtime marker and web runtime detection for that marker so Electron is recognized even when the bridge is not ready and the user agent lacks an Electron token.","status":"closed","priority":1,"issue_type":"bug","owner":"dishes@dpdrm.com","created_at":"2026-05-20T23:42:58Z","created_by":"dirtydishes","updated_at":"2026-05-20T23:51:43Z","closed_at":"2026-05-20T23:51:43Z","close_reason":"Follow-up fix added an explicit islandflowDesktopRuntime preload marker and taught the web runtime to recognize that marker plus IslandflowDesktop user-agent tokens, so Electron no longer falls into the browser-only fallback when the AI bridge is delayed or unavailable. Desktop build and focused desktop/web tests pass; full web build still blocked by islandflow-c8f.","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"islandflow-hj3","title":"Fix Electron preload for desktop AI bridge","description":"## Why\\nThe desktop settings page reports the native AI bridge as unavailable because Electron fails to load the preload script in local dev.\\n\\n## What\\nUpdate the desktop preload implementation/build so Electron can execute it, restore window.islandflowDesktop, and verify the Copilot settings panel detects the bridge again.\\n\\n## Acceptance Criteria\\n- Electron no longer logs a preload syntax error\\n- window.islandflowDesktop is available in the desktop renderer\\n- The settings page no longer shows bridge unavailable solely because preload failed\\n- Relevant desktop/web tests pass","status":"closed","priority":1,"issue_type":"bug","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-05-20T23:16:39Z","created_by":"dirtydishes","updated_at":"2026-05-20T23:20:20Z","started_at":"2026-05-20T23:16:48Z","closed_at":"2026-05-20T23:20:20Z","close_reason":"Closed","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"islandflow-199","title":"fix desktop copilot fallback inside electron","description":"## Why\\nThe settings page can render the browser-only fallback even when Islandflow is running inside the Electron desktop shell.\\n\\n## What\\nSeparate desktop-shell detection from desktop AI transport state, make the provider recover if the bridge appears late or initial state loading fails, and cover the regression with tests.\\n\\n## Acceptance Criteria\\n- The desktop shell no longer shows the browser-only fallback solely because initial bridge state failed or arrived late\\n- Desktop-only actions can distinguish between missing Electron bridge and transport/auth problems\\n- Automated tests cover the recovery behavior","status":"closed","priority":1,"issue_type":"bug","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-05-20T22:30:16Z","created_by":"dirtydishes","updated_at":"2026-05-20T22:37:21Z","started_at":"2026-05-20T22:30:23Z","closed_at":"2026-05-20T22:37:21Z","close_reason":"Fixed desktop-shell Copilot fallback handling, added bridge recovery logic, updated desktop-vs-bridge UI messaging, and added regression tests. Follow-up tracked in islandflow-c8f for unrelated web build blocker.","dependency_count":0,"dependent_count":0,"comment_count":0}
@ -72,6 +72,7 @@
{"_type":"issue","id":"islandflow-zs0","title":"Migrate terminal UI to smart-money profiles","description":"Migrate apps/web terminal rendering to consume SmartMoneyEvent directly: primary profile, probability ladder, reason codes, and suppression/abstention state, while preserving legacy alert/classifier displays during the bridge.","status":"closed","priority":2,"issue_type":"task","owner":"dishes@dpdrm.com","created_at":"2026-05-04T21:35:23Z","created_by":"dirtydishes","updated_at":"2026-05-05T05:39:58Z","closed_at":"2026-05-05T05:39:58Z","close_reason":"Completed terminal smart-money profile migration","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"islandflow-igk","title":"Add plan mode","description":"Implement a user-facing plan mode in the application so users can switch into planning before taking action. Scope to be clarified from existing app patterns.","status":"closed","priority":2,"issue_type":"feature","owner":"dishes@dpdrm.com","created_at":"2026-05-04T04:22:37Z","created_by":"dirtydishes","updated_at":"2026-05-04T04:26:18Z","started_at":"2026-05-04T04:22:40Z","closed_at":"2026-05-04T04:26:18Z","close_reason":"Implemented as a global pi extension toggled with Shift+P","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"islandflow-biq","title":"Finish raw live options delivery and filter/backpressure observability","description":"The smart-money signal path and Tape filters are in place, but the next firehose pass should finish server-side selective raw live delivery for options subscriptions and add explicit filtered-out/backpressure observability for API/web counters. This was discovered while landing islandflow-e4r.\n","status":"in_progress","priority":2,"issue_type":"task","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-04-28T20:28:58Z","created_by":"dirtydishes","updated_at":"2026-04-29T03:54:12Z","started_at":"2026-04-29T03:54:12Z","dependencies":[{"issue_id":"islandflow-biq","depends_on_id":"islandflow-e4r","type":"discovered-from","created_at":"2026-04-28T16:28:58Z","created_by":"auto-import","metadata":"{}"}],"dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"islandflow-hpf","title":"add anatomy explainer for options print and smart money flow","description":"Create a standalone docs/anatomy.html reference page that explains the end-to-end lifecycle of an options print through enrichment, signal filtering, compute clustering, flow packet creation, smart-money evaluation, classifier hits, alerts, and API/live consumption. The page should be polished, user-readable, and visually strong enough to serve as a reusable reference artifact for both technical and non-technical readers.","notes":"Added docs/anatomy.html as a standalone reference page for the options-print to smart-money pipeline, styled in the repo product register and layered for executive, mixed technical, and operator-level readers. Regenerated docs/index.html so the page is discoverable from the docs surface.","status":"closed","priority":3,"issue_type":"task","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-05-23T02:18:48Z","created_by":"dirtydishes","updated_at":"2026-05-23T02:24:58Z","started_at":"2026-05-23T02:18:53Z","closed_at":"2026-05-23T02:24:58Z","close_reason":"Closed","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"islandflow-4ca","title":"Publish May 21 standup git summary","description":"Create the daily standup-ready git activity summary for 2026-05-21, save the HTML artifact under docs/general, add the required turn document, and push the result so the automation leaves a durable record.","status":"closed","priority":3,"issue_type":"task","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-05-22T13:03:00Z","created_by":"dirtydishes","updated_at":"2026-05-22T13:05:05Z","started_at":"2026-05-22T13:03:03Z","closed_at":"2026-05-22T13:05:05Z","close_reason":"Created the 2026-05-21 standup summary in docs/general, added the required turn document, and prepared the repo for commit/push.","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"islandflow-hgm","title":"Publish May 20 standup git summary","description":"Create the daily standup-ready git activity summary for 2026-05-20, save the HTML artifact under docs/general, and push the result so the automation leaves a durable record.","status":"closed","priority":3,"issue_type":"task","owner":"dishes@dpdrm.com","created_at":"2026-05-21T13:02:38Z","created_by":"dirtydishes","updated_at":"2026-05-21T13:05:16Z","closed_at":"2026-05-21T13:05:16Z","close_reason":"Closed","dependency_count":0,"dependent_count":0,"comment_count":0}
{"_type":"issue","id":"islandflow-4q0","title":"refresh readme app description with current classification approach","description":"Update README intro content to better describe the app's current architecture and include a concise explanation of how Islandflow classifies prints, aligned with smartmoney.md and current services.","status":"closed","priority":3,"issue_type":"task","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-05-21T01:53:30Z","created_by":"dirtydishes","updated_at":"2026-05-21T01:55:01Z","started_at":"2026-05-21T01:53:33Z","closed_at":"2026-05-21T01:55:01Z","close_reason":"Closed","dependency_count":0,"dependent_count":0,"comment_count":0}

954
docs/anatomy.html Normal file
View file

@ -0,0 +1,954 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title>The Anatomy of an Options Print and Smart Money</title>
<style>
:root {
color-scheme: dark;
--bg-core: oklch(0.14 0.015 240);
--bg-shell: oklch(0.17 0.018 240);
--bg-pane: oklch(0.2 0.02 240);
--bg-pane-2: oklch(0.23 0.02 240);
--bg-soft: oklch(0.28 0.022 240 / 0.48);
--line: oklch(0.48 0.03 240 / 0.34);
--line-strong: oklch(0.67 0.06 75 / 0.48);
--text: oklch(0.93 0.01 240);
--text-dim: oklch(0.74 0.02 240);
--text-faint: oklch(0.62 0.016 240);
--amber: oklch(0.79 0.16 76);
--amber-soft: oklch(0.33 0.07 76 / 0.32);
--green: oklch(0.74 0.15 154);
--green-soft: oklch(0.31 0.06 154 / 0.3);
--blue: oklch(0.72 0.12 244);
--blue-soft: oklch(0.31 0.05 244 / 0.28);
--red: oklch(0.69 0.17 28);
--red-soft: oklch(0.31 0.06 28 / 0.32);
--shadow: 0 30px 80px rgba(0, 0, 0, 0.38);
--radius-xl: 20px;
--radius-lg: 16px;
--radius-md: 12px;
--radius-sm: 10px;
--mono: "IBM Plex Mono", "SFMono-Regular", Consolas, "Liberation Mono", Menlo, monospace;
--sans: "IBM Plex Sans", "Avenir Next", "Segoe UI", sans-serif;
--display: "Quantico", "IBM Plex Sans", sans-serif;
}
* {
box-sizing: border-box;
}
html {
scroll-behavior: smooth;
}
body {
margin: 0;
font-family: var(--sans);
color: var(--text);
background:
radial-gradient(circle at 0% 0%, rgba(245, 166, 35, 0.09), transparent 24%),
radial-gradient(circle at 100% 0%, rgba(77, 163, 255, 0.12), transparent 26%),
linear-gradient(180deg, var(--bg-shell) 0%, var(--bg-core) 100%);
}
a {
color: var(--amber);
}
code,
pre,
.eyebrow,
.chip,
.mini-label,
th,
.lane-label {
font-family: var(--mono);
}
main {
width: min(1440px, calc(100vw - 32px));
margin: 0 auto;
padding: 28px 0 64px;
}
.hero {
display: grid;
gap: 22px;
padding: 28px;
border: 1px solid var(--line);
border-radius: 28px;
background:
linear-gradient(180deg, rgba(255, 255, 255, 0.035) 0%, rgba(255, 255, 255, 0.015) 100%),
linear-gradient(135deg, rgba(17, 24, 32, 0.96) 0%, rgba(8, 11, 16, 0.98) 100%);
box-shadow: var(--shadow);
}
.hero-top {
display: flex;
flex-wrap: wrap;
align-items: center;
justify-content: space-between;
gap: 12px;
}
.eyebrow {
display: inline-flex;
align-items: center;
gap: 10px;
padding: 8px 12px;
border-radius: 999px;
border: 1px solid var(--line);
background: rgba(255, 255, 255, 0.03);
color: var(--text-dim);
font-size: 0.72rem;
letter-spacing: 0.14em;
text-transform: uppercase;
}
h1,
h2,
h3 {
margin: 0;
line-height: 1.08;
}
h1 {
font-family: var(--display);
font-size: clamp(2.35rem, 5vw, 4.8rem);
letter-spacing: 0.04em;
text-transform: uppercase;
}
h2 {
font-family: var(--display);
font-size: 1.5rem;
letter-spacing: 0.04em;
text-transform: uppercase;
}
h3 {
font-size: 1.04rem;
font-weight: 650;
}
.hero-copy {
display: grid;
grid-template-columns: minmax(0, 1.7fr) minmax(280px, 0.85fr);
gap: 26px;
align-items: start;
}
.lede {
margin: 0;
max-width: 68ch;
color: var(--text-dim);
font-size: 1rem;
line-height: 1.62;
}
.hero-note {
padding: 18px;
border-radius: var(--radius-lg);
border: 1px solid var(--line);
background: rgba(255, 255, 255, 0.03);
}
.hero-note p {
margin: 0;
color: var(--text-dim);
line-height: 1.55;
}
.hero-note p + p {
margin-top: 12px;
}
.toolbar {
display: flex;
flex-wrap: wrap;
gap: 10px;
}
.chip {
display: inline-flex;
align-items: center;
gap: 8px;
padding: 7px 12px;
border-radius: 999px;
border: 1px solid var(--line);
text-decoration: none;
color: var(--text);
background: rgba(255, 255, 255, 0.03);
font-size: 0.75rem;
letter-spacing: 0.08em;
text-transform: uppercase;
}
.chip:hover {
border-color: var(--line-strong);
}
.chip .dot {
width: 8px;
height: 8px;
border-radius: 999px;
flex: 0 0 auto;
}
.chip.raw .dot {
background: var(--blue);
}
.chip.derived .dot {
background: var(--amber);
}
.chip.store .dot {
background: var(--green);
}
.chip.live .dot {
background: var(--red);
}
.stack {
display: grid;
gap: 22px;
margin-top: 22px;
}
.section {
border: 1px solid var(--line);
border-radius: 24px;
padding: 24px;
background:
linear-gradient(180deg, rgba(255, 255, 255, 0.035) 0%, rgba(255, 255, 255, 0.016) 100%),
var(--bg-pane);
box-shadow: var(--shadow);
}
.section-head {
display: grid;
gap: 8px;
margin-bottom: 18px;
}
.section-head p,
.copy p,
.copy li,
.table-wrap td,
.table-wrap th {
color: var(--text-dim);
line-height: 1.58;
}
.section-head p,
.copy p {
margin: 0;
max-width: 74ch;
}
.summary-grid {
display: grid;
grid-template-columns: repeat(3, minmax(0, 1fr));
gap: 16px;
}
.summary-card {
padding: 18px;
border-radius: var(--radius-lg);
border: 1px solid var(--line);
background: linear-gradient(180deg, rgba(255, 255, 255, 0.025) 0%, rgba(255, 255, 255, 0.01) 100%);
}
.summary-card p {
margin: 10px 0 0;
}
.mini-label {
display: inline-block;
margin-bottom: 8px;
color: var(--text-faint);
font-size: 0.72rem;
letter-spacing: 0.12em;
text-transform: uppercase;
}
.flow-scroll {
overflow-x: auto;
padding-bottom: 6px;
}
.flow-board {
min-width: 1320px;
display: grid;
gap: 18px;
}
.lanes {
display: grid;
grid-template-columns: 160px repeat(6, minmax(180px, 1fr));
gap: 14px;
align-items: stretch;
}
.lane-label {
display: flex;
align-items: center;
justify-content: center;
padding: 16px;
border-radius: var(--radius-lg);
border: 1px solid var(--line);
background: rgba(255, 255, 255, 0.025);
color: var(--text-faint);
font-size: 0.74rem;
letter-spacing: 0.12em;
text-transform: uppercase;
text-align: center;
}
.node {
position: relative;
min-height: 138px;
padding: 16px;
border-radius: 18px;
border: 1px solid var(--line);
background: var(--bg-pane-2);
display: grid;
gap: 10px;
}
.node strong {
font-size: 1.02rem;
line-height: 1.25;
}
.node p {
margin: 0;
color: var(--text-dim);
font-size: 0.92rem;
line-height: 1.48;
}
.node ul {
margin: 0;
padding-left: 18px;
color: var(--text-dim);
}
.node li + li {
margin-top: 6px;
}
.node.raw {
box-shadow: inset 0 0 0 1px rgba(77, 163, 255, 0.18);
}
.node.derived {
box-shadow: inset 0 0 0 1px rgba(245, 166, 35, 0.24);
}
.node.store {
box-shadow: inset 0 0 0 1px rgba(37, 193, 122, 0.24);
}
.node.live {
box-shadow: inset 0 0 0 1px rgba(255, 107, 95, 0.24);
}
.node-arrow::after {
content: "";
position: absolute;
top: 50%;
right: -12px;
width: 24px;
height: 2px;
background: linear-gradient(90deg, var(--line-strong) 0%, var(--amber) 100%);
}
.node-arrow::before {
content: "";
position: absolute;
top: calc(50% - 5px);
right: -12px;
border-top: 6px solid transparent;
border-bottom: 6px solid transparent;
border-left: 8px solid var(--amber);
}
.branch-grid {
display: grid;
grid-template-columns: 160px repeat(6, minmax(180px, 1fr));
gap: 14px;
align-items: stretch;
}
.branch-empty {
border-radius: var(--radius-lg);
border: 1px dashed transparent;
min-height: 112px;
}
.branch-box {
min-height: 112px;
}
.branch-box p {
margin: 0;
}
.copy {
display: grid;
gap: 16px;
}
.copy ul {
margin: 0;
padding-left: 18px;
}
.copy li + li {
margin-top: 8px;
}
.detail-grid {
display: grid;
grid-template-columns: minmax(0, 1.2fr) minmax(320px, 0.8fr);
gap: 18px;
}
.callout {
padding: 18px;
border-radius: var(--radius-lg);
border: 1px solid var(--line);
background: rgba(255, 255, 255, 0.025);
}
.callout p {
margin: 0;
}
.callout p + p {
margin-top: 10px;
}
.table-wrap {
overflow-x: auto;
border: 1px solid var(--line);
border-radius: var(--radius-lg);
}
table {
width: 100%;
border-collapse: collapse;
min-width: 720px;
}
th,
td {
padding: 12px 14px;
border-bottom: 1px solid var(--line);
text-align: left;
vertical-align: top;
}
th {
color: var(--text);
background: rgba(255, 255, 255, 0.035);
font-size: 0.76rem;
letter-spacing: 0.1em;
text-transform: uppercase;
}
tr:last-child td {
border-bottom: 0;
}
pre {
margin: 0;
padding: 18px;
border-radius: var(--radius-lg);
background: oklch(0.13 0.012 240);
border: 1px solid var(--line);
color: var(--text);
overflow-x: auto;
line-height: 1.5;
font-size: 0.88rem;
}
.footer-note {
color: var(--text-faint);
font-size: 0.88rem;
}
@media (max-width: 1080px) {
.hero-copy,
.detail-grid,
.summary-grid {
grid-template-columns: 1fr;
}
}
@media (max-width: 720px) {
main {
width: min(100vw, calc(100vw - 20px));
padding-top: 12px;
}
.hero,
.section {
padding: 18px;
border-radius: 18px;
}
h1 {
font-size: clamp(1.9rem, 8vw, 2.8rem);
}
}
</style>
</head>
<body>
<main>
<section class="hero">
<div class="hero-top">
<span class="eyebrow">Islandflow Reference · Options Flow Pipeline</span>
<nav class="toolbar" aria-label="Page navigation">
<a class="chip raw" href="#flow-chart"><span class="dot"></span>Flow Chart</a>
<a class="chip derived" href="#executive"><span class="dot"></span>Executive</a>
<a class="chip store" href="#technical"><span class="dot"></span>Technical</a>
<a class="chip live" href="#operator"><span class="dot"></span>Operator Detail</a>
</nav>
</div>
<div class="hero-copy">
<div>
<h1>The Anatomy of an Options Print and Smart Money</h1>
<p class="lede">
This page explains how a single options print moves through Islandflow under normal market conditions,
how the signal gate decides whether compute should care, how a parent flow packet is assembled, and how
smart-money, classifier-hit, and alert events emerge from that packet. It is designed as one artifact
with three reading depths: executive, mixed technical, and operator-level.
</p>
</div>
<aside class="hero-note">
<p>
The key distinction is structural: the options tape is print-level, flow is packet-level, and smart
money is model output on those packets.
</p>
<p>
Synthetic mode changes the source of prints and NBBO context, not the downstream architecture.
</p>
</aside>
</div>
</section>
<div class="stack">
<section class="section">
<div class="section-head">
<h2>Legend</h2>
<p>Color coding is semantic, not decorative, so you can scan the diagram without relearning the vocabulary.</p>
</div>
<div class="toolbar" aria-label="Legend">
<span class="chip raw"><span class="dot"></span>Raw market or synthetic input</span>
<span class="chip derived"><span class="dot"></span>Derived compute stage</span>
<span class="chip store"><span class="dot"></span>Stored or persisted state</span>
<span class="chip live"><span class="dot"></span>API, websocket, or user-facing surface</span>
</div>
</section>
<section class="section" id="flow-chart">
<div class="section-head">
<h2>Main Flow Chart</h2>
<p>
The first row shows the common path every print touches. The second row shows the branch between prints
that remain tape-only and prints that become packet candidates for smart-money evaluation.
</p>
</div>
<div class="flow-scroll">
<div class="flow-board">
<div class="lanes">
<div class="lane-label">Input</div>
<article class="node raw node-arrow">
<span class="mini-label">Stage 1</span>
<strong>Option print candidate arrives</strong>
<p>
The source can be a native market adapter or the synthetic adapter. Synthetic mode can also emit a
matching NBBO update.
</p>
</article>
<article class="node raw node-arrow">
<span class="mini-label">Stage 2</span>
<strong>ingest-options enriches the print</strong>
<p>
The service joins recent option NBBO and underlying equity quote context, derives metadata, and
computes <code>signal_pass</code>.
</p>
</article>
<article class="node store node-arrow">
<span class="mini-label">Stage 3</span>
<strong>Raw print is written and published</strong>
<ul>
<li>ClickHouse: <code>option_prints</code></li>
<li>NATS: <code>options.prints</code></li>
</ul>
</article>
<article class="node derived node-arrow">
<span class="mini-label">Stage 4</span>
<strong>Signal gate decides if compute should care</strong>
<p>
Only <code>signal_pass=true</code> prints are published to <code>options.prints.signal</code> and
consumed by compute.
</p>
</article>
<article class="node derived node-arrow">
<span class="mini-label">Stage 5</span>
<strong>compute builds or updates a parent cluster</strong>
<p>
Nearby signal prints for the same contract are grouped inside the cluster window while NBBO and
equity-quote caches supply context.
</p>
</article>
<article class="node live">
<span class="mini-label">Stage 6</span>
<strong>API and UI consume the resulting streams</strong>
<p>
The API hydrates hot snapshots, history endpoints read ClickHouse, and the terminal surfaces tape,
flow, smart-money, classifier, and alert views.
</p>
</article>
</div>
<div class="branch-grid">
<div class="lane-label">Tape-only branch</div>
<div class="branch-empty"></div>
<div class="branch-empty"></div>
<article class="node store branch-box">
<span class="mini-label">Branch A</span>
<strong>Raw print remains visible</strong>
<p>
Even if the print does not pass the signal gate, it still exists in ClickHouse and can appear in
raw tape or history views.
</p>
</article>
<article class="node derived branch-box">
<span class="mini-label">Branch A outcome</span>
<strong>No compute packet path</strong>
<p>
No <code>FlowPacket</code>, no smart-money evaluation, no classifier hits, and no alert emission.
</p>
</article>
<div class="branch-empty"></div>
<div class="branch-empty"></div>
</div>
<div class="branch-grid">
<div class="lane-label">Smart-money branch</div>
<div class="branch-empty"></div>
<div class="branch-empty"></div>
<div class="branch-empty"></div>
<article class="node derived branch-box">
<span class="mini-label">Branch B</span>
<strong>Signal print enters compute</strong>
<p>
compute subscribes to <code>options.prints.signal</code>, not raw <code>options.prints</code>.
</p>
</article>
<article class="node derived branch-box">
<span class="mini-label">Branch B outcome</span>
<strong>FlowPacket is emitted</strong>
<ul>
<li>ClickHouse: <code>flow_packets</code></li>
<li>NATS: <code>flow.packets</code></li>
</ul>
</article>
<article class="node live branch-box">
<span class="mini-label">Branch B continuation</span>
<strong>Smart-money, classifier hits, alerts</strong>
<p>
The packet is scored into a <code>SmartMoneyEvent</code>, which may abstain, produce classifier
hits, and finally emit an alert.
</p>
</article>
</div>
</div>
</div>
</section>
<section class="section" id="executive">
<div class="section-head">
<h2>Executive Read</h2>
<p>
The shortest truthful version of the system: not every options print is considered meaningful, and smart
money is not detected directly from a single print.
</p>
</div>
<div class="summary-grid">
<article class="summary-card">
<span class="mini-label">1. Tape</span>
<h3>Every print is stored</h3>
<p>
All enriched prints are written to ClickHouse and published to the raw options subject. This preserves
evidence even when the print is uninteresting for higher-order inference.
</p>
</article>
<article class="summary-card">
<span class="mini-label">2. Compute</span>
<h3>Only signal prints reach the parent-event engine</h3>
<p>
A print must pass the signal gate before compute clusters it with neighboring prints and builds a
packet that represents a possible parent order.
</p>
</article>
<article class="summary-card">
<span class="mini-label">3. Smart money</span>
<h3>Smart money is a scored interpretation</h3>
<p>
The model evaluates the packet using quote quality, aggressor mix, size, structure, DTE, IV, and event
context. It can still abstain if the evidence is weak or suppressed.
</p>
</article>
</div>
</section>
<section class="section" id="technical">
<div class="section-head">
<h2>Mixed Technical Walkthrough</h2>
<p>
This layer is for teammates who know the product and want the exact branching logic without reading
through service code first.
</p>
</div>
<div class="detail-grid">
<div class="copy">
<p>
<strong>Step 1:</strong> a candidate print enters <code>ingest-options</code>. In synthetic mode this
print was manufactured by the synthetic adapter, which may also emit a synthetic NBBO update for the
same contract.
</p>
<p>
<strong>Step 2:</strong> the print is enriched with the most recent option NBBO and underlying equity
quote at or before the print timestamp. The service derives metadata, execution-side context, and the
<code>signal_pass</code> decision.
</p>
<p>
<strong>Step 3:</strong> the enriched print is persisted to ClickHouse and published to
<code>options.prints</code>. If <code>signal_pass=true</code>, the same print is also published to
<code>options.prints.signal</code>.
</p>
<p>
<strong>Step 4:</strong> compute subscribes to the signal subject plus NBBO and equity-quote subjects.
It does not build packet candidates from every raw print. It only clusters signal prints.
</p>
<p>
<strong>Step 5:</strong> compute aggregates nearby signal prints for the same option contract into a
cluster, then flushes that cluster into a <code>FlowPacket</code> with features such as total premium,
print count, aggressor ratios, NBBO coverage, stale-quote counts, IV context, and structure clues.
</p>
<p>
<strong>Step 6:</strong> the packet is transformed into a <code>SmartMoneyEvent</code>. If suppression
rules trip or the top profile probability is too weak, the event abstains. Otherwise, it can emit
classifier hits and finally an alert with evidence references back to the packet and member prints.
</p>
</div>
<aside class="callout">
<span class="mini-label">Important distinction</span>
<p>
A <code>FlowPacket</code> is already a derived parent-event candidate. It is not just another name for
the options tape.
</p>
<p>
A <code>SmartMoneyEvent</code> is model output on that packet, not a raw tape fact. The system treats
it as evidence-backed interpretation with explicit abstention and suppression paths.
</p>
</aside>
</div>
</section>
<section class="section" id="operator">
<div class="section-head">
<h2>Operator and Code-Level Detail</h2>
<p>
This section is for someone tracing the live pipeline, debugging a regression, or trying to understand
exactly why a given print surfaced on tape but did or did not become a smart-money event.
</p>
</div>
<div class="copy">
<p>
The first fork is the signal gate in <code>ingest-options</code>. The enriched print is always stored and
published raw. The only thing <code>signal_pass</code> controls is whether compute receives that print on
<code>options.prints.signal</code>.
</p>
<p>
The compute service maintains separate caches for option NBBO and underlying equity quotes. When signal
prints arrive, it flushes aged clusters, extends the active cluster for that contract if the print lands
within the configured window, or emits the old cluster and starts a new one.
</p>
<p>
The cluster becomes a <code>FlowPacket</code> only after compute summarizes parent-level features. That
packet then passes through smart-money scoring. The scoring layer derives a profile set such as
institutional directional, retail whale, event driven, vol seller, arbitrage, or hedge reactive.
</p>
<p>
A packet can still fail to produce actionable downstream artifacts. Suppression rules down-rank special
print context, stale or missing quote context, and cross-like execution patterns. The top profile must
also clear the probability threshold. If it does not, the smart-money event is emitted in abstained form
and classifier hits stop there.
</p>
<p>
If the packet does clear those checks, compute writes and publishes the smart-money event, derives up to
a few classifier hits from the top profile set, scores a final alert, and publishes all three derived
streams. The API subscribes to those subjects and fans them out into live websocket channels while
ClickHouse remains the history source behind <code>/history/*</code>.
</p>
</div>
<div class="table-wrap">
<table>
<thead>
<tr>
<th>Subject or table</th>
<th>Produced by</th>
<th>Carries</th>
<th>Why it exists</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>options.prints</code></td>
<td><code>ingest-options</code></td>
<td>All enriched option prints</td>
<td>Preserves the full tape, even when a print is not interesting enough for compute.</td>
</tr>
<tr>
<td><code>options.prints.signal</code></td>
<td><code>ingest-options</code></td>
<td>Signal-passing option prints</td>
<td>Acts as the compute admission gate so packet building starts from a filtered tape.</td>
</tr>
<tr>
<td><code>flow.packets</code></td>
<td><code>compute</code></td>
<td>Parent-event candidates</td>
<td>Turns several child prints into one summarized event with market-structure features.</td>
</tr>
<tr>
<td><code>flow.smart_money</code></td>
<td><code>compute</code></td>
<td>Smart-money evaluations</td>
<td>Publishes the scored interpretation of a packet, including abstained outcomes.</td>
</tr>
<tr>
<td><code>flow.classifier_hits</code></td>
<td><code>compute</code></td>
<td>Top classifier consequences</td>
<td>Exposes the strongest profile-level labels that downstream UX and alerting can decorate.</td>
</tr>
<tr>
<td><code>flow.alerts</code></td>
<td><code>compute</code></td>
<td>Alert events with evidence refs</td>
<td>Packages the final severity and supporting evidence into a user-facing alert stream.</td>
</tr>
</tbody>
</table>
</div>
</section>
<section class="section">
<div class="section-head">
<h2>Normal Path Versus Smart-Money Path</h2>
<p>
These two sequences are easy to confuse, especially because both begin with the same enriched tape
record.
</p>
</div>
<div class="detail-grid">
<div class="callout">
<span class="mini-label">Normal market path</span>
<p>
Print arrives, gets enriched, gets stored, appears on the raw tape, and stops there unless it passes
the signal gate. This is the dominant path for ordinary or low-signal activity.
</p>
</div>
<div class="callout">
<span class="mini-label">Smart-money path</span>
<p>
Print arrives, passes the signal gate, joins a cluster, becomes a packet, receives a smart-money score,
then may emit classifier hits and an alert if the packet is not suppressed or abstained.
</p>
</div>
</div>
</section>
<section class="section">
<div class="section-head">
<h2>Annotated Event Sequence</h2>
<p>
The example below is the shortest operator-friendly way to think about the branch that leads to a
smart-money result.
</p>
</div>
<pre><code>1. Synthetic or market adapter emits OptionPrint candidate
2. ingest-options enriches it with latest NBBO and underlying quote context
3. Enriched print is written to ClickHouse option_prints
4. Enriched print is published to options.prints
5. If signal_pass=true, the same print is also published to options.prints.signal
6. compute consumes options.prints.signal and updates the active contract cluster
7. Cluster flush builds a FlowPacket with parent-level features
8. FlowPacket is written to ClickHouse flow_packets and published to flow.packets
9. compute scores the packet into a SmartMoneyEvent
10. If suppressed or low-confidence, the SmartMoneyEvent abstains and stops there
11. Otherwise classifier hits are emitted
12. Alert scoring emits a final alert with evidence refs to smart-money event, flow packet, and member prints
13. API subscribes to these streams and exposes them through live websocket channels and ClickHouse-backed history</code></pre>
</section>
<section class="section">
<div class="section-head">
<h2>What Synthetic Mode Changes</h2>
<p>
Synthetic mode can make the upstream generator artificial, but the downstream branch logic stays
identical.
</p>
</div>
<div class="copy">
<p>
The synthetic adapter constructs an <code>OptionPrint</code> with fields such as
<code>execution_iv_source="synthetic_pressure_model"</code>, and it may emit a synthetic NBBO for the
same contract. From that point forward, the pipeline is the same one used for normal ingest.
</p>
<p>
That means synthetic smart-money is not a special smart-money subsystem. It is the standard
signal-to-packet-to-smart-money pipeline running on synthetic upstream events.
</p>
</div>
</section>
<section class="section">
<div class="section-head">
<h2>Code Anchors</h2>
<p>
If you want to confirm this page against the code, these are the most useful entry points.
</p>
</div>
<div class="copy">
<ul>
<li><code>services/ingest-options/src/enrichment.ts</code>: enriches the print and decides <code>signal_pass</code>.</li>
<li><code>services/ingest-options/src/index.ts</code>: writes prints and publishes raw versus signal subjects.</li>
<li><code>services/compute/src/index.ts</code>: subscribes to signal prints, maintains clusters, emits packets, smart money, hits, and alerts.</li>
<li><code>services/compute/src/parent-events.ts</code>: builds <code>SmartMoneyEvent</code>, suppression rules, primary profile, abstention, and classifier derivation.</li>
<li><code>packages/bus/src/subjects.ts</code>: canonical subject names for the pipeline.</li>
</ul>
</div>
<p class="footer-note">
This document is intended as a living product reference, not a turn artifact. If the packet features,
thresholds, or stream names change, update this page alongside the relevant pipeline code.
</p>
</section>
</div>
</main>
</body>
</html>

222
docs/index.html
View file

@ -207,36 +207,76 @@
</header>
<section class="toolbar">
<div class="stats"><strong id="visible-count">35</strong> of <strong>35</strong> files shown</div>
<div class="stats"><strong id="visible-count">47</strong> of <strong>47</strong> files shown</div>
<input id="doc-search" class="search" type="search" placeholder="Filter by filename or folder..." autocomplete="off" />
<nav class="chips"><a class="chip" href="#category-turns">turns <span>28</span></a>
<nav class="chips"><a class="chip" href="#category-turns">turns <span>37</span></a>
<a class="chip" href="#category-daily-git">daily-git <span>1</span></a>
<a class="chip" href="#category-general">general <span>2</span></a>
<a class="chip" href="#category-general">general <span>4</span></a>
<a class="chip" href="#category-plans">plans <span>2</span></a>
<a class="chip" href="#category-root">root <span>2</span></a></nav>
<a class="chip" href="#category-root">root <span>3</span></a></nav>
</section>
<section class="groups" id="groups">
<section class="group" id="category-turns">
<h2>turns <span>28</span></h2>
<h2>turns <span>37</span></h2>
<ul class="doc-list">
<li class="doc-item" data-search="turns/2026-05-19-publish-docs-pages-index.html turns">
<a class="doc-link" href="./turns/2026-05-19-publish-docs-pages-index.html">turns/2026-05-19-publish-docs-pages-index.html</a>
<li class="doc-item" data-search="turns/2026-05-22-stabilize-live-api-memory.html turns">
<a class="doc-link" href="./turns/2026-05-22-stabilize-live-api-memory.html">turns/2026-05-22-stabilize-live-api-memory.html</a>
<div class="meta">
<span class="tag">html</span>
<span>6.7 KB</span>
<span>May 19, 2026, 2:59 PM</span>
<span>26 KB</span>
<span>May 22, 2026, 9:47 PM</span>
</div>
</li>
<li class="doc-item" data-search="turns/2026-05-18-native-public-edge-cutover.html turns">
<a class="doc-link" href="./turns/2026-05-18-native-public-edge-cutover.html">turns/2026-05-18-native-public-edge-cutover.html</a>
<li class="doc-item" data-search="turns/2026-05-22-publish-standup-summary-2026-05-21.html turns">
<a class="doc-link" href="./turns/2026-05-22-publish-standup-summary-2026-05-21.html">turns/2026-05-22-publish-standup-summary-2026-05-21.html</a>
<div class="meta">
<span class="tag">html</span>
<span>19 KB</span>
<span>May 19, 2026, 2:48 PM</span>
<span>5.5 KB</span>
<span>May 22, 2026, 9:04 AM</span>
</div>
</li>
<li class="doc-item" data-search="turns/2026-05-21-publish-standup-summary-2026-05-20.html turns">
<a class="doc-link" href="./turns/2026-05-21-publish-standup-summary-2026-05-20.html">turns/2026-05-21-publish-standup-summary-2026-05-20.html</a>
<div class="meta">
<span class="tag">html</span>
<span>5.0 KB</span>
<span>May 21, 2026, 9:05 AM</span>
</div>
</li>
<li class="doc-item" data-search="turns/2026-05-20-refresh-readme-github-description.html turns">
<a class="doc-link" href="./turns/2026-05-20-refresh-readme-github-description.html">turns/2026-05-20-refresh-readme-github-description.html</a>
<div class="meta">
<span class="tag">html</span>
<span>7.7 KB</span>
<span>May 20, 2026, 9:54 PM</span>
</div>
</li>
<li class="doc-item" data-search="turns/2026-05-20-remote-backfill-sync.html turns">
<a class="doc-link" href="./turns/2026-05-20-remote-backfill-sync.html">turns/2026-05-20-remote-backfill-sync.html</a>
<div class="meta">
<span class="tag">html</span>
<span>4.3 KB</span>
<span>May 20, 2026, 9:26 PM</span>
</div>
</li>
<li class="doc-item" data-search="turns/2026-05-20-fix-alert-flow-packet-history.html turns">
<a class="doc-link" href="./turns/2026-05-20-fix-alert-flow-packet-history.html">turns/2026-05-20-fix-alert-flow-packet-history.html</a>
<div class="meta">
<span class="tag">html</span>
<span>14 KB</span>
<span>May 20, 2026, 9:26 PM</span>
</div>
</li>
@ -246,37 +286,7 @@
<div class="meta">
<span class="tag">html</span>
<span>9.8 KB</span>
<span>May 19, 2026, 2:48 PM</span>
</div>
</li>
<li class="doc-item" data-search="turns/2026-05-18-native-fast-iterative-deploy.html turns">
<a class="doc-link" href="./turns/2026-05-18-native-fast-iterative-deploy.html">turns/2026-05-18-native-fast-iterative-deploy.html</a>
<div class="meta">
<span class="tag">html</span>
<span>9.0 KB</span>
<span>May 19, 2026, 2:48 PM</span>
</div>
</li>
<li class="doc-item" data-search="turns/2026-05-19-0805-clarify-repo-turn-doc-rules.html turns">
<a class="doc-link" href="./turns/2026-05-19-0805-clarify-repo-turn-doc-rules.html">turns/2026-05-19-0805-clarify-repo-turn-doc-rules.html</a>
<div class="meta">
<span class="tag">html</span>
<span>6.4 KB</span>
<span>May 19, 2026, 8:05 AM</span>
</div>
</li>
<li class="doc-item" data-search="turns/2026-05-19-0739-update-readme-current-state.html turns">
<a class="doc-link" href="./turns/2026-05-19-0739-update-readme-current-state.html">turns/2026-05-19-0739-update-readme-current-state.html</a>
<div class="meta">
<span class="tag">html</span>
<span>9.8 KB</span>
<span>May 19, 2026, 7:39 AM</span>
<span>May 20, 2026, 9:26 PM</span>
</div>
</li>
@ -286,7 +296,87 @@
<div class="meta">
<span class="tag">html</span>
<span>9.0 KB</span>
<span>May 19, 2026, 7:31 AM</span>
<span>May 20, 2026, 9:26 PM</span>
</div>
</li>
<li class="doc-item" data-search="turns/2026-05-19-harden-native-ssh-deploy-checks.html turns">
<a class="doc-link" href="./turns/2026-05-19-harden-native-ssh-deploy-checks.html">turns/2026-05-19-harden-native-ssh-deploy-checks.html</a>
<div class="meta">
<span class="tag">html</span>
<span>7.0 KB</span>
<span>May 20, 2026, 9:26 PM</span>
</div>
</li>
<li class="doc-item" data-search="turns/2026-05-19-native-options-recovery-guardrails.html turns">
<a class="doc-link" href="./turns/2026-05-19-native-options-recovery-guardrails.html">turns/2026-05-19-native-options-recovery-guardrails.html</a>
<div class="meta">
<span class="tag">html</span>
<span>7.7 KB</span>
<span>May 20, 2026, 9:26 PM</span>
</div>
</li>
<li class="doc-item" data-search="turns/2026-05-19-publish-docs-pages-index.html turns">
<a class="doc-link" href="./turns/2026-05-19-publish-docs-pages-index.html">turns/2026-05-19-publish-docs-pages-index.html</a>
<div class="meta">
<span class="tag">html</span>
<span>6.7 KB</span>
<span>May 20, 2026, 9:26 PM</span>
</div>
</li>
<li class="doc-item" data-search="turns/2026-05-19-0739-update-readme-current-state.html turns">
<a class="doc-link" href="./turns/2026-05-19-0739-update-readme-current-state.html">turns/2026-05-19-0739-update-readme-current-state.html</a>
<div class="meta">
<span class="tag">html</span>
<span>9.8 KB</span>
<span>May 20, 2026, 9:26 PM</span>
</div>
</li>
<li class="doc-item" data-search="turns/2026-05-19-0805-clarify-repo-turn-doc-rules.html turns">
<a class="doc-link" href="./turns/2026-05-19-0805-clarify-repo-turn-doc-rules.html">turns/2026-05-19-0805-clarify-repo-turn-doc-rules.html</a>
<div class="meta">
<span class="tag">html</span>
<span>6.4 KB</span>
<span>May 20, 2026, 9:26 PM</span>
</div>
</li>
<li class="doc-item" data-search="turns/2026-05-19-fix-native-alpaca-news.html turns">
<a class="doc-link" href="./turns/2026-05-19-fix-native-alpaca-news.html">turns/2026-05-19-fix-native-alpaca-news.html</a>
<div class="meta">
<span class="tag">html</span>
<span>12 KB</span>
<span>May 20, 2026, 9:26 PM</span>
</div>
</li>
<li class="doc-item" data-search="turns/2026-05-18-native-fast-iterative-deploy.html turns">
<a class="doc-link" href="./turns/2026-05-18-native-fast-iterative-deploy.html">turns/2026-05-18-native-fast-iterative-deploy.html</a>
<div class="meta">
<span class="tag">html</span>
<span>9.0 KB</span>
<span>May 20, 2026, 9:26 PM</span>
</div>
</li>
<li class="doc-item" data-search="turns/2026-05-18-native-public-edge-cutover.html turns">
<a class="doc-link" href="./turns/2026-05-18-native-public-edge-cutover.html">turns/2026-05-18-native-public-edge-cutover.html</a>
<div class="meta">
<span class="tag">html</span>
<span>19 KB</span>
<span>May 20, 2026, 9:26 PM</span>
</div>
</li>
@ -296,7 +386,7 @@
<div class="meta">
<span class="tag">html</span>
<span>7.0 KB</span>
<span>May 18, 2026, 4:54 PM</span>
<span>May 20, 2026, 9:26 PM</span>
</div>
</li>
@ -513,7 +603,7 @@
<div class="meta">
<span class="tag">html</span>
<span>16 KB</span>
<span>May 19, 2026, 2:55 PM</span>
<span>May 20, 2026, 9:26 PM</span>
</div>
</li>
@ -522,15 +612,35 @@
<section class="group" id="category-general">
<h2>general <span>2</span></h2>
<h2>general <span>4</span></h2>
<ul class="doc-list">
<li class="doc-item" data-search="general/2026-05-22-standup-summary-2026-05-21.html general">
<a class="doc-link" href="./general/2026-05-22-standup-summary-2026-05-21.html">general/2026-05-22-standup-summary-2026-05-21.html</a>
<div class="meta">
<span class="tag">html</span>
<span>11 KB</span>
<span>May 22, 2026, 9:04 AM</span>
</div>
</li>
<li class="doc-item" data-search="general/2026-05-21-standup-summary-2026-05-20.html general">
<a class="doc-link" href="./general/2026-05-21-standup-summary-2026-05-20.html">general/2026-05-21-standup-summary-2026-05-20.html</a>
<div class="meta">
<span class="tag">html</span>
<span>16 KB</span>
<span>May 21, 2026, 9:05 AM</span>
</div>
</li>
<li class="doc-item" data-search="general/2026-05-18-standup-summary-2026-05-17.html general">
<a class="doc-link" href="./general/2026-05-18-standup-summary-2026-05-17.html">general/2026-05-18-standup-summary-2026-05-17.html</a>
<div class="meta">
<span class="tag">html</span>
<span>19 KB</span>
<span>May 18, 2026, 9:05 AM</span>
<span>May 20, 2026, 9:26 PM</span>
</div>
</li>
@ -557,7 +667,7 @@
<div class="meta">
<span class="tag">html</span>
<span>3.8 KB</span>
<span>May 19, 2026, 2:48 PM</span>
<span>May 20, 2026, 9:26 PM</span>
</div>
</li>
@ -576,9 +686,19 @@
<section class="group" id="category-root">
<h2>root <span>2</span></h2>
<h2>root <span>3</span></h2>
<ul class="doc-list">
<li class="doc-item" data-search="anatomy.html root">
<a class="doc-link" href="./anatomy.html">anatomy.html</a>
<div class="meta">
<span class="tag">html</span>
<span>33 KB</span>
<span>May 22, 2026, 10:22 PM</span>
</div>
</li>
<li class="doc-item" data-search="clickhouse-reset-runbook.md root">
<a class="doc-link" href="./clickhouse-reset-runbook.md">clickhouse-reset-runbook.md</a>
<div class="meta">

584
docs/turns/2026-05-22-add-options-anatomy-explainer.html Normal file
View file

@ -0,0 +1,584 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title>Turn Record: Add Options Anatomy Explainer</title>
<style>
:root {
color-scheme: dark;
--bg: #070a0d;
--surface: #10161d;
--surface-2: #131b23;
--surface-3: #19232d;
--ink: #e7edf4;
--muted: #96a5b7;
--faint: #6f7f92;
--line: rgba(255, 255, 255, 0.1);
--accent: #f5a623;
--accent-soft: rgba(245, 166, 35, 0.14);
--good: #25c17a;
--good-soft: rgba(37, 193, 122, 0.14);
--shadow: 0 28px 70px rgba(0, 0, 0, 0.38);
}
* {
box-sizing: border-box;
}
body {
margin: 0;
background:
radial-gradient(circle at top right, rgba(245, 166, 35, 0.08), transparent 26%),
linear-gradient(180deg, #0a0d11 0%, var(--bg) 100%);
color: var(--ink);
font: 16px/1.62 "IBM Plex Sans", "Avenir Next", "Segoe UI", sans-serif;
}
main {
width: min(1120px, calc(100vw - 40px));
margin: 28px auto 60px;
}
.hero,
.card {
border: 1px solid var(--line);
border-radius: 24px;
background:
linear-gradient(180deg, rgba(255, 255, 255, 0.03) 0%, rgba(255, 255, 255, 0.015) 100%),
var(--surface);
box-shadow: var(--shadow);
}
.hero {
padding: 30px;
}
.eyebrow,
h1,
h2,
h3,
code,
pre,
.meta-label {
font-family: "IBM Plex Mono", "SFMono-Regular", Consolas, monospace;
}
.eyebrow {
display: inline-flex;
align-items: center;
gap: 8px;
padding: 8px 12px;
border-radius: 999px;
background: var(--accent-soft);
color: var(--accent);
font-size: 0.78rem;
letter-spacing: 0.12em;
text-transform: uppercase;
}
h1,
h2,
h3 {
margin: 0;
line-height: 1.08;
}
h1 {
margin-top: 16px;
font-size: clamp(2.3rem, 4.8vw, 4.2rem);
letter-spacing: 0.04em;
text-transform: uppercase;
}
h2 {
font-size: 1.28rem;
letter-spacing: 0.04em;
text-transform: uppercase;
}
h3 {
font-size: 1rem;
}
.lede {
max-width: 72ch;
margin-top: 16px;
color: var(--muted);
}
.meta {
display: grid;
grid-template-columns: repeat(auto-fit, minmax(180px, 1fr));
gap: 14px;
margin-top: 24px;
}
.meta-card {
padding: 16px 18px;
border-radius: 16px;
border: 1px solid var(--line);
background: var(--surface-2);
}
.meta-label {
display: block;
color: var(--faint);
font-size: 0.72rem;
letter-spacing: 0.1em;
text-transform: uppercase;
}
.meta-value {
display: block;
margin-top: 8px;
font-weight: 650;
}
.grid {
display: grid;
gap: 18px;
margin-top: 20px;
}
.card {
padding: 24px;
}
p {
margin: 0;
}
p + p,
p + ul,
ul + p,
ul + ul {
margin-top: 12px;
}
ul {
margin: 0;
padding-left: 20px;
}
li + li {
margin-top: 8px;
}
.callout {
margin-top: 16px;
padding: 16px 18px;
border-radius: 16px;
border: 1px solid rgba(37, 193, 122, 0.2);
background: var(--good-soft);
}
.callout strong {
color: var(--good);
}
code {
padding: 0.08rem 0.35rem;
border-radius: 0.42rem;
background: rgba(255, 255, 255, 0.06);
}
pre {
margin: 12px 0 0;
padding: 16px;
border-radius: 16px;
overflow-x: auto;
background: #0a0f15;
border: 1px solid var(--line);
color: var(--ink);
font-size: 0.88rem;
line-height: 1.5;
}
.two-up {
display: grid;
grid-template-columns: repeat(auto-fit, minmax(280px, 1fr));
gap: 18px;
}
.small {
color: var(--muted);
font-size: 0.92rem;
}
.metrics {
display: grid;
grid-template-columns: repeat(auto-fit, minmax(160px, 1fr));
gap: 12px;
margin-top: 16px;
}
.metric {
padding: 14px 16px;
border-radius: 16px;
border: 1px solid var(--line);
background: var(--surface-3);
}
.metric strong {
display: block;
font: 700 1.18rem/1.15 "IBM Plex Sans", "Avenir Next", sans-serif;
}
.metric span {
color: var(--muted);
font-size: 0.9rem;
}
.diff-grid {
display: grid;
gap: 18px;
margin-top: 18px;
}
.diff-shell {
padding: 16px;
border-radius: 18px;
border: 1px solid var(--line);
background: linear-gradient(180deg, rgba(255, 255, 255, 0.025) 0%, rgba(255, 255, 255, 0.01) 100%);
}
.diff-shell h3 {
margin-bottom: 10px;
}
.diff-render {
min-height: 120px;
}
details {
margin-top: 12px;
}
summary {
cursor: pointer;
color: var(--accent);
font-weight: 600;
}
a {
color: var(--accent);
}
</style>
</head>
<body>
<main>
<section class="hero">
<span class="eyebrow">Turn Record · May 22, 2026</span>
<h1>Add Options Anatomy Explainer</h1>
<p class="lede">
Added a standalone <code>docs/anatomy.html</code> reference page that explains the
full lifecycle of an options print, from ingest and signal gating through flow packet
construction, smart-money scoring, classifier hits, alerts, and API/live consumption.
The page is styled to match Islandflow’s product register and layered so exec, mixed
technical, and operator-level readers can all use the same artifact.
</p>
<div class="meta">
<div class="meta-card">
<span class="meta-label">Beads</span>
<span class="meta-value"><code>islandflow-hpf</code></span>
</div>
<div class="meta-card">
<span class="meta-label">Artifact</span>
<span class="meta-value"><code>docs/anatomy.html</code></span>
</div>
<div class="meta-card">
<span class="meta-label">Register</span>
<span class="meta-value">Product, evidence-console styling</span>
</div>
<div class="meta-card">
<span class="meta-label">Secondary Change</span>
<span class="meta-value">Regenerated <code>docs/index.html</code></span>
</div>
</div>
</section>
<div class="grid">
<section class="card">
<h2>Summary</h2>
<p>
The repo now includes a reusable explainer page for one of the most important pieces of
Islandflow’s mental model: how a raw or synthetic options print turns into visible tape,
a flow packet, and sometimes a smart-money or alert event. Instead of scattering that
explanation across chat answers and source code, the new page centralizes the pipeline in
a designed HTML document that can be browsed directly under <code>docs/</code>.
</p>
<div class="callout">
<strong>Primary outcome:</strong> the new page makes the option-print pipeline legible at
three reading depths without forcing someone to reconstruct the architecture from service
code.
</div>
</section>
<section class="card">
<h2>Changes Made</h2>
<ul>
<li>
Added <code>docs/anatomy.html</code> as a standalone explainer page titled
<em>The Anatomy of an Options Print and Smart Money</em>.
</li>
<li>
Built a large flow-chart section that distinguishes the common tape path from the
signal-to-packet-to-smart-money branch.
</li>
<li>
Layered the page into executive, mixed technical, and operator-level explanations so
one artifact works for multiple audiences.
</li>
<li>
Included subject/table mapping, annotated sequence detail, synthetic-mode notes, and
code anchors back into the real repo.
</li>
<li>
Regenerated <code>docs/index.html</code> so the new explainer is discoverable from the
existing docs index.
</li>
</ul>
</section>
<section class="card">
<h2>Context</h2>
<p>
The user asked for a true flow-chart explanation of what happens when options tape comes
in under normal market scenarios and when smart-money behavior is detected, with the
important caveat that the current environment is using synthetic prints. The repo already
had the implementation details, but not a clear product artifact that unified ingest,
compute, storage, bus subjects, and API/live consumption into one readable document.
</p>
<p>
Because Islandflow’s UI language is already defined as an “evidence console,” the new
page needed to feel operational and precise rather than like a generic landing page or a
decorative infographic.
</p>
</section>
<section class="card">
<h2>Important Implementation Details</h2>
<div class="two-up">
<div>
<h3>Information architecture</h3>
<ul>
<li>
The page starts with a semantic legend and a visual flow board so readers can build
the correct mental model before diving into prose.
</li>
<li>
The explanation then deepens in three layers: executive read, mixed technical
walkthrough, and operator/code-level detail.
</li>
<li>
The normal tape path and the smart-money path are split explicitly so readers do not
confuse raw tape visibility with compute-derived inference.
</li>
</ul>
</div>
<div>
<h3>Design choices</h3>
<ul>
<li>
The visual treatment follows the repo’s product register: dark, stable, evidence-first,
amber used as a sparse signal, monospace labels for pipeline semantics.
</li>
<li>
The flow chart is pure HTML and CSS, not a JavaScript diagram dependency, so the
page remains portable and straightforward to keep in sync with the repo.
</li>
<li>
<code>docs/index.html</code> was regenerated with the existing script so the page
participates in the current docs navigation surface instead of becoming a hidden one-off.
</li>
</ul>
</div>
</div>
</section>
<section class="card">
<h2>Relevant Diff Snippets</h2>
<p class="small">
These snippets are rendered with the Diffs library from
<a href="https://diffs.com/docs">diffs.com</a>, with a plain-text fallback kept inline.
</p>
<div class="diff-grid">
<article class="diff-shell">
<h3><code>docs/anatomy.html</code>: new explainer page and flow-board structure</h3>
<div class="diff-render" id="diff-anatomy"></div>
<details>
<summary>Plain-text fallback</summary>
<pre>+ Added docs/anatomy.html
+ Product-register dark evidence-console styling
+ Main flow chart with common path, tape-only branch, and smart-money branch
+ Layered explanation sections for executive, mixed technical, and operator audiences
+ Subject map, annotated sequence, synthetic mode notes, and code anchors</pre>
</details>
</article>
<article class="diff-shell">
<h3><code>docs/index.html</code>: regenerated docs surface with new entry count</h3>
<div class="diff-render" id="diff-index"></div>
<details>
<summary>Plain-text fallback</summary>
<pre>- 35 files shown
+ 47 files shown
- root/general counts from prior docs set
+ updated counts after regenerating the index, including the new anatomy explainer entry</pre>
</details>
</article>
</div>
</section>
<section class="card">
<h2>Expected Impact for End-Users</h2>
<ul>
<li>
Teammates and operators now have a single place to understand why a print can appear on
tape without ever becoming a smart-money event.
</li>
<li>
The synthetic-print caveat is captured directly in the artifact, which should reduce
confusion when debugging or demoing the current environment.
</li>
<li>
The docs surface becomes more useful as a living product reference, not just a collection
of turn records and plans.
</li>
</ul>
</section>
<section class="card">
<h2>Validation</h2>
<ul>
<li>
Generated the new page at <code>docs/anatomy.html</code> and verified the title and
major sections are present.
</li>
<li>
Regenerated the docs index with
<code>node scripts/generate-docs-index.mjs</code>, which completed successfully and
reported <code>47 entries</code>.
</li>
<li>
Confirmed the new explainer page is included in the docs surface by regenerating
<code>docs/index.html</code>.
</li>
</ul>
<div class="metrics">
<div class="metric">
<strong>1</strong>
<span>new reusable explainer page</span>
</div>
<div class="metric">
<strong>47</strong>
<span>docs index entries after regeneration</span>
</div>
<div class="metric">
<strong>3</strong>
<span>reader depth layers on the page</span>
</div>
</div>
</section>
<section class="card">
<h2>Issues, Limitations, and Mitigations</h2>
<ul>
<li>
The page is intentionally hand-authored HTML rather than a generated diagram artifact.
That keeps it portable, but it also means future pipeline changes should update this page
manually.
</li>
<li>
The docs index regeneration reflects the full current <code>docs/</code> tree, so the
visible counts changed by more than one file compared with the previously committed
index.
</li>
<li>
This validation pass verified structure and generation success, but did not include a
browser-rendered visual QA step against multiple viewport sizes.
</li>
</ul>
</section>
<section class="card">
<h2>Follow-up Work</h2>
<ul>
<li>
Add reciprocal links from more domain-specific docs such as <code>smartmoney.md</code>
back to <code>docs/anatomy.html</code>.
</li>
<li>
Consider a second reference page focused specifically on one concrete synthetic example,
from a burst of prints to the final alert payload.
</li>
<li>
If the flow-packet feature set evolves, keep the anatomy page in lockstep with those
changes so it remains a trustworthy operator reference.
</li>
</ul>
</section>
</div>
<script type="module">
const snippets = {
anatomy: `diff --git a/docs/anatomy.html b/docs/anatomy.html
new file mode 100644
--- /dev/null
+++ b/docs/anatomy.html
@@
+<title>The Anatomy of an Options Print and Smart Money</title>
+<section class="hero">...</section>
+<section id="flow-chart">main flow board with common path and branch rows</section>
+<section id="executive">executive read</section>
+<section id="technical">mixed technical walkthrough</section>
+<section id="operator">operator and code-level detail</section>
+<section>subject map, annotated event sequence, synthetic mode notes, code anchors</section>`,
index: `diff --git a/docs/index.html b/docs/index.html
--- a/docs/index.html
+++ b/docs/index.html
@@
-<div class="stats"><strong id="visible-count">35</strong> of <strong>35</strong> files shown</div>
+<div class="stats"><strong id="visible-count">47</strong> of <strong>47</strong> files shown</div>
@@
-<a class="chip" href="#category-general">general <span>2</span></a>
+<a class="chip" href="#category-general">general <span>4</span></a>
@@
+<a class="doc-link" href="./anatomy.html">anatomy.html</a>`
};
const fallbackRender = (id, text) => {
const target = document.getElementById(id);
if (!target) {
return;
}
const pre = document.createElement("pre");
pre.textContent = text;
target.replaceChildren(pre);
};
try {
const { FileDiff } = await import("https://esm.sh/@pierre/diffs");
const targets = [
["diff-anatomy", snippets.anatomy],
["diff-index", snippets.index]
];
for (const [id, text] of targets) {
const mount = document.getElementById(id);
if (!mount) {
continue;
}
const diff = new FileDiff(text, { language: "diff" });
mount.appendChild(diff.element);
}
} catch (error) {
console.warn("Failed to load diffs.com renderer", error);
fallbackRender("diff-anatomy", snippets.anatomy);
fallbackRender("diff-index", snippets.index);
}
</script>
</main>
</body>
</html>