Simplified Overview
The current synthetic system is strong at coverage and weaker at realism. It can produce the categories, but the tape often reveals the machinery: option bursts appear on a rhythm, quotes are too consistently clean, equities and options are only loosely related, and the market context around a labeled event is not convincing enough.
The redesign introduces a shared market regime engine. Instead of “emit a category-shaped burst now,” the system will model a believable session state first, then let both options and equities express that state. That keeps the smart-money demo behavior while making the experience feel more grounded.
Scope
In scope
- Hosted synthetic regime engine
- Options and equities generator redesign
- Hidden subtype scenario families
- Soft coverage logic
- Internal control API
- Internal control drawer in the terminal shell
- Regression and realism tests
Out of scope
- Changing public smart-money categories
- General settings-page work
- User profile or token-spend UI
- Public simulator controls
- Live-feed product changes
Services Affected
| Area | Files | Why they change |
|---|---|---|
| Shared types and regime model | packages/types/src/synthetic-market.ts, packages/types/src/events.ts |
Introduce the control-state model and the deterministic shared market regime. |
| Hosted API | services/api/src/index.ts |
Add internal synthetic-control status and mutation endpoints. |
| Options ingest | services/ingest-options/src/index.ts, services/ingest-options/src/adapters/synthetic.ts |
Swap burst scheduling for regime-driven scenario selection and coverage debt. |
| Equities ingest | services/ingest-equities/src/index.ts, services/ingest-equities/src/adapters/synthetic.ts |
Make equity prints and quotes react to the same latent regime as the options side. |
| Web and Electron shell | apps/web/app/terminal.tsx, apps/web/app/api/admin/synthetic/* |
Add the internal-only gear trigger, drawer, and secure proxy layer. |
| Tests | Options tests, API tests, web tests | Protect determinism, realism, UI visibility rules, and classification alignment. |
Locked Decisions
Full Architecture
1. Replace the burst pulse with a shared regime engine
Expand packages/types/src/synthetic-market.ts into the shared deterministic engine used by both ingest services.
Shared functions:
getSyntheticSessionState(ts, control)getSyntheticUnderlyingState(symbol, ts, control, sessionState)getSyntheticScenarioWeights(symbol, ts, control, sessionState)getSyntheticCoverageBoost(profileId, coverageState, control)
sessionState includes:
session_phase:open | midday | power_hour | after_eventregime:trend_up | trend_down | mean_revert | retail_chase | event_ramp | dealer_gamma | arb_calmvolatility_levelliquidity_levelquote_cleanlinessfocus_symbolsevent_symbolsseed_bucket
2. Add hosted synthetic control state
Add internal control schemas in packages/types:
SyntheticControlPresetIdSyntheticControlStateSyntheticProfileWeightMapSyntheticCoverageConfigSyntheticDerivedStatus
type SyntheticControlState = {
preset_id: "balanced_demo" | "event_day" | "dealer_day" | "retail_chase" | "quiet_range";
coverage_assist: boolean;
coverage_window_minutes: 10 | 20 | 30;
shared_seed: number;
profile_weights: {
institutional_directional: 0.6 | 1.0 | 1.6;
retail_whale: 0.6 | 1.0 | 1.6;
event_driven: 0.6 | 1.0 | 1.6;
vol_seller: 0.6 | 1.0 | 1.6;
arbitrage: 0.6 | 1.0 | 1.6;
hedge_reactive: 0.6 | 1.0 | 1.6;
};
updated_at: number;
updated_by: string;
};
Defaults: preset_id: balanced_demo, coverage_assist: true, coverage_window_minutes: 20, all profile weights 1.0.
3. Persist and distribute control state through NATS
- Use JetStream KV bucket
synthetic_control - Use key
global services/apireads and writes the KV entryservices/ingest-optionsloads on boot and watches for updatesservices/ingest-equitiesdoes the same
4. Rebuild options scenarios as hidden subtype families
institutional_directional:call_sweep,put_sweep,ask_lift_accumulation,far_dated_convictionretail_whale:0dte_call_chase,short_dated_put_panic,attention_contract_spikeevent_driven:earnings_vol_probe,pre_event_directional_ramp,post_gap_followthroughvol_seller:covered_call_overwrite,cash_secured_put_write,short_straddle_harvestarbitrage:parity_vertical,conversion_reversal,box_spreadhedge_reactive:gamma_pinch_call_hedge,reactive_put_wall,dealer_unwindneutral_noise:single_print_mid,two_sided_scalp,stale_quote_noise
Hidden subtype labels remain internal and test-only. They should never appear on emitted option prints or public smart-money events.
5. Make equities and options react to the same latent state
Equities changes
- Remove the fixed dark-sequence loop
- Make lit versus dark balance regime-dependent
- Make spread, quote cleanliness, off-exchange frequency, and clustering regime-dependent
- Use shared focus symbols
- Make
event_rampandretail_chaseshow modest trend and wider quotes - Make
dealer_gammashow choppier reversals and denser quote changes - Make
arb_calmquieter and more neutral
Options changes
- Replace hardcoded coverage forcing with weighted family selection plus coverage debt
- Make venue count, placement, stale or missing quote probability, and structure prevalence regime-sensitive
- Derive
execution_iv_shock,underlying_move_bps, andnbbo_spread_zfrom shared state - Generate event-driven timestamps and symbols from shared regime state
6. Add soft coverage accounting
- Track rolling coverage debt per public profile inside each ingest service
- Maintain a rolling counter across the selected
coverage_window_minutes - Only public profiles count toward coverage
- Missing profiles get a bounded weight boost
- Noise and low-key scenarios continue to appear between labeled bursts
7. Add internal hosted control endpoints
Add routes in services/api/src/index.ts:
GET /admin/synthetic/statusGET /admin/synthetic/controlPUT /admin/synthetic/control
{
enabled: boolean;
backend_mode: "synthetic" | "mixed" | "live";
adapters: {
options: string;
equities: string;
};
control: SyntheticControlState | null;
derived: {
session_phase: string;
regime: string;
focus_symbols: string[];
profile_hit_counts: Record<SmartMoneyProfileId, number>;
coverage_window_minutes: number;
} | null;
disabled_reason?: string;
}
Behavior: return 404 when admin mode is disabled, return 409 when hosted adapters are not synthetic, validate full payloads on PUT, and keep public smart-money interfaces unchanged.
8. Keep secrets out of the browser with Next.js proxy routes
apps/web/app/api/admin/synthetic/status/route.tsapps/web/app/api/admin/synthetic/control/route.ts
The proxy reads server-only SYNTHETIC_ADMIN_TOKEN, forwards to NEXT_PUBLIC_API_URL, returns 404 when the internal UI flag is off, and never exposes the token client-side.
9. Add an internal control surface
UI rules for the first pass:
- Small floating gear in the bottom-right corner
- Opens a right-edge non-modal drawer
- Internal-only visibility
- Preset dropdown:
Balanced Demo,Event Day,Dealer Day,Retail Chase,Quiet Range - Coverage assist toggle
- Coverage window selector:
10m,20m,30m - Six profile-weight controls:
Low,Normal,High - Read-only live status: regime, session phase, focus symbols, rolling hit counts, backend state
- Optimistic updates with rollback on error
- Debounced writes at
250ms - Status polling every
5s, no admin websocket in v1
Interfaces and Environment
Public contracts unchanged
SmartMoneyProfileIdSmartMoneyEvent/flow/smart-money/history/smart-money/replay/smart-money/ws/smart-money
New internal contracts
SyntheticControlStateSyntheticControlPresetIdSyntheticDerivedStatus
New internal endpoints
GET /admin/synthetic/statusGET /admin/synthetic/controlPUT /admin/synthetic/control
New env vars
Backend
SYNTHETIC_CONTROL_ENABLED=0|1SYNTHETIC_ADMIN_TOKEN=...
Web
NEXT_PUBLIC_SYNTHETIC_ADMIN=0|1SYNTHETIC_ADMIN_TOKEN=...for the Next server proxy only
Implementation Phases
-
Phase 1. Shared types and regime engine
Touch
packages/types/src/synthetic-market.tsand related exports and tests. Deliver control schemas, preset definitions, deterministic session and regime functions, and coverage boost helpers. -
Phase 2. Hosted control plane
Touch
services/api/src/index.tsand NATS or KV helpers as needed. Deliver admin endpoints, KV persistence, status payloads, and disabled or error behavior. -
Phase 3. Ingest service coupling
Touch both ingest services and their synthetic adapters. Deliver boot-time control loading, KV watch updates, shared regime-driven generation, and removal of visibly scripted fixed sequences.
-
Phase 4. Internal control UI
Touch
apps/web/app/terminal.tsxand the internal admin proxy routes. Deliver the floating gear, non-modal drawer, polling, optimistic updates, and disabled state. -
Phase 5. Regression and realism tests
Deliver determinism tests, control API tests, scenario coverage tests, UI visibility tests, and classifier-alignment tests for hidden subtype families.
Tests and Acceptance
Shared engine
- Same
timestamp + control snapshot + seedyields the same regime and focus symbols in both ingest services. - Presets materially change regime weights without breaking determinism.
balanced_demoyields mixed regimes over a session.quiet_rangeyields lower volatility, tighter spreads, and fewer labeled events thanretail_chase.
Cross-asset coupling
event_rampproduces event-aligned option scenarios and synchronized underlying drift and spread behavior.dealer_gammaproduces short-dated ATM-heavy options plus choppier underlying reversals.arb_calmincreases neutral multi-leg structures without strong directional underlying moves.retail_chaseincreases short-dated OTM call behavior, IV shock, and louder underlying momentum.
Coverage and classification
- With default controls, every public smart-money profile appears at least once in a 20-minute synthetic session sample.
- With
coverage_assist=false, there is no forced coverage logic. - Raising one profile to
Highincreases its frequency without starving other categories. - Neutral noise remains below the smart-money emission threshold.
- Each hidden subtype family still classifies into the intended public profile.
Admin API and UI
- Disabled admin mode returns
404. - Non-synthetic hosted mode returns
409with a useful reason. - Valid
PUTpersists to KV and becomes visible to both ingest services. - The floating gear is hidden when
NEXT_PUBLIC_SYNTHETIC_ADMINis off. - The browser client never receives the backend admin token.
Assumptions and Defaults
- Hosted synthetic control applies only when both options and equities ingest adapters are synthetic.
- No general settings page, user-info work, or token-spend work is in scope here.
- Hidden subtype labels remain internal and test-only and never attach to emitted prints.
- The first pass uses polling for admin status rather than a new admin websocket.
- The default operator experience is
Balanced Demowith soft coverage on and a 20-minute window. - The repo currently lacks local
PRODUCT.md,DESIGN.md, and the local impeccable loader path. This version therefore follows the spirit of the terminal shell and impeccable product-UI principles rather than project-specific design-context files.