News Wire View via Alpaca Feed
Added an Alpaca-backed live news pipeline end to end: normalized NewsStory types,
a dedicated JetStream subject/stream, ClickHouse storage helpers with latest-revision semantics,
a new services/ingest-news service, API endpoints and live fanout, and a web
/news route plus Home preview with a right-side story drawer.
Changes Made
- Added
NewsStorySchema, thenewslive channel, and subscription parsing support inpackages/types. - Added bus constants for the
flow.newssubject andNEWSstream. - Added ClickHouse news storage helpers, including recent, before-cursor, and after-cursor queries that collapse provider revisions to the latest row per
provider + story_id. - Created
services/ingest-newswith Alpaca REST backfill, Alpaca websocket streaming, normalization, and deterministic ticker resolution. - Extended the API service to persist live news in the shared cache, expose
GET /newsandGET /history/news, and fan outnewsevents on/ws/live. - Added a top-level
/newsroute, primary nav entry, Home preview pane, replay-mode live-only empty states, and a sanitized full-story drawer. - Updated dev and deployment wiring so the new service is included in local runners and the Docker workspace snapshot.
Context
The plan called for a free-provider v1 news surface that behaves like the rest of Islandflow: compact, evidence-first, and live-native. The implementation keeps replay intentionally out of scope for news while still integrating news into the same live manifest, history pagination, rail navigation, and drawer language used elsewhere in the terminal.
Important Implementation Details
- Ticker resolution prefers provider symbols first, then falls back only to structured patterns in provider HTML: ticker anchors,
EXCHANGE:SYM, and$SYM. - News history uses
published_tsas the visible cursor while revisions are collapsed with a window function overprovider, story_idordered byupdated_ts,ingest_ts, andseq. - The web drawer sanitizes provider HTML by removing scripts, inline event handlers, and unsupported tags; if sanitization yields nothing useful, the drawer falls back to stripped plain text.
- Replay mode intentionally renders a clear empty state for news on both Home and
/newsinstead of pretending news is replay-synced.
resolved_symbols = provider_symbols
or ticker anchors in content_html
or EXCHANGE:SYM matches
or $SYM matches
Expected Impact for End-Users
Traders can now monitor a dedicated live news wire inside Islandflow, spot symbol-linked headlines from the Home view, and open full stories in-context without leaving the app. The displayed ticker chips are grounded in stored provider and derived symbol metadata, which makes the feed safer to filter and trust.
Validation
- Ran targeted Bun tests covering types, storage, API live-state behavior, ingest-news symbol resolution, route wiring, and terminal helpers.
- Built the Next.js web app with
bun --cwd=apps/web run build. - Ran
bun run check:docker-workspaceafter syncing the deployment workspace snapshot.
Issues, Limitations, and Mitigations
- Replay support remains intentionally absent in v1; the UI now states that explicitly instead of showing misleading empty historical behavior.
- The sanitizer is intentionally conservative and custom, which keeps dependencies light but may strip some harmless provider formatting.
- The ingest service assumes Alpaca’s current REST and websocket news contracts; if Alpaca changes those payload shapes, the normalization layer will need adjustment.
Follow-up Work
- No additional follow-up issue was required during this turn.
- Future extensions are still available behind the same contract: multi-provider aggregation, server-side symbol filtering, and replay-aware news history.