Touched deployment/docker/workspace-root/package.json with one-line change.
+ Added one line to deployment/docker/workspace-root/package.json. The local git history does not
+ show more context beyond the file touch and commit subject.
+
-
-
- Expanded
scripts/deploy.tsfor native deploy runtime behavior.
- - Added native user-unit templates and rollback/health tooling in
deployment/native/.
- - Added associated plan and turn documents in
docs/plansanddocs/turns.
-
Removed duplicate import in services/api/src/index.ts and added a turn doc.
Added docs/general/2026-05-18-standup-summary-2026-05-17.html and updated beads state.
+ Added the prior day’s report at docs/general/2026-05-18-standup-summary-2026-05-17.html and
+ updated .beads/issues.jsonl.
+
-
-
- Created
services/ingest-newsand wired Alpaca backfill/websocket ingestion.
- - Added news types/storage contracts in
packages/typesandpackages/storage.
- - Extended API live/history endpoints and web terminal/news route rendering. +
-
+ Added a new ingest service in
services/ingest-news/src/index.tsthat backfills Alpaca news, + subscribes to the Alpaca news websocket, resolves symbols, and publishesNewsStorypayloads to + NATS. +
+ -
+ Extended shared contracts in
packages/types/src/events.tsand +packages/types/src/live.ts, plus new storage support in +packages/storage/src/news.tsandpackages/storage/src/clickhouse.ts. +
+ -
+ Wired the API to store, fan out, and expose news via
/newsand/history/newsin +services/api/src/index.tsand live-session updates inservices/api/src/live.ts. +
+ -
+ Added a web route in
apps/web/app/news/page.tsx, a news pane and drawer in +apps/web/app/terminal.tsx, and related styling inapps/web/app/globals.css. +
+ -
+ Updated runtime packaging and local/dev deployment surfaces, including
+
deployment/docker/docker-compose.yml, Dockerfiles,scripts/dev.ts, and +scripts/deploy.ts. +
+ -
+ Added tests in
packages/storage/tests/news.test.ts, +services/ingest-news/tests/symbols.test.ts, and adjusted +apps/web/app/terminal.test.tspluspackages/types/tests/live.test.ts. +
-
-
- Added native infra system units and scripts for bootstrap/start/stop/cutover/full rollback. -
- Updated deploy docs and runtime config files under
deployment/native/config.
- - Added turn doc
docs/turns/2026-05-18-native-public-edge-cutover.html.
-
Merged native-deploy into main within the May 18 US/Eastern day window.
Updated repository-level instructions in AGENTS.md.
+ Edited AGENTS.md to update turn-document and beads workflow guidance.
+
Context
- The earlier report was generated before merged history included the native deployment branch on main.
- This recreation uses git log --all over the same date window, so it captures both feature work and
- merged operational/deployment work visible after PR merge.
+ This summary is based on local git history between 2026-05-18 00:00 -0400 and
+ 2026-05-19 00:00 -0400. The repository uses Bun, TypeScript, NATS/JetStream, ClickHouse, and a Next.js
+ web app, so the main feature commit spans service ingestion, shared types, persistence, API delivery, and the UI.
Important Implementation Details
-News wire ingestion and delivery path
+News ingestion was introduced as a first-class service
- The news pipeline added a new ingest service and API fanout channel, then exposed UI surfaces in
- /news and terminal panes.
+ services/ingest-news/src/index.ts authenticates against Alpaca, backfills recent news, subscribes
+ to live updates, resolves symbols, validates payloads with NewsStorySchema, and publishes them onto
+ the repo’s bus layer.
+
const backfill = await fetchBackfill();
+for (const item of backfill.reverse()) {
+ await publishStory(item);
+}
+
+if (msg === "authenticated") {
+ ws.send(JSON.stringify({ action: "subscribe", news: ["*"] }));
+}API and live session support were expanded for news
+
+ services/api/src/index.ts now ensures the news table exists, subscribes to a news consumer, fans
+ out live updates, and exposes both recent and paginated history endpoints.
+
if (req.method === "GET" && url.pathname === "/news") {
+ const limit = parseLimit(url.searchParams.get("limit") ?? "100");
+ const data = await fetchRecentNews(clickhouse, limit);
+ return jsonResponse({ data });
+}The web terminal gained a dedicated news surface
+
+ apps/web/app/terminal.tsx added a live-only news pane, a per-story drawer, history loading, and a
+ new /news route entry point via apps/web/app/news/page.tsx.
if (features.news) {
subscriptions.push({ channel: "news", snapshot_limit: LIVE_OPTIONS_HEAD_LIMIT });
+}
+
+export function NewsRoute() {
+ const state = useTerminal();
+ return (
+ <PageFrame title="News">
+ <div className="page-grid page-grid-news">
+ <NewsPane state={state} className="news-pane-full" />
+ </div>
+ </PageFrame>
+ );
}Native deployment hardening
-
- Deployment scripts and unit templates now include direct scripts for cutover and rollback, with infra and
- service checks under deployment/native/.
-
deployment/native/cutover.sh
-deployment/native/full-rollback.sh
-deployment/native/install-infra-units.shMerged history effect on standup scope
-- The merged view increased the standup scope from 4 to 8 commits and from 35 to 68 unique files touched for the - same local-day window. -
-Expected Impact for End-Users
-Live news wire data is now available in terminal surfaces alongside existing market/event feeds.
++ Users now have a dedicated news wire surface in the web terminal, including summary rows, story details, and + a direct link to the source article. +
Native deployment and rollback procedures now have first-class scripted and documented paths.
++ News is now available alongside the repo’s existing live feeds, with shared symbol resolution and storage that + make the data retrievable through API history endpoints. +
This standup report now matches merged repository history instead of pre-merge branch-local history.
++ The UI copy in the news pane explicitly marks news as live-only in v1, so replay users should not expect the + same behavior there yet. +
Validation
-
-
- Used
git fetch --all --prunebefore recomputing history.
- - Used
git log --allover the May 18 ET window to include merged commits.
- - Used
git log --stat --summaryand--numstatto ground file and line-count statements.
- - No build/test commands were run because this task only regenerates reporting documentation. +
- Reviewed local git history with
git log --since='2026-05-18 00:00' --until='2026-05-19 00:00'.
+ - Used
git log --stat,git show, and file-level history to anchor each summary item to specific commits and files.
+ - No builds or tests were run for this reporting task because the work product is a git summary document, not a behavior change.
Issues, Limitations, and Mitigations
-
-
- This report describes commit history only and does not infer intent beyond commit messages and touched files. -
- Commit
8f0794ddis timestamped in UTC; it still falls on May 18 in US/Eastern, so it is included.
- - Metrics are based on local git history at regeneration time and can change if additional backdated commits appear. +
- + This report is grounded in local commit metadata only. No pull request identifiers were present in the inspected + git history, so the summary references commits and files instead of PR numbers. + +
-
+ The
update beadscommit touched onlydeployment/docker/workspace-root/package.jsonin + visible git output, so this report does not infer intent beyond that recorded file change. +
+ - + Counts here describe May 18 commits only and exclude any uncommitted work present after that date. +
Follow-up Work
-
-
- This regeneration is tracked by beads issue
islandflow-0ty.
- - No additional follow-up work was identified during this documentation-only task. +
-
+ No new product follow-up items were derived from this reporting pass. The only beads item created for this task
+ is
islandflow-2df, which tracks publication of this summary document. +
Islandflow docs index
-A browsable index of files under docs/ with filtering and grouped navigation.
turns 28
--
-
-
- - turns/2026-05-19-publish-docs-pages-index.html - - - - -
- - turns/2026-05-18-native-public-edge-cutover.html - - - - -
- - turns/2026-05-19-reconcile-pr-conflicts.html - - - - -
- - turns/2026-05-18-native-fast-iterative-deploy.html - - - - -
- - turns/2026-05-19-0805-clarify-repo-turn-doc-rules.html - - - - -
- - turns/2026-05-19-0739-update-readme-current-state.html - - - - -
- - turns/2026-05-19-upgrade-nextjs-16.html - - - - -
- - turns/2026-05-18-news-wire-view.html - - - - -
- - turns/2026-05-17-forgejo-deploy-remote-resolution.html - - - - -
- - turns/2026-05-17-add-fast-deploy-mode.html - - - - -
- - turns/2026-05-17-1101-clickhouse-alert-context.html - - - - -
- - turns/2026-05-17-clickhouse-alert-context.html - - - - -
- - turns/2026-05-17-deploy-allowlist-pr-packaging.html - - - - -
- - turns/2026-05-17-configure-beads-dolt-remote.html - - - - -
- - turns/2026-05-16-live-tape-scroll-hold-history.html - - - - -
- - turns/2026-05-17-0331-fix-live-tape-scroll-stability.html - - - - -
- - turns/2026-05-16-1725-durable-options-tape-history.html - - - - -
- - turns/2026-05-16-1752-speed-up-docker-deploys.html - - - - -
- - turns/2026-05-16-2159-fix-durable-options-history-routing.html - - - - -
- - turns/2026-05-15-add-duplicate-vps-compose-warning.html - - - - -
- - turns/2026-05-15-clarify-docker-first-deploy-workflow.html - - - - -
- - turns/2026-05-15-dual-runtime-deploy-workflow.html - - - - -
- - turns/2026-05-15-deploy-preflight-docker-workspace-check.html - - - - -
- - turns/2026-05-15-fix-docker-workspace-lockfile-sync.html - - - - -
- - turns/2026-05-15-quiet-terminal-view.html - - - - -
- - turns/2026-05-14-1824-adapt-terminal-view.html - - - - -
- - turns/2026-05-14-1833-reconcile-pr-conflicts.html - - - - -
- - turns/2026-05-14-harden-terminal-view.html - - - -
daily-git 1
--
-
-
- - daily-git/2026-05-19-standup-summary-2026-05-18.html - - - -
general 2
--
-
-
- - general/2026-05-18-standup-summary-2026-05-17.html - - - - -
- - general/2026-05-17-standup-summary-2026-05-16.html - - - -
plans 2
- -root 2
--
-
-
- - clickhouse-reset-runbook.md - - - - -
- - terminal-audit-2026-05-14-0432.html - - - -
No files match that filter.
-Harden Native SSH Deploy Checks
-- Native deploys over SSH were failing for avoidable operator reasons: the remote shell did not inherit Bun's install path, and native verification assumed it was already running from the repository root before it called checked-in health scripts. This patch makes the SSH path more forgiving and fixes the verification working directory. -
- -Summary
-
- Updated scripts/deploy.ts so native SSH deploys prepend $HOME/.bun/bin when it exists, and native verification now explicitly cds into the remote repo before running the checked-in health helpers.
-
Changes Made
--
-
- Prepended
$HOME/.bun/binduring native remote precheck when available.
- - Prepended
$HOME/.bun/binduring native remote rollout when available.
- - Changed native remote verification to run from
/home/delta/islandflowbefore callingdeployment/native/check-native-infra.sh.
-
Context
-
- During a live native rollout, the deploy helper failed first because the non-login SSH shell could not find bun even though it was installed under the deploy user's home directory. After that was corrected on the host, worker rollout still reported failure because remote verification executed from the home directory and could not resolve the relative path to the checked-in infra check script.
-
Important Implementation Details
--
-
- The fallback only adjusts
PATHwhen$HOME/.bun/bin/bunexists, so it stays harmless on hosts that already expose Bun globally.
- - The repo-root
cdkeeps the existing relative helper calls intact instead of hardcoding every individual script path in multiple places.
- - This change improves SSH-based deploys without changing local-server deploy behavior. -
Relevant Diff Snippets
-Unified diff blocks below are formatted for diffs-compatible rendering.
-diff --git a/scripts/deploy.ts b/scripts/deploy.ts
-@@ -754,6 +754,10 @@ set -euo pipefail
-
- cd ${shellEscape(REMOTE_REPO)}
-
-+if [[ -x "$HOME/.bun/bin/bun" ]]; then
-+ export PATH="$HOME/.bun/bin:$PATH"
-+fi
-+
- if ! command -v bun >/dev/null 2>&1; then
-
-@@ -855,6 +859,10 @@ set -euo pipefail
-
-+if [[ -x "$HOME/.bun/bin/bun" ]]; then
-+ export PATH="$HOME/.bun/bin:$PATH"
-+fi
-+
- ${remoteGitUpdateScript(mode, remote, branch)}
-
-@@ -943,6 +951,12 @@ set -euo pipefail
-
-+cd ${shellEscape(REMOTE_REPO)}
-+
-+if [[ -x "$HOME/.bun/bin/bun" ]]; then
-+ export PATH="$HOME/.bun/bin:$PATH"
-+fi
-+
- declare -a units=(${units})Expected Impact for End-Users
-- End users should see fewer failed native deploy attempts and fewer partial restarts caused by tooling assumptions rather than application health. This lowers the odds of avoidable downtime during native rollouts. -
-Validation
--
-
- Observed the original failures during live rollout: missing
bunin SSH PATH and missingdeployment/native/check-native-infra.shduring remote verification.
- - Used the patched operational path to complete native worker, API, and web rollouts successfully on the VPS. -
- Verified API health at
http://127.0.0.1:4000/healthand web health at bothhttp://127.0.0.1:3000/andhttps://flow.deltaisland.io.
-
Issues, Limitations, and Mitigations
--
-
- This patch does not solve the separate
ingest-newscredential problem. Full native deploys still need that unit and provider path to be made healthy before they are completely clean.
- - The VPS also needed a host-level Bun symlink during this recovery. The repo patch reduces dependence on that fix for future SSH deploys but does not remove it retroactively. -
Follow-up Work
--
-
islandflow-fmg: Keep the deploy helper aligned with the actual VPS runtime assumptions and add regression checks around native verification paths.
- islandflow-wf5: Decide whetheringest-newsand live options should stay provider-backed or remain intentionally synthetic until auth is hardened.
-
Native Options Recovery Guardrails
-
- The production outage turned out to be a native deployment config mismatch, not a data-pipeline code failure. I restored the VPS to the last known-good synthetic options adapter, then tightened the checked-in native deployment assets so they no longer imply a systemd override will beat the repo .env.
-
Summary
-
- The repo-side change is small and targeted: remove the misleading Environment=OPTIONS_INGEST_ADAPTER=synthetic line from the checked-in native ingest-options unit, and document that Bun-launched services effectively take adapter selection from /home/delta/islandflow/.env.
-
Changes Made
--
-
- Removed the checked-in systemd override from
deployment/native/systemd/user/islandflow-ingest-options.service.
- - Added an explicit env-precedence warning to
deployment/native/README.md.
- - Captured the live diagnosis that the native server had drifted to
OPTIONS_INGEST_ADAPTER=alpacawhile the prior Docker deployment was running synthetic options.
-
Context
-
- On the VPS, islandflow-ingest-options.service was crash-looping with repeated 401 Unauthorized responses from Alpaca while the rest of the native stack stayed healthy. The previous Docker-owned islandflow-vps-ingest-options-1 container showed OPTIONS_INGEST_ADAPTER=synthetic, which explains why the UI had been healthy before the runtime transition.
-
Important Implementation Details
--
-
- The checked-in unit already referenced
/home/delta/islandflow/.env, and Bun's runtime env loading meant a conflicting adapter value there still won in practice.
- - The static key currently stored as
ALPACA_API_KEYdoes not authenticate the failing market-data snapshot request as a Bearer token.
- - Because the real outage fix required a server-side
.envcorrection, this repo patch focuses on preventing operator confusion during the next native cutover.
-
Relevant Diff Snippets
-Unified diff blocks below are formatted for diffs-compatible rendering.
-diff --git a/deployment/native/README.md b/deployment/native/README.md
-@@ -98,6 +98,8 @@ These are written for the current VPS layout:
- - Bun binary: `/home/delta/.bun/bin/bun`
- - env file: `/home/delta/islandflow/.env`
-
-+Important: treat `/home/delta/islandflow/.env` as the effective source of truth for adapter selection. The Bun-launched services read that file directly at runtime, so a conflicting `OPTIONS_INGEST_ADAPTER` value in `.env` can still win over a systemd-only override and push `ingest-options` onto the wrong provider path.
-+
- ### Install the units
-
-diff --git a/deployment/native/systemd/user/islandflow-ingest-options.service b/deployment/native/systemd/user/islandflow-ingest-options.service
-@@ -7,7 +7,6 @@ Wants=network-online.target
- Type=simple
- WorkingDirectory=/home/delta/islandflow
- EnvironmentFile=/home/delta/islandflow/.env
--Environment=OPTIONS_INGEST_ADAPTER=synthetic
- ExecStart=/home/delta/.bun/bin/bun services/ingest-options/src/index.tsExpected Impact for End-Users
-- End users should not see the options tape stall the next time native units are installed or audited by following the checked-in assets. Operators now have a clearer paper trail that the actual runtime adapter comes from the deployment env file. -
-Validation
--
-
- Verified the native outage mode on the VPS:
islandflow-ingest-options.servicecrash-looped on Alpaca401responses.
- - Confirmed the previous Docker container had been running
OPTIONS_INGEST_ADAPTER=synthetic.
- - After the server-side env fix, confirmed fresh rows in
default.option_printsand new compute emissions in the native logs.
- - Ran
git diffto verify the repo change stayed scoped to the deployment README and the checked-in user unit.
-
Issues, Limitations, and Mitigations
--
-
- The repo patch does not add new credential support for Alpaca. It only documents the current env-precedence behavior and removes a misleading override. -
- The live server is restored with synthetic options, which matches the last known-good Docker behavior, but it is not a true live Alpaca ingest path. -
Follow-up Work
--
-
islandflow-wf5: Decide whether production options should remain synthetic or move to a fully supported live provider configuration.
- islandflow-wf5: If Alpaca live data is still desired, add a validated auth flow and a deploy-time smoke test that catches provider auth failures before cutover.
-
Publish docs/ to GitHub Pages with navigable index
- - -Summary
-
- Added an automated docs publishing flow to GitHub Pages and generated a new
- docs/index.html browsing experience so docs are easy to navigate at
- /islandflow/docs/.
-
Changes Made
--
-
- Added
scripts/generate-docs-index.mjsto build a browsable index of files underdocs/.
- - Added
.github/workflows/docs-pages.ymlto publish docs to GitHub Pages on pushes tomain.
- - Generated
docs/index.htmlfrom current docs content.
- - Configured deployment artifact layout so docs are available at
/docs/under the project Pages site.
-
Context
-
- The repository already stores operational and implementation documentation under
- docs/, but there was no dedicated GitHub Pages pipeline and no curated
- index page for discovery. This task focused on syncing that folder to Pages and
- making it easy to browse by category and filename.
-
Important Implementation Details
--
-
- The index generator excludes hidden files and avoids self-including
docs/index.html.
- - Files are grouped by first path segment (
turns,general,plans, and others) with quick category chips.
- - The index includes client-side filtering so users can search docs by path text in-browser. -
- Pages deployment packages a
site/payload where docs are copied intosite/docsand root redirects to./docs/.
-
Relevant Diff Snippets
-- Snippets are shown in a compact style aligned with diffs.com presentation patterns. -
-+++ .github/workflows/docs-pages.yml
-name: Publish Docs
-on:
- push:
- branches: [main]
- paths:
- - "docs/**"
- - "scripts/generate-docs-index.mjs"
- - ".github/workflows/docs-pages.yml"
- workflow_dispatch:
-
-jobs:
- build:
- steps:
- - uses: actions/checkout@v4
- - uses: actions/configure-pages@v5
- - run: node scripts/generate-docs-index.mjs
- - run: cp -R docs/. site/docs/
- - uses: actions/upload-pages-artifact@v3
- deploy:
- needs: build
- steps:
- - uses: actions/deploy-pages@v4+++ scripts/generate-docs-index.mjs
-const files = await collectDocsFiles(docsDir);
-const html = renderDocument(files);
-await fs.writeFile(outputFile, html, "utf8");
-
-// Generated index features:
-// - grouped sections
-// - search filter
-// - file size and modified time metadata
-// - links preserving docs folder structureExpected Impact for End-Users
--
-
- Docs are reachable via a stable Pages URL path:
dirtydishes.github.io/islandflow/docs/.
- - Readers can quickly scan categories and search by filename instead of relying on raw directory browsing. -
- New docs added to the repository are published automatically on
mainpushes.
-
Validation
--
-
- Ran
node scripts/generate-docs-index.mjssuccessfully.
- - Ran
node --check scripts/generate-docs-index.mjsfor syntax validation.
- - Confirmed generated index contains expected navigation/search markers and category anchors. -
Issues, Limitations, and Mitigations
--
-
- GitHub Pages must be enabled for this repository and set to GitHub Actions deployment. -
- The index reflects files present at build time and does not include full-text search inside documents. -
- Markdown files are linked as-is; rendering behavior depends on GitHub Pages static hosting behavior. -
Follow-up Work
--
-
- Add a docs landing page summary for key collections (turn docs, runbooks, daily notes). -
- Optionally add link-checking in CI for docs URLs and local references. -
- Consider tagging docs with metadata for richer filtering by date, topic, and type. -
${escapeHtml(category)} ${entries.length}
--
- ${entryMarkup}
-
Islandflow docs index
-A browsable index of files under docs/ with filtering and grouped navigation.
No files match that filter.
-