Created 2026-05-19 20:05 EDT · Branch: alpaca-news · Issue: islandflow-laq

Fix Native Alpaca News

Restored the native Alpaca news pipeline on the VPS by correcting Alpaca auth to use key ID + secret, adding the missing native islandflow-ingest-news unit and worker-scope wiring, fixing the Alpaca news backfill defaults to match the current API contract, requesting article content explicitly, and repairing API-side news persistence so the feed is both live and queryable.

VPS unit installed and enabled Alpaca auth aligned to current docs Live news confirmed ClickHouse news history confirmed

Summary

The original native news rollout failed for two separate reasons: the repo never fully wired ingest-news into the native worker templates, and the service was still using bearer-style Alpaca auth plus an oversized backfill limit that Alpaca's current News API rejects. After the service started flowing again, one more pipeline gap appeared: the API fanned news out live but never persisted it to ClickHouse, so /news stayed empty even when headlines showed up in the UI.

Changes Made

Context

Alpaca's current official auth docs require the APCA-API-KEY-ID and APCA-API-SECRET-KEY header pair for market-data requests, and the current News endpoint documents a limit range of 1..50 plus optional include_content. This turn aligned Islandflow's native news path with those present-day contracts instead of relying on the older single-token assumption that had drifted into the repo.

Important Implementation Details

Relevant Diff Snippets

diff --git a/packages/config/src/alpaca.ts b/packages/config/src/alpaca.ts
+export const buildAlpacaAuthHeaders = (credentials) => ({
+  "APCA-API-KEY-ID": credentials.keyId,
+  "APCA-API-SECRET-KEY": credentials.secret
+})
+export const buildAlpacaWebSocketAuthMessage = (credentials) => ({
+  action: "auth",
+  key: credentials.keyId,
+  secret: credentials.secret
+})
diff --git a/services/ingest-news/src/index.ts b/services/ingest-news/src/index.ts
-  ALPACA_NEWS_BACKFILL_LIMIT: z.coerce.number().int().positive().max(200).default(100),
+  ALPACA_NEWS_BACKFILL_LIMIT: z.coerce.number().int().positive().max(50).default(50),
+  url.searchParams.set("include_content", "true");
+  const contentHtml = item.content?.trim() || (summary ? `<p>${escapeHtml(summary)}</p>` : "");
diff --git a/services/api/src/index.ts b/services/api/src/index.ts
   const payload = NewsStorySchema.parse(newsSubscription.decode(msg));
+  await insertNewsStory(clickhouse, payload);
   await fanoutLive({ channel: "news" }, payload, "news");
   msg.ack();

These snippets are included in a diff-style rendering format for fast review.

Expected Impact for End-Users

Native Islandflow deployments on the VPS now have a real Alpaca-backed news worker instead of a missing unit and a crash loop. News stories populate with actual article body content in the feed more reliably, and the API's /news path can serve persisted recent stories instead of only depending on live websocket state.

Validation

Issues, Limitations, and Mitigations

Follow-up Work