Islandflow Turn Document

# Native Public Edge Cutover

Completed the VPS native-first cutover for Islandflow infrastructure and app services while keeping Nginx Proxy Manager as the outer edge and Docker as the rollback path. The final state now serves `flow.deltaisland.io` and `api.flow.deltaisland.io` from the native web and API processes, with verified public routing and a documented follow-up for the long-term API Cloudflare posture.

Generated
2026-05-18 19:52 EDT
Primary Issue
`islandflow-vvw`
Follow-up
`islandflow-fl5`
Runtime State
Native active, Docker retained for rollback

## Summary

The repository now contains the native infra units, native cutover scripts, Docker fallback adjustments, and public-edge retargeting logic required to run Islandflow natively on the VPS. During validation, the live NPM edge was switched from Docker container-name upstreams to native host ports, the host firewall was adjusted so the NPM bridge could reach the native API, and the separate public API TLS problem was resolved by correcting the Cloudflare DNS state for `api.flow.deltaisland.io`.

## Changes Made

- Added checked-in native infra operations under `deployment/native/`, including `bootstrap-infra.sh`, `check-native-infra.sh`, `cutover.sh`, `full-rollback.sh`, `start-infra.sh`, and the native system units for NATS, Redis, and ClickHouse.
- Extended native app runtime units so the web and API bind on host-reachable interfaces, and forced the native options ingest service to use the synthetic adapter during the cutover.
- Updated `services/api` to support explicit host binding through `API_HOST`, and fixed JetStream retention conversion in `packages/bus` so native services can start cleanly with the configured max-age values.
- Updated the Docker fallback assets to publish loopback web/API ports, share durable host data under `/var/lib/islandflow`, and document the native-to-Docker rollback path.
- Reworked `deployment/native/switch-npm-edge.sh` so it targets the NPM bridge gateway IP instead of `host.docker.internal`, handles the root-owned NPM SQLite database, synchronizes generated `proxy_host` configs, and reloads NPM deterministically after the edge switch.
- Created Beads follow-up issue `islandflow-fl5` for the remaining decision about whether `api.flow.deltaisland.io` should remain DNS-only or be re-proxied through Cloudflare.

## Context

The migration started from a Docker-owned production baseline where NATS, Redis, ClickHouse, API, workers, and web all ran in Compose, while NPM routed Islandflow traffic to Docker service names. That setup blocked a safe native cutover for two reasons: the native services could not reach Docker-only infra reliably, and NPM could not send public traffic to host-native processes without a deliberate upstream retarget.

The runtime model for this work is exclusive ownership. Native and Docker are not allowed to run the same API or worker scopes in parallel because JetStream durable consumers would conflict. The objective was therefore a phased handoff, not a mixed soak for the same queues.

## Important Implementation Details

NPM edge targeting

NPM generates `proxy_pass` from a runtime-resolved `$server` variable, so the Docker `/etc/hosts` alias for `host.docker.internal` was not sufficient. The switch helper now detects the NPM bridge gateway and uses that IP for native upstreams.

Firewall path

The host UFW policy already allowed port `3000` but not `4000`. The live fix was a source-scoped allow for the NPM bridge subnet so the containerized edge could reach the native API.

Cloudflare API hostname

The API hostname failure was separate from the native cutover. The hostname is now a DNS-only `A` record pointing at the VPS, which restored public TLS and health responses.

Area | Implementation detail
--- | ---
Native API | `services/api/src/index.ts` now accepts `API_HOST` and passes it to `Bun.serve`. The native unit sets `API_HOST=0.0.0.0` and `API_PORT=4000`.
Native web | The native web unit now starts from `apps/web` with `bun x next start -H "$WEB_HOST" -p "$WEB_PORT"`, avoiding the earlier repo-root startup failure and binding the service on `0.0.0.0:3000`.
JetStream retention | Native startup exposed a retention-unit bug. The shared bus layer now converts stream max-age values with `nanos(...)` and formats them back with `millis(...)`.
Docker fallback | Docker Compose now uses `ISLANDFLOW_DATA_ROOT=/var/lib/islandflow`, publishes loopback ports, and keeps the fallback runtime compatible with the same durable data directories as the native services.
NPM switch helper | The helper now updates both the NPM database and the generated `/data/nginx/proxy_host/*.conf` files, because a DB-only restart did not reliably rewrite the live configs for Islandflow.

    sudo ufw allow proto tcp from 172.18.0.0/16 to any port 4000 comment 'npm bridge to native api'

## Expected Impact for End-Users

- Public web and API traffic now reaches the native Islandflow services, which removes Docker from the primary live request path while keeping the outer edge unchanged.
- Same-origin public API routes such as `/prints`, `/history`, `/replay`, `/nbbo`, and `/ws/live` continue to resolve correctly through the main app hostname.
- Rollback remains fast and explicit: NPM can be pointed back at Docker service names and the Docker runtime can reclaim the same durable data directories if native operation needs to be abandoned.

## Validation

Static checks

- `bun run check:docker-workspace`
- `docker compose -f deployment/docker/docker-compose.yml config --quiet`
- `docker compose -f /home/delta/nginx-proxy-manager/docker-compose.yml config --quiet`
- `bash -n deployment/native/*.sh`
- `systemd-analyze verify deployment/native/systemd/user/*.service deployment/native/systemd/system/*.service`
- `bun build services/api/src/index.ts --target=bun`
- `bun build scripts/deploy.ts --target=bun`

Native runtime

- `./deployment/native/check-native-health.sh full`
- `curl http://127.0.0.1:4000/health`
- `curl -I http://127.0.0.1:3000/`

Public edge

- `curl -I -fksS https://flow.deltaisland.io`
- `curl -fksS https://api.flow.deltaisland.io/health`
- `bun run scripts/check-public-api-routes.ts https://flow.deltaisland.io`

## Issues, Limitations, and Mitigations

- The native ingest-options service required an explicit synthetic-adapter override because the environment file still pointed at an Alpaca adapter that was returning `401` responses. The service now starts cleanly for native cutover, but production adapter selection remains an operational decision.
- The NPM helper still relies on direct config synchronization because NPM did not reliably regenerate the Islandflow proxy files from SQLite changes alone. This is mitigated by keeping the synchronization logic checked in and by reloading NPM as part of the helper itself.
- The final public API recovery currently leaves `api.flow.deltaisland.io` as a DNS-only hostname. That restored service, but it changes the edge posture relative to the web hostname and should be reviewed deliberately.
- A temporary Cloudflare API token was used to inspect and correct zone state during validation. That token should be rotated outside this repository workflow.

## Follow-up Work

- `islandflow-fl5`: decide whether `api.flow.deltaisland.io` should remain DNS-only or be re-proxied through Cloudflare, then re-validate TLS, websocket, and operational behavior for the chosen posture.
- After operational soak, decide whether native should become the default production runtime or remain a supported alternative with Docker as the preferred steady-state runtime.