Implement native public edge cutover
This commit is contained in:
parent
d589858c03
commit
bdb9d9a95a
29 changed files with 1215 additions and 31 deletions
|
|
@ -9,12 +9,14 @@ This directory documents the host-native Islandflow rollout path used by:
|
|||
|
||||
## Current operating model
|
||||
|
||||
Native runtime is now intended for **fast iterative backend deploys first**, while Docker remains the supported public production edge until a deliberate cutover is completed.
|
||||
Native runtime is now intended for a phased VPS cutover. Docker remains the supported rollback runtime, but Docker and native app services must not own the same Islandflow scope at the same time because the workers and API use durable JetStream consumers.
|
||||
|
||||
Today, the recommended split is:
|
||||
|
||||
- **Docker runtime** for the live public `web` + `api` path
|
||||
- **Native runtime** for worker-only iteration (`compute`, `candles`, `ingest-options`, `ingest-equities`)
|
||||
- **Nginx Proxy Manager** remains the public `:80/:443` edge
|
||||
- **Native system services** own NATS, Redis, and ClickHouse after infra cutover
|
||||
- **Native user services** own `web`, `api`, and workers after app cutover
|
||||
- **Docker Compose** remains available as the rollback runtime
|
||||
- local development stays:
|
||||
- Docker infra: `bun run dev:infra`
|
||||
- native backend services: `bun run dev:services`
|
||||
|
|
@ -47,6 +49,38 @@ That means native worker deploy support is now provisioned on the host, but nati
|
|||
|
||||
## Checked-in native ops assets
|
||||
|
||||
### Infra system units
|
||||
|
||||
Checked-in system service units and config live under:
|
||||
|
||||
- `deployment/native/systemd/system/islandflow-nats.service`
|
||||
- `deployment/native/systemd/system/islandflow-redis.service`
|
||||
- `deployment/native/systemd/system/islandflow-clickhouse.service`
|
||||
- `deployment/native/config/redis.conf`
|
||||
- `deployment/native/config/clickhouse-listen.xml`
|
||||
|
||||
Install and start them on the VPS with:
|
||||
|
||||
```bash
|
||||
./deployment/native/bootstrap-infra.sh
|
||||
```
|
||||
|
||||
Or install and start manually:
|
||||
|
||||
```bash
|
||||
sudo ./deployment/native/install-infra-units.sh
|
||||
sudo ./deployment/native/start-infra.sh
|
||||
./deployment/native/check-native-infra.sh
|
||||
```
|
||||
|
||||
The native infra services bind to loopback and use stable host data paths:
|
||||
|
||||
- NATS JetStream: `/var/lib/islandflow/nats`
|
||||
- Redis: `/var/lib/islandflow/redis`
|
||||
- ClickHouse: `/var/lib/islandflow/clickhouse`
|
||||
|
||||
The Docker fallback compose file uses the same `ISLANDFLOW_DATA_ROOT` default of `/var/lib/islandflow`, so rollback can preserve durable state when only one runtime is active.
|
||||
|
||||
### User unit templates
|
||||
|
||||
Checked-in unit files live under:
|
||||
|
|
@ -89,10 +123,29 @@ Install script behavior:
|
|||
|
||||
This validates:
|
||||
|
||||
- native infra health for `full`, `api`, `services`, and `workers`
|
||||
- `systemctl --user is-active` for the selected units
|
||||
- local API health at `http://127.0.0.1:4000/health` when API scope is included
|
||||
- local web health at `http://127.0.0.1:3000/` when web scope is included
|
||||
|
||||
### App cutover and edge switch helpers
|
||||
|
||||
```bash
|
||||
./deployment/native/cutover.sh full
|
||||
./deployment/native/switch-npm-edge.sh native
|
||||
./deployment/native/full-rollback.sh
|
||||
```
|
||||
|
||||
The edge switch helper updates the Nginx Proxy Manager database entries for `flow.deltaisland.io` and `api.flow.deltaisland.io`, preserving the same-origin Islandflow API location matcher:
|
||||
|
||||
```nginx
|
||||
^/(ws|replay|prints|joins|nbbo|dark|flow|candles|history)/
|
||||
```
|
||||
|
||||
For native cutover, the helper targets the NPM bridge gateway IP by default, not `host.docker.internal`. NPM generates `proxy_pass` with a runtime-resolved `$server` variable, so Docker's `/etc/hosts` alias is not sufficient for these proxy hosts. On the current VPS that native target resolves to `172.18.0.1`, which reaches the host-native `3000` and `4000` listeners from the NPM container.
|
||||
|
||||
Switching back to Docker restores upstreams to the Compose service names `web:3000` and `api:4000`.
|
||||
|
||||
### Rollback helper
|
||||
|
||||
```bash
|
||||
|
|
@ -184,7 +237,7 @@ Without that variable, these commands are refused:
|
|||
- `./deploy main --runtime native --api-only`
|
||||
- `./deploy main --runtime native --services-only`
|
||||
|
||||
This keeps the native path focused on safe worker iteration until proxy routing and public unit ownership are switched deliberately.
|
||||
This keeps native app ownership explicit until infra, app health, and proxy routing are switched deliberately.
|
||||
|
||||
## Running deploy from the VPS itself
|
||||
|
||||
|
|
|
|||
24
deployment/native/bootstrap-infra.sh
Executable file
24
deployment/native/bootstrap-infra.sh
Executable file
|
|
@ -0,0 +1,24 @@
|
|||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||
|
||||
if [[ "${EUID}" -eq 0 ]]; then
|
||||
"$repo_root/deployment/native/install-infra-units.sh"
|
||||
else
|
||||
sudo "$repo_root/deployment/native/install-infra-units.sh"
|
||||
fi
|
||||
|
||||
echo "Stopping Docker Islandflow services before native infra opens durable data."
|
||||
(
|
||||
cd "$repo_root/deployment/docker"
|
||||
docker compose stop web api compute candles ingest-options ingest-equities nats redis clickhouse
|
||||
)
|
||||
|
||||
if [[ "${EUID}" -eq 0 ]]; then
|
||||
"$repo_root/deployment/native/start-infra.sh"
|
||||
else
|
||||
sudo "$repo_root/deployment/native/start-infra.sh"
|
||||
fi
|
||||
|
||||
"$repo_root/deployment/native/check-native-infra.sh"
|
||||
|
|
@ -2,6 +2,7 @@
|
|||
set -euo pipefail
|
||||
|
||||
scope="${1:-full}"
|
||||
repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||
units=()
|
||||
|
||||
case "$scope" in
|
||||
|
|
@ -27,6 +28,12 @@ case "$scope" in
|
|||
;;
|
||||
esac
|
||||
|
||||
case "$scope" in
|
||||
full|api|services|workers)
|
||||
"$repo_root/deployment/native/check-native-infra.sh"
|
||||
;;
|
||||
esac
|
||||
|
||||
for unit in "${units[@]}"; do
|
||||
systemctl --user is-active --quiet "$unit"
|
||||
echo "ok $unit"
|
||||
|
|
|
|||
24
deployment/native/check-native-infra.sh
Executable file
24
deployment/native/check-native-infra.sh
Executable file
|
|
@ -0,0 +1,24 @@
|
|||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
systemctl is-active --quiet islandflow-nats.service
|
||||
echo "ok islandflow-nats.service"
|
||||
|
||||
systemctl is-active --quiet islandflow-redis.service
|
||||
echo "ok islandflow-redis.service"
|
||||
|
||||
systemctl is-active --quiet islandflow-clickhouse.service
|
||||
echo "ok islandflow-clickhouse.service"
|
||||
|
||||
if command -v redis-cli >/dev/null 2>&1; then
|
||||
redis-cli -h 127.0.0.1 -p 6379 ping | grep -q PONG
|
||||
else
|
||||
timeout 2 bash -c '</dev/tcp/127.0.0.1/6379'
|
||||
fi
|
||||
echo "ok redis-ping"
|
||||
|
||||
curl -fksS http://127.0.0.1:8123/ping | grep -q Ok
|
||||
echo "ok clickhouse-ping"
|
||||
|
||||
timeout 2 bash -c '</dev/tcp/127.0.0.1/4222'
|
||||
echo "ok nats-port"
|
||||
6
deployment/native/config/clickhouse-listen.xml
Normal file
6
deployment/native/config/clickhouse-listen.xml
Normal file
|
|
@ -0,0 +1,6 @@
|
|||
<clickhouse>
|
||||
<listen_host>127.0.0.1</listen_host>
|
||||
<path>/var/lib/islandflow/clickhouse/</path>
|
||||
<tmp_path>/var/lib/islandflow/clickhouse/tmp/</tmp_path>
|
||||
<user_files_path>/var/lib/islandflow/clickhouse/user_files/</user_files_path>
|
||||
</clickhouse>
|
||||
10
deployment/native/config/redis.conf
Normal file
10
deployment/native/config/redis.conf
Normal file
|
|
@ -0,0 +1,10 @@
|
|||
bind 127.0.0.1
|
||||
protected-mode yes
|
||||
port 6379
|
||||
dir /var/lib/islandflow/redis
|
||||
appendonly yes
|
||||
save 900 1
|
||||
save 300 10
|
||||
save 60 10000
|
||||
loglevel notice
|
||||
databases 16
|
||||
34
deployment/native/cutover.sh
Executable file
34
deployment/native/cutover.sh
Executable file
|
|
@ -0,0 +1,34 @@
|
|||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
scope="${1:-full}"
|
||||
repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||
|
||||
case "$scope" in
|
||||
full|services|workers|api|web)
|
||||
;;
|
||||
*)
|
||||
echo "Usage: deployment/native/cutover.sh [full|services|workers|api|web]" >&2
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
|
||||
echo "Stopping Docker-owned Islandflow app services before native ownership starts."
|
||||
(
|
||||
cd "$repo_root/deployment/docker"
|
||||
docker compose stop web api compute candles ingest-options ingest-equities
|
||||
)
|
||||
|
||||
if [[ "$scope" == "full" || "$scope" == "services" || "$scope" == "api" || "$scope" == "web" ]]; then
|
||||
"$repo_root/deployment/native/check-native-infra.sh"
|
||||
fi
|
||||
|
||||
systemctl --user restart $(case "$scope" in
|
||||
full) echo islandflow-web.service islandflow-api.service islandflow-compute.service islandflow-candles.service islandflow-ingest-options.service islandflow-ingest-equities.service ;;
|
||||
services) echo islandflow-api.service islandflow-compute.service islandflow-candles.service islandflow-ingest-options.service islandflow-ingest-equities.service ;;
|
||||
workers) echo islandflow-compute.service islandflow-candles.service islandflow-ingest-options.service islandflow-ingest-equities.service ;;
|
||||
api) echo islandflow-api.service ;;
|
||||
web) echo islandflow-web.service ;;
|
||||
esac)
|
||||
|
||||
"$repo_root/deployment/native/check-native-health.sh" "$scope"
|
||||
27
deployment/native/full-rollback.sh
Executable file
27
deployment/native/full-rollback.sh
Executable file
|
|
@ -0,0 +1,27 @@
|
|||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||
|
||||
echo "Stopping native app services."
|
||||
systemctl --user stop islandflow-web.service islandflow-api.service islandflow-compute.service islandflow-candles.service islandflow-ingest-options.service islandflow-ingest-equities.service || true
|
||||
|
||||
echo "Stopping native infra before Docker reopens durable data."
|
||||
if [[ "${EUID}" -eq 0 ]]; then
|
||||
systemctl stop islandflow-nats.service islandflow-redis.service islandflow-clickhouse.service || true
|
||||
else
|
||||
sudo systemctl stop islandflow-nats.service islandflow-redis.service islandflow-clickhouse.service || true
|
||||
fi
|
||||
|
||||
echo "Switching NPM Islandflow upstreams back to Docker service names."
|
||||
"$repo_root/deployment/native/switch-npm-edge.sh" docker
|
||||
|
||||
echo "Restarting Docker Islandflow runtime."
|
||||
(
|
||||
cd "$repo_root/deployment/docker"
|
||||
docker compose up -d web api compute candles ingest-options ingest-equities
|
||||
)
|
||||
|
||||
curl -I -fksS "${DEPLOY_PUBLIC_APP_URL:-https://flow.deltaisland.io}" >/dev/null
|
||||
curl -fksS "${DEPLOY_PUBLIC_API_HEALTH_URL:-https://api.flow.deltaisland.io/health}" >/dev/null
|
||||
echo "Rollback validation passed."
|
||||
72
deployment/native/install-infra-units.sh
Executable file
72
deployment/native/install-infra-units.sh
Executable file
|
|
@ -0,0 +1,72 @@
|
|||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||
system_unit_source_dir="$repo_root/deployment/native/systemd/system"
|
||||
config_source_dir="$repo_root/deployment/native/config"
|
||||
|
||||
if [[ "${EUID}" -ne 0 ]]; then
|
||||
echo "Run as root: sudo $0" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
resolve_binary() {
|
||||
local name="$1"
|
||||
local path=""
|
||||
|
||||
path="$(command -v "$name" 2>/dev/null || true)"
|
||||
if [[ -n "$path" ]]; then
|
||||
printf '%s\n' "$path"
|
||||
return 0
|
||||
fi
|
||||
|
||||
for candidate in "/usr/bin/$name" "/usr/sbin/$name" "/usr/local/bin/$name" "/usr/local/sbin/$name"; do
|
||||
if [[ -x "$candidate" ]]; then
|
||||
printf '%s\n' "$candidate"
|
||||
return 0
|
||||
fi
|
||||
done
|
||||
|
||||
return 1
|
||||
}
|
||||
|
||||
missing=()
|
||||
for command in nats-server redis-server clickhouse-server; do
|
||||
if ! resolve_binary "$command" >/dev/null; then
|
||||
missing+=("$command")
|
||||
fi
|
||||
done
|
||||
|
||||
if [[ ${#missing[@]} -gt 0 ]]; then
|
||||
echo "Missing native infra binaries: ${missing[*]}" >&2
|
||||
echo "Install NATS Server, Redis Server, and ClickHouse Server before bootstrapping native infra." >&2
|
||||
echo "On Debian, Redis is usually available as redis-server; ClickHouse and NATS may require their vendor repositories or packaged binaries." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
ensure_system_user() {
|
||||
local name="$1"
|
||||
local home="$2"
|
||||
|
||||
getent group "$name" >/dev/null || groupadd --system "$name"
|
||||
getent passwd "$name" >/dev/null || useradd --system --gid "$name" --home-dir "$home" --shell /usr/sbin/nologin "$name"
|
||||
}
|
||||
|
||||
ensure_system_user nats /var/lib/islandflow/nats
|
||||
ensure_system_user redis /var/lib/islandflow/redis
|
||||
ensure_system_user clickhouse /var/lib/islandflow/clickhouse
|
||||
|
||||
install -d -m 0755 /etc/islandflow
|
||||
install -m 0644 "$config_source_dir/redis.conf" /etc/islandflow/redis.conf
|
||||
install -d -m 0755 /etc/clickhouse-server/config.d
|
||||
install -m 0644 "$config_source_dir/clickhouse-listen.xml" /etc/clickhouse-server/config.d/islandflow-listen.xml
|
||||
|
||||
install -d -o nats -g nats -m 0750 /var/lib/islandflow/nats
|
||||
install -d -o redis -g redis -m 0750 /var/lib/islandflow/redis
|
||||
install -d -o clickhouse -g clickhouse -m 0750 /var/lib/islandflow/clickhouse
|
||||
|
||||
install -m 0644 "$system_unit_source_dir"/islandflow-*.service /etc/systemd/system/
|
||||
systemctl daemon-reload
|
||||
|
||||
echo "Installed native infra system units and config."
|
||||
echo "Start infra with: sudo deployment/native/start-infra.sh"
|
||||
17
deployment/native/start-infra.sh
Executable file
17
deployment/native/start-infra.sh
Executable file
|
|
@ -0,0 +1,17 @@
|
|||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
if [[ "${EUID}" -ne 0 ]]; then
|
||||
echo "Run as root: sudo $0" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
for unit in redis-server.service nats-server.service clickhouse-server.service; do
|
||||
if systemctl list-unit-files "$unit" >/dev/null 2>&1; then
|
||||
systemctl disable --now "$unit" >/dev/null 2>&1 || true
|
||||
fi
|
||||
done
|
||||
|
||||
systemctl reset-failed islandflow-nats.service islandflow-redis.service islandflow-clickhouse.service || true
|
||||
systemctl enable --now islandflow-nats.service islandflow-redis.service islandflow-clickhouse.service
|
||||
"$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/check-native-infra.sh"
|
||||
9
deployment/native/stop-infra.sh
Executable file
9
deployment/native/stop-infra.sh
Executable file
|
|
@ -0,0 +1,9 @@
|
|||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
if [[ "${EUID}" -ne 0 ]]; then
|
||||
echo "Run as root: sudo $0" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
systemctl stop islandflow-nats.service islandflow-redis.service islandflow-clickhouse.service
|
||||
285
deployment/native/switch-npm-edge.sh
Executable file
285
deployment/native/switch-npm-edge.sh
Executable file
|
|
@ -0,0 +1,285 @@
|
|||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
target="${1:-native}"
|
||||
npm_root="${NPM_ROOT:-/home/delta/nginx-proxy-manager}"
|
||||
db_path="${NPM_DB_PATH:-$npm_root/data/database.sqlite}"
|
||||
app_domain="${ISLANDFLOW_APP_DOMAIN:-flow.deltaisland.io}"
|
||||
api_domain="${ISLANDFLOW_API_DOMAIN:-api.flow.deltaisland.io}"
|
||||
native_host="${ISLANDFLOW_NATIVE_HOST:-}"
|
||||
docker_web_host="${ISLANDFLOW_DOCKER_WEB_HOST:-web}"
|
||||
docker_api_host="${ISLANDFLOW_DOCKER_API_HOST:-api}"
|
||||
web_port="${ISLANDFLOW_WEB_PORT:-3000}"
|
||||
api_port="${ISLANDFLOW_API_PORT:-4000}"
|
||||
restart_npm="${NPM_RESTART:-1}"
|
||||
npm_container="${NPM_CONTAINER_NAME:-nginx-proxy-manager}"
|
||||
sudo_cmd=()
|
||||
|
||||
case "$target" in
|
||||
native|docker)
|
||||
;;
|
||||
*)
|
||||
echo "Usage: deployment/native/switch-npm-edge.sh [native|docker]" >&2
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
|
||||
resolve_native_host() {
|
||||
if [[ -n "$native_host" ]]; then
|
||||
printf '%s\n' "$native_host"
|
||||
return
|
||||
fi
|
||||
|
||||
if command -v docker >/dev/null 2>&1 && docker ps --format '{{.Names}}' | grep -qx "$npm_container"; then
|
||||
native_host="$(docker inspect "$npm_container" --format '{{range .NetworkSettings.Networks}}{{println .Gateway}}{{end}}' | sed '/^$/d' | head -n1)"
|
||||
if [[ -n "$native_host" ]]; then
|
||||
printf '%s\n' "$native_host"
|
||||
return
|
||||
fi
|
||||
fi
|
||||
|
||||
echo "Unable to determine the native upstream host for NPM." >&2
|
||||
echo "Set ISLANDFLOW_NATIVE_HOST explicitly or start the $npm_container container first." >&2
|
||||
exit 1
|
||||
}
|
||||
|
||||
if [[ "$target" == "native" ]]; then
|
||||
native_host="$(resolve_native_host)"
|
||||
fi
|
||||
|
||||
if [[ ! -w "$db_path" || ! -w "$(dirname "$db_path")" ]]; then
|
||||
if [[ "${EUID}" -eq 0 ]]; then
|
||||
sudo_cmd=()
|
||||
elif command -v sudo >/dev/null 2>&1; then
|
||||
sudo_cmd=(sudo)
|
||||
else
|
||||
echo "NPM database path is not writable and sudo is unavailable: $db_path" >&2
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
|
||||
if [[ ! -f "$db_path" ]]; then
|
||||
echo "NPM database not found: $db_path" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
backup="$db_path.before-islandflow-$target-$(date +%Y%m%d%H%M%S)"
|
||||
"${sudo_cmd[@]}" cp "$db_path" "$backup"
|
||||
echo "Backed up NPM database to $backup"
|
||||
|
||||
"${sudo_cmd[@]}" python3 - "$db_path" "$target" "$app_domain" "$api_domain" "$native_host" "$docker_web_host" "$docker_api_host" "$web_port" "$api_port" <<'PY'
|
||||
import json
|
||||
import sqlite3
|
||||
import sys
|
||||
|
||||
db_path, target, app_domain, api_domain, native_host, docker_web_host, docker_api_host, web_port, api_port = sys.argv[1:]
|
||||
web_host = native_host if target == "native" else docker_web_host
|
||||
api_host = native_host if target == "native" else docker_api_host
|
||||
|
||||
advanced_config = f"""location ~ ^/(ws|replay|prints|joins|nbbo|dark|flow|candles|history)/ {{
|
||||
set $forward_scheme http;
|
||||
set $server "{api_host}";
|
||||
set $port {api_port};
|
||||
|
||||
proxy_set_header Upgrade $http_upgrade;
|
||||
proxy_set_header Connection $http_connection;
|
||||
proxy_http_version 1.1;
|
||||
|
||||
include conf.d/include/proxy.conf;
|
||||
}}"""
|
||||
|
||||
def has_domain(raw, domain):
|
||||
try:
|
||||
return domain in json.loads(raw)
|
||||
except Exception:
|
||||
return domain in raw
|
||||
|
||||
con = sqlite3.connect(db_path)
|
||||
cur = con.cursor()
|
||||
rows = list(cur.execute("select id, domain_names from proxy_host where is_deleted = 0"))
|
||||
app_ids = [row_id for row_id, domains in rows if has_domain(domains, app_domain)]
|
||||
api_ids = [row_id for row_id, domains in rows if has_domain(domains, api_domain)]
|
||||
|
||||
if len(app_ids) != 1 or len(api_ids) != 1:
|
||||
raise SystemExit(f"Expected one app and one API proxy host, found app={app_ids} api={api_ids}")
|
||||
|
||||
cur.execute(
|
||||
"update proxy_host set forward_scheme = 'http', forward_host = ?, forward_port = ?, allow_websocket_upgrade = 1, advanced_config = ?, modified_on = datetime('now') where id = ?",
|
||||
(web_host, int(web_port), advanced_config, app_ids[0]),
|
||||
)
|
||||
cur.execute(
|
||||
"update proxy_host set forward_scheme = 'http', forward_host = ?, forward_port = ?, allow_websocket_upgrade = 1, modified_on = datetime('now') where id = ?",
|
||||
(api_host, int(api_port), api_ids[0]),
|
||||
)
|
||||
con.commit()
|
||||
print(f"Updated {app_domain} -> {web_host}:{web_port}")
|
||||
print(f"Updated {api_domain} -> {api_host}:{api_port}")
|
||||
PY
|
||||
|
||||
if command -v python3 >/dev/null 2>&1; then
|
||||
"${sudo_cmd[@]}" python3 - "$npm_root" "$db_path" "$target" "$app_domain" "$api_domain" "$native_host" "$docker_web_host" "$docker_api_host" "$web_port" "$api_port" <<'PY'
|
||||
import json
|
||||
import re
|
||||
import sqlite3
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
(
|
||||
npm_root,
|
||||
db_path,
|
||||
target,
|
||||
app_domain,
|
||||
api_domain,
|
||||
native_host,
|
||||
docker_web_host,
|
||||
docker_api_host,
|
||||
web_port,
|
||||
api_port,
|
||||
) = sys.argv[1:]
|
||||
|
||||
web_host = native_host if target == "native" else docker_web_host
|
||||
api_host = native_host if target == "native" else docker_api_host
|
||||
|
||||
def has_domain(raw, domain):
|
||||
try:
|
||||
return domain in json.loads(raw)
|
||||
except Exception:
|
||||
return domain in raw
|
||||
|
||||
def replace_nth(text, pattern, replacement, index):
|
||||
matches = list(pattern.finditer(text))
|
||||
if len(matches) < index:
|
||||
raise SystemExit(f"Unable to rewrite generated proxy config; expected match {index} for {pattern.pattern!r}")
|
||||
match = matches[index - 1]
|
||||
return text[:match.start()] + replacement(match) + text[match.end():]
|
||||
|
||||
server_pattern = re.compile(r'^(?P<prefix>\s*set \$server\s+)".*?";\s*$', re.M)
|
||||
port_pattern = re.compile(r'^(?P<prefix>\s*set \$port\s+)\d+;\s*$', re.M)
|
||||
|
||||
def replace_server(text, host, index):
|
||||
return replace_nth(text, server_pattern, lambda m: f'{m.group("prefix")}"{host}";', index)
|
||||
|
||||
def replace_port(text, port, index):
|
||||
return replace_nth(text, port_pattern, lambda m: f'{m.group("prefix")}{port};', index)
|
||||
|
||||
con = sqlite3.connect(db_path)
|
||||
rows = list(con.execute("select id, domain_names from proxy_host where is_deleted = 0"))
|
||||
app_ids = [row_id for row_id, domains in rows if has_domain(domains, app_domain)]
|
||||
api_ids = [row_id for row_id, domains in rows if has_domain(domains, api_domain)]
|
||||
if len(app_ids) != 1 or len(api_ids) != 1:
|
||||
raise SystemExit(f"Expected one app and one API proxy host, found app={app_ids} api={api_ids}")
|
||||
|
||||
api_conf = Path(npm_root) / "data/nginx/proxy_host" / f"{api_ids[0]}.conf"
|
||||
app_conf = Path(npm_root) / "data/nginx/proxy_host" / f"{app_ids[0]}.conf"
|
||||
|
||||
if api_conf.exists():
|
||||
text = api_conf.read_text()
|
||||
text = replace_server(text, api_host, 1)
|
||||
text = replace_port(text, int(api_port), 1)
|
||||
api_conf.write_text(text)
|
||||
print(f"Synchronized {api_conf.name} -> {api_host}:{api_port}")
|
||||
|
||||
if app_conf.exists():
|
||||
text = app_conf.read_text()
|
||||
text = replace_server(text, web_host, 1)
|
||||
text = replace_port(text, int(web_port), 1)
|
||||
text = replace_server(text, api_host, 2)
|
||||
text = replace_port(text, int(api_port), 2)
|
||||
app_conf.write_text(text)
|
||||
print(f"Synchronized {app_conf.name} -> {web_host}:{web_port} and API matcher -> {api_host}:{api_port}")
|
||||
PY
|
||||
fi
|
||||
|
||||
if [[ "$restart_npm" == "0" ]]; then
|
||||
echo "NPM container restart skipped because NPM_RESTART=0."
|
||||
elif command -v docker >/dev/null 2>&1 && docker ps --format '{{.Names}}' | grep -qx nginx-proxy-manager; then
|
||||
docker restart nginx-proxy-manager >/dev/null
|
||||
echo "Restarted nginx-proxy-manager"
|
||||
else
|
||||
echo "NPM container restart skipped; restart it manually if it is not managed by Docker on this host."
|
||||
fi
|
||||
|
||||
if command -v docker >/dev/null 2>&1 && docker ps --format '{{.Names}}' | grep -qx "$npm_container"; then
|
||||
"${sudo_cmd[@]}" python3 - "$npm_root" "$db_path" "$target" "$app_domain" "$api_domain" "$native_host" "$docker_web_host" "$docker_api_host" "$web_port" "$api_port" <<'PY'
|
||||
import json
|
||||
import re
|
||||
import sqlite3
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
(
|
||||
npm_root,
|
||||
db_path,
|
||||
target,
|
||||
app_domain,
|
||||
api_domain,
|
||||
native_host,
|
||||
docker_web_host,
|
||||
docker_api_host,
|
||||
web_port,
|
||||
api_port,
|
||||
) = sys.argv[1:]
|
||||
|
||||
web_host = native_host if target == "native" else docker_web_host
|
||||
api_host = native_host if target == "native" else docker_api_host
|
||||
|
||||
def has_domain(raw, domain):
|
||||
try:
|
||||
return domain in json.loads(raw)
|
||||
except Exception:
|
||||
return domain in raw
|
||||
|
||||
def replace_nth(text, pattern, replacement, index):
|
||||
matches = list(pattern.finditer(text))
|
||||
if len(matches) < index:
|
||||
raise SystemExit(f"Unable to rewrite generated proxy config; expected match {index} for {pattern.pattern!r}")
|
||||
match = matches[index - 1]
|
||||
return text[:match.start()] + replacement(match) + text[match.end():]
|
||||
|
||||
server_pattern = re.compile(r'^(?P<prefix>\s*set \$server\s+)".*?";\s*$', re.M)
|
||||
port_pattern = re.compile(r'^(?P<prefix>\s*set \$port\s+)\d+;\s*$', re.M)
|
||||
|
||||
def replace_server(text, host, index):
|
||||
return replace_nth(text, server_pattern, lambda m: f'{m.group("prefix")}"{host}";', index)
|
||||
|
||||
def replace_port(text, port, index):
|
||||
return replace_nth(text, port_pattern, lambda m: f'{m.group("prefix")}{port};', index)
|
||||
|
||||
con = sqlite3.connect(db_path)
|
||||
rows = list(con.execute("select id, domain_names from proxy_host where is_deleted = 0"))
|
||||
app_ids = [row_id for row_id, domains in rows if has_domain(domains, app_domain)]
|
||||
api_ids = [row_id for row_id, domains in rows if has_domain(domains, api_domain)]
|
||||
if len(app_ids) != 1 or len(api_ids) != 1:
|
||||
raise SystemExit(f"Expected one app and one API proxy host, found app={app_ids} api={api_ids}")
|
||||
|
||||
api_conf = Path(npm_root) / "data/nginx/proxy_host" / f"{api_ids[0]}.conf"
|
||||
app_conf = Path(npm_root) / "data/nginx/proxy_host" / f"{app_ids[0]}.conf"
|
||||
|
||||
if api_conf.exists():
|
||||
text = api_conf.read_text()
|
||||
text = replace_server(text, api_host, 1)
|
||||
text = replace_port(text, int(api_port), 1)
|
||||
api_conf.write_text(text)
|
||||
|
||||
if app_conf.exists():
|
||||
text = app_conf.read_text()
|
||||
text = replace_server(text, web_host, 1)
|
||||
text = replace_port(text, int(web_port), 1)
|
||||
text = replace_server(text, api_host, 2)
|
||||
text = replace_port(text, int(api_port), 2)
|
||||
app_conf.write_text(text)
|
||||
PY
|
||||
reloaded=0
|
||||
for _ in 1 2 3 4 5; do
|
||||
if docker exec "$npm_container" nginx -s reload >/dev/null 2>&1; then
|
||||
reloaded=1
|
||||
break
|
||||
fi
|
||||
sleep 1
|
||||
done
|
||||
if [[ "$reloaded" == "1" ]]; then
|
||||
echo "Reloaded nginx-proxy-manager"
|
||||
else
|
||||
echo "Warning: nginx-proxy-manager reload did not succeed after restart; verify the container is healthy." >&2
|
||||
fi
|
||||
fi
|
||||
|
|
@ -0,0 +1,17 @@
|
|||
[Unit]
|
||||
Description=Islandflow ClickHouse
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
ExecStart=/usr/bin/env clickhouse-server --config-file=/etc/clickhouse-server/config.xml
|
||||
Restart=always
|
||||
RestartSec=5
|
||||
User=clickhouse
|
||||
Group=clickhouse
|
||||
StateDirectory=clickhouse
|
||||
LimitNOFILE=262144
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
18
deployment/native/systemd/system/islandflow-nats.service
Normal file
18
deployment/native/systemd/system/islandflow-nats.service
Normal file
|
|
@ -0,0 +1,18 @@
|
|||
[Unit]
|
||||
Description=Islandflow NATS JetStream
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
ExecStart=/usr/sbin/nats-server -js -sd /var/lib/islandflow/nats -a 127.0.0.1 -p 4222 -m 8222
|
||||
Restart=always
|
||||
RestartSec=2
|
||||
User=nats
|
||||
Group=nats
|
||||
RuntimeDirectory=islandflow-nats
|
||||
StateDirectory=islandflow/nats
|
||||
LimitNOFILE=1048576
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
18
deployment/native/systemd/system/islandflow-redis.service
Normal file
18
deployment/native/systemd/system/islandflow-redis.service
Normal file
|
|
@ -0,0 +1,18 @@
|
|||
[Unit]
|
||||
Description=Islandflow Redis
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Service]
|
||||
Type=notify
|
||||
ExecStart=/usr/bin/env redis-server /etc/islandflow/redis.conf --supervised systemd --daemonize no
|
||||
Restart=always
|
||||
RestartSec=2
|
||||
User=redis
|
||||
Group=redis
|
||||
RuntimeDirectory=islandflow-redis
|
||||
StateDirectory=islandflow/redis
|
||||
LimitNOFILE=65535
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
|
|
@ -6,6 +6,8 @@ Wants=network-online.target
|
|||
[Service]
|
||||
Type=simple
|
||||
WorkingDirectory=/home/delta/islandflow
|
||||
Environment=API_HOST=0.0.0.0
|
||||
Environment=API_PORT=4000
|
||||
EnvironmentFile=/home/delta/islandflow/.env
|
||||
ExecStart=/home/delta/.bun/bin/bun services/api/src/index.ts
|
||||
Restart=always
|
||||
|
|
|
|||
|
|
@ -7,6 +7,7 @@ Wants=network-online.target
|
|||
Type=simple
|
||||
WorkingDirectory=/home/delta/islandflow
|
||||
EnvironmentFile=/home/delta/islandflow/.env
|
||||
Environment=OPTIONS_INGEST_ADAPTER=synthetic
|
||||
ExecStart=/home/delta/.bun/bin/bun services/ingest-options/src/index.ts
|
||||
Restart=always
|
||||
RestartSec=2
|
||||
|
|
|
|||
|
|
@ -6,8 +6,10 @@ Wants=network-online.target
|
|||
[Service]
|
||||
Type=simple
|
||||
WorkingDirectory=/home/delta/islandflow
|
||||
Environment=WEB_HOST=0.0.0.0
|
||||
Environment=WEB_PORT=3000
|
||||
EnvironmentFile=/home/delta/islandflow/.env
|
||||
ExecStart=/home/delta/.bun/bin/bun --cwd apps/web run start
|
||||
ExecStart=/bin/sh -lc 'cd /home/delta/islandflow/apps/web && exec /home/delta/.bun/bin/bun x next start -H "$WEB_HOST" -p "$WEB_PORT"'
|
||||
Restart=always
|
||||
RestartSec=2
|
||||
KillSignal=SIGINT
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue