From 3a7267865219192159216cb35443dee68de66cc6 Mon Sep 17 00:00:00 2001 From: dirtydishes Date: Mon, 1 Jun 2026 09:46:23 -0400 Subject: [PATCH] harden docs pages publish --- .beads/issues.jsonl | 2 +- .github/workflows/docs-pages.yml | 1 + docs/index.html | 2 +- docs/turns/2026-05-31-fix-docs-pages-mirror.html | 8 ++++---- 4 files changed, 7 insertions(+), 6 deletions(-) diff --git a/.beads/issues.jsonl b/.beads/issues.jsonl index b896909..0b4c346 100644 --- a/.beads/issues.jsonl +++ b/.beads/issues.jsonl @@ -25,7 +25,7 @@ {"_type":"issue","id":"islandflow-ayo","title":"Drop stale backlog events from live fanout","description":"Follow-up to live freshness rollout: /ws/live was still fanning out stale backlog events for freshness-gated channels, which kept tape panes in Live feed behind despite active synthetic ingest. Gate fanout and cache ingest by freshness for options/nbbo/equities/flow.","status":"closed","priority":1,"issue_type":"bug","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-04-28T21:26:39Z","created_by":"dirtydishes","updated_at":"2026-04-28T21:26:44Z","started_at":"2026-04-28T21:26:44Z","closed_at":"2026-04-28T21:26:44Z","close_reason":"Completed","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"islandflow-0v6","title":"Fix tape freshness, NBBO coverage, pause controls, and filter popup","description":"Implement the tape fixes requested for synthetic options notional sizing, strict live freshness, live-mode pause/resume behavior, stronger NBBO snapshot coverage, and moving flow filters behind a popup. Includes server-side live cache changes, web terminal state/UI changes, and tests for synthetic pricing, live snapshot freshness/NBBO retention, and live pause/filter interactions.","status":"closed","priority":1,"issue_type":"task","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-04-28T21:02:52Z","created_by":"dirtydishes","updated_at":"2026-04-28T21:13:38Z","started_at":"2026-04-28T21:02:57Z","closed_at":"2026-04-28T21:13:38Z","close_reason":"Completed","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"islandflow-e4r","title":"Implement smart-money flow filtering and synthetic firehose modes","description":"Implement the approved multi-surface plan for named synthetic market profiles, options raw-vs-signal filtering, live/API filter contracts, Tape page client-side flow filters, firehose-readiness improvements, tests, and README updates.","status":"closed","priority":1,"issue_type":"feature","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-04-28T20:10:49Z","created_by":"dirtydishes","updated_at":"2026-04-28T20:29:29Z","started_at":"2026-04-28T20:10:53Z","closed_at":"2026-04-28T20:29:29Z","close_reason":"Implemented synthetic market profiles, options signal-path filtering, signal-aware API/replay contracts, Tape page filters, tests, and README updates. Follow-up tracked in islandflow-biq.","dependency_count":0,"dependent_count":0,"comment_count":0} -{"_type":"issue","id":"islandflow-8a6","title":"verify github pages token for docs mirror","description":"The docs mirror workflow now publishes islandflow/docs into dirtydishes/dirtydishes.github.io, but the GitHub Actions secret DOCS_PAGES_TOKEN must exist and have permission to push to that Pages repository. Verify the secret is configured and manually run the Publish Docs workflow after the mirror branch lands.","status":"open","priority":2,"issue_type":"task","owner":"dishes@dpdrm.com","created_at":"2026-05-31T22:12:27Z","created_by":"dirtydishes","updated_at":"2026-05-31T22:12:27Z","dependency_count":0,"dependent_count":0,"comment_count":0} +{"_type":"issue","id":"islandflow-8a6","title":"verify github pages token for docs mirror","description":"The docs mirror workflow now publishes islandflow/docs into dirtydishes/dirtydishes.github.io, but the GitHub Actions secret DOCS_PAGES_TOKEN must exist and have permission to push to that Pages repository. Verify the secret is configured and manually run the Publish Docs workflow after the mirror branch lands.","notes":"Direct manual publish to dirtydishes/dirtydishes.github.io succeeded on 2026-06-01 and https://dirtydishes.github.io/islandflow/docs/ returned HTTP 200. Remaining work is to verify DOCS_PAGES_TOKEN so the islandflow docs mirror workflow can publish future updates automatically.","status":"open","priority":2,"issue_type":"task","owner":"dishes@dpdrm.com","created_at":"2026-05-31T22:12:27Z","created_by":"dirtydishes","updated_at":"2026-06-01T13:45:34Z","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"islandflow-5jt","title":"Add anatomy reference page","description":"Create a standalone docs/anatomy.html reference explaining how prints move through ingest, tape, flow packets, smart-money events, classifier hits, and alerts, including diagrams.","status":"closed","priority":2,"issue_type":"task","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-05-31T21:20:34Z","created_by":"dirtydishes","updated_at":"2026-05-31T21:25:54Z","started_at":"2026-05-31T21:20:44Z","closed_at":"2026-05-31T21:25:54Z","close_reason":"Added the standalone anatomy reference page and linked it from the docs index.","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"islandflow-cig","title":"Expand CI quality gates","description":"Add a more robust CI workflow for the Bun/TypeScript monorepo, including formatting, linting, type checking, builds, and tests where appropriate.","status":"closed","priority":2,"issue_type":"task","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-05-30T06:29:33Z","created_by":"dirtydishes","updated_at":"2026-05-30T06:34:11Z","started_at":"2026-05-30T06:29:41Z","closed_at":"2026-05-30T06:34:11Z","close_reason":"Expanded CI quality gates with Biome formatting/linting, public API route checks, Docker snapshot validation, tests, typecheck, and web build validation.","dependency_count":0,"dependent_count":0,"comment_count":0} {"_type":"issue","id":"islandflow-3l6","title":"fix ci typecheck bun path resolution","description":"Forgejo CI fails in scripts/typecheck.ts because the script shells out to bunx, which expects bun on PATH. The runner installs Bun by absolute path, so the typecheck helper should use the current Bun executable instead of PATH lookup.","status":"closed","priority":2,"issue_type":"bug","assignee":"dirtydishes","owner":"dishes@dpdrm.com","created_at":"2026-05-30T05:34:55Z","created_by":"dirtydishes","updated_at":"2026-05-30T06:00:31Z","started_at":"2026-05-30T05:35:02Z","closed_at":"2026-05-30T06:00:31Z","close_reason":"Fixed the Forgejo CI terminal import mismatch by switching the terminal client component to a namespace import; verified locally and on Forgejo run #56.","dependency_count":0,"dependent_count":0,"comment_count":0} diff --git a/.github/workflows/docs-pages.yml b/.github/workflows/docs-pages.yml index 5fc6bcb..65df8fc 100644 --- a/.github/workflows/docs-pages.yml +++ b/.github/workflows/docs-pages.yml @@ -36,6 +36,7 @@ jobs: rm -rf site mkdir -p site/docs cp -R docs/. site/docs/ + find site -name .DS_Store -delete printf '%s\n' 'Islandflow DocsContinue to docs' > site/index.html touch site/.nojekyll diff --git a/docs/index.html b/docs/index.html index b511833..0e3ed60 100644 --- a/docs/index.html +++ b/docs/index.html @@ -224,7 +224,7 @@
html 203 KB - May 31, 2026, 6:12 PM + Jun 1, 2026, 9:45 AM
  • diff --git a/docs/turns/2026-05-31-fix-docs-pages-mirror.html b/docs/turns/2026-05-31-fix-docs-pages-mirror.html index b065c76..a15c6e2 100644 --- a/docs/turns/2026-05-31-fix-docs-pages-mirror.html +++ b/docs/turns/2026-05-31-fix-docs-pages-mirror.html @@ -40,7 +40,7 @@
    May 31, 2026Beads: islandflow-m3dWorkflow: .github/workflows/docs-pages.yml

    Summary

    Repaired the Pages publishing target so repository docs are copied to dirtydishes.github.io/islandflow/docs/ instead of attempting to force-push a gh-pages branch back to the mirrored islandflow repository.

    -

    Changes Made

    • Changed the workflow permission from write to read for the source repository, because publishing now happens through the target Pages checkout.
    • Added explicit Pages target settings for dirtydishes/dirtydishes.github.io, branch master, and prefix islandflow.
    • Added a target-repository checkout that uses secrets.DOCS_PAGES_TOKEN.
    • Mirrors the static payload into pages/islandflow/, which serves the docs at /islandflow/docs/.
    • Commits and pushes only when the generated mirror content actually changed.
    • Tightened the docs index generator so regenerated HTML does not introduce trailing whitespace.
    • Regenerated docs/index.html so the docs index reflects the current repository docs set.
    +

    Changes Made

    • Changed the workflow permission from write to read for the source repository, because publishing now happens through the target Pages checkout.
    • Added explicit Pages target settings for dirtydishes/dirtydishes.github.io, branch master, and prefix islandflow.
    • Added a target-repository checkout that uses secrets.DOCS_PAGES_TOKEN.
    • Mirrors the static payload into pages/islandflow/, which serves the docs at /islandflow/docs/.
    • Commits and pushes only when the generated mirror content actually changed.
    • Deletes .DS_Store files from the generated site payload before publishing.
    • Tightened the docs index generator so regenerated HTML does not introduce trailing whitespace.
    • Regenerated docs/index.html so the docs index reflects the current repository docs set.
    • Manually published the current docs snapshot to dirtydishes/dirtydishes.github.io while the automated mirror path awaits secret verification.

    Context

    The previous workflow built a valid static payload, but its publish step initialized a temporary Git repository and pushed gh-pages to github.com/$GITHUB_REPOSITORY. That targets the dirtydishes/islandflow mirror, not the user Pages repository that serves dirtydishes.github.io.

    Cross-repository publishing cannot rely on the default GITHUB_TOKEN from the source workflow. The fixed workflow uses a dedicated secret token for the Pages site checkout and push.

    Important Implementation Details

    • The public URL remains https://dirtydishes.github.io/islandflow/docs/.
    • The target repository branch is master, matching the current remote state of dirtydishes/dirtydishes.github.io.
    • The workflow removes and replaces only the islandflow/ subtree in the Pages repository, leaving any other site content alone.
    • The workflow requires a GitHub Actions secret named DOCS_PAGES_TOKEN with permission to read and write dirtydishes/dirtydishes.github.io.

    Relevant Diff Snippets

    Rendered with @pierre/diffs/ssr using server-side generated markup so this report is readable offline without loading remote JavaScript.

    128 unmodified lines
    129
    130
    131
    132
    133
    134
    135
    2 unmodified lines
    138
    139
    140
    141
    142
    143
    144
    145
    146
    147
    148
    149
    150
    151
    152
    153
    128 unmodified lines
    .map((entry) => {
    const extension = path.extname(entry.relativePath).replace(".", "") || "file";
    const searchable = `${entry.relativePath} ${category}`.toLowerCase();
    return ` <li class="doc-item" data-search="${escapeHtml(searchable)}">
    <a class="doc-link" href="${docsHref(entry.relativePath)}">${escapeHtml(
    entry.relativePath
    )}</a>
    2 unmodified lines
    <span>${escapeHtml(formatBytes(entry.sizeBytes))}</span>
    <span>${escapeHtml(dateFormatter.format(entry.modifiedAt))}</span>
    </div>
    </li>`;
    })
    .join("\n");
    return ` <section class="group" id="category-${escapeHtml(category)}">
    <h2>${escapeHtml(category)} <span>${entries.length}</span></h2>
    <ul class="doc-list">
    ${entryMarkup}
    </ul>
    </section>`;
    })
    .join("\n");
    -

    Expected Impact for End-Users

    Once the workflow runs on GitHub with DOCS_PAGES_TOKEN configured, readers should be able to browse the Islandflow docs at dirtydishes.github.io/islandflow/docs/ from the actual Pages site instead of depending on a missing or unconfigured project gh-pages branch.

    -

    Validation

    • Passed: Ran node scripts/generate-docs-index.mjs; it generated docs/index.html with 64 entries.
    • Passed: Ran node --check scripts/generate-docs-index.mjs.
    • Passed: Parsed .github/workflows/docs-pages.yml with Ruby YAML.
    • Passed: Ran git diff --check after tightening the generated index output.
    • Not completed: bunx actionlint .github/workflows/docs-pages.yml still failed because the package did not expose a runnable executable in this environment.
    -

    Issues, Limitations, and Mitigations

    • The workflow will fail until DOCS_PAGES_TOKEN exists in the GitHub mirror repository secrets and can push to dirtydishes/dirtydishes.github.io.
    • This was validated locally for generation and YAML syntax, but the cross-repository push itself can only be fully verified by a GitHub Actions run.
    • The docs index includes file modified timestamps, so regenerating it can change ordering and metadata when local mtimes differ from the committed version.
    +

    Expected Impact for End-Users

    Readers can browse the Islandflow docs at dirtydishes.github.io/islandflow/docs/ from the actual Pages site. Once DOCS_PAGES_TOKEN is configured, future updates should arrive through the automated mirror workflow instead of manual publication.

    +

    Validation

    • Passed: Ran node scripts/generate-docs-index.mjs; it generated docs/index.html with 65 entries.
    • Passed: Ran node --check scripts/generate-docs-index.mjs.
    • Passed: Parsed .github/workflows/docs-pages.yml with Ruby YAML.
    • Passed: Ran git diff --check after tightening the generated index output.
    • Passed: Published islandflow/docs to dirtydishes/dirtydishes.github.io and confirmed https://dirtydishes.github.io/islandflow/docs/ returns HTTP 200 with the docs index.
    • Not completed: bunx actionlint .github/workflows/docs-pages.yml still failed because the package did not expose a runnable executable in this environment.
    +

    Issues, Limitations, and Mitigations

    • The automated workflow will fail until DOCS_PAGES_TOKEN exists in the GitHub mirror repository secrets and can push to dirtydishes/dirtydishes.github.io.
    • The live site is currently repaired by a direct Pages repository publish; the workflow still needs a real GitHub Actions run after secret verification.
    • The docs index includes file modified timestamps, so regenerating it can change ordering and metadata when local mtimes differ from the committed version.

    Follow-up Work

    • islandflow-8a6: Add or verify DOCS_PAGES_TOKEN in the GitHub mirror repository secrets, then manually run the Publish Docs workflow after this branch lands on the mirrored main.
    • Consider replacing timestamp-based docs index ordering with commit dates or filenames so generated output is less sensitive to local filesystem mtimes.