--- url: /conventions/build-and-docs.md --- # Build, release, and the docs site ## Use the `vp` toolchain, never pnpm directly * `vp` / `vpr` / `vpx` are **global** bins. Run tasks via `vp run `, execute bins via `vp exec [-F ] `. * **Package scripts must never shell out to `pnpm`.** CI (`.github/workflows/docs.yml`) provides `vp` via `voidzero-dev/setup-vp` but **not** `pnpm` on PATH, so a `pnpm run …` inside any script breaks the docs deploy with `pnpm: command not found`. Local dev has pnpm, so a top-level `pnpm run X` is fine — but scripts it calls must stay pnpm-free internally. * `.css` formatting: `vp fmt` (oxc) is JS/TS/JSON only and no-ops on `.css`. Stylelint owns core CSS correctness; cssdoc owns documentation comments. `lint:css` and `lint:cssdoc` run in `ready`, and both are wired into the `vite.config.ts` staged hook. ## The gate * `pnpm run ready` — the pass/fail gate. It's a `vp` task DAG (`ready:all`), not a serial chain: `build:all` runs once, then `check:all` (`vp check`), `test:all` (`vp run -r test`), `lint:css`, `lint:cssdoc`, `validate:generated:only`, and `lint:markdown` fan out concurrently. Everything that reads generated output depends on `build:all`, so generation happens exactly once (no concurrent codegen race). Must pass before you're done. * `pnpm run check:publish` — the publish gate (`gate:publish`): `gate:repository` (asserts every publishable manifest has the `repository.url` npm OIDC provenance needs), `gate:publint`, and `gate:attw`. Publint/attw depend on `build:all`; the repository check is a pure manifest read. * **CI runs the same checks on every PR** (`.github/workflows/ci.yml`): `build` → parallel `typecheck`/`test`/`lint`/`publint`/`attw`, plus a `repository` job and a `commitlint` job that lints the PR's commit range. Jobs share a persisted vp task cache (`node_modules/.vite/task-cache`), so the `build` job warms it and downstream jobs restore it and re-materialize generated dirs from cache. * **`publint`/`attw` are scoped to what the PR touches.** `scripts/release/changed-packages.ts` maps the diff against the base branch to the affected publishable packages (each changed package plus its workspace dependents, since a dependency change can break a dependent's pack/types), and the jobs run `vp exec -F ` instead of packing all ~70. A change to a global file (root `package.json`, lockfile, `vite.config.ts`, …) widens back to the full gate; a change touching no publishable package skips them. * **Commit messages are conventional-commit-linted** locally by the `.vite-hooks/commit-msg` hook (`vp exec commitlint --edit`) and in CI by the `commitlint` job. * Release automation uses Changesets and package-tag workflows in `.github/workflows/release.yml`. That workflow verifies a clean build/typecheck/test + `gate:repository`, then scopes the pack-heavy `publint`/`attw` gates to exactly the publish set (the full `ready` already ran on the merge-to-main the tag points at). * Use `pnpm run release:version` to apply package changelog/version updates, then `vp run release:changelog:root` to rebuild the strict chronological root `CHANGELOG.md` before creating package tags. * Generated artifacts are gitignored and reproduced on build: `platforms/tokens/src/generated/`, `platforms/css/style.css`, each preprocessor's static file, and web-components `src/generated/`. `build:all` produces them before any gate that reads them runs. See `docs/internal/release-strategy.md` for the runbook, dist-tag mapping, prerelease flow, and npm organization governance model. ## Linting CSS Root `stylelint.config.js` runs error-only core CSS rules; anchor-positioning props are ignored and `@scope` is allowed. `lint:cssdoc` runs `@cssdoc/cli` with `--max-warnings 0` over the same web-components sources and generated components CSS, making it the single cssdoc lint instance. Stylelint remains because the CLI covers doc hygiene, not the 24 core CSS correctness rules. ## The docs site There are three site builds, and they exist for different jobs: | Script | Locale API | Where it runs | Why | | ------------------- | ---------- | ------------------------------- | ---------------------------------------------------------------------------------------- | | `docs:build` | English | CI, every PR and main push | A fast breakage check. Sets `DOCS_ROOT_LOCALE_ONLY=1`, so `/api/**` is excluded. | | `docs:build:deploy` | All | The `Deploy docs` workflow only | What ships. Deterministic `glossary` adapter — no AI, no network. | | `docs:build:all` | All | Local | `docs:build:deploy` plus `docs:check:locales` (the parity gate). | `docs:build` renders a site whose non-root locales have no API tree, so under `DOCS_ROOT_LOCALE_ONLY=1` `config.ts` points every localized API nav link, sidebar route, and home-hero action at the English `/api/` tree instead of a route it didn't build. Don't remove that fallback — it's what keeps the English-only build internally consistent. **Deploys are diff-scoped.** `Deploy docs` can run from three paths: a docs-affecting push to `main`, the `Release` workflow's publish path via its `docs-deploy-request` marker artifact, or a manual `workflow_dispatch`. The workflow first runs `docs/scripts/changed-pages.ts` against the selected base ref. Page-scoped guide/API/home changes can render through `docs:build:deploy:partial`; global inputs (VitePress config/theme, docs scripts, public assets, shared component/token/plugin sources, lockfile, and deleted pages) fall back to the full `docs:build:deploy` route. Manual dispatches default to the full build unless a base ref is supplied. Partial deploy builds write VitePress output to `docs/.vitepress/partial-dist`, then overlay that onto a restored complete `docs/.vitepress/dist` cache before preparing the deploy. If the full dist cache is missing, a partial candidate falls back to the full docs build. The deploy workflow builds the site itself; CI no longer uploads a `docs-site` artifact. **The site deploys as a static site tree plus an asset bucket behind an edge router.** The full-locale site is ~42k pages / ~138k files / ~3.8 GB, which is past the size, timeout, and per-directory file caps of typical static hosts. The deploy workflow splits the output via `docs/scripts/prepare-deploy.ts`, which also fails the deploy if the static tree exceeds 100,000 files or any file exceeds 25 MiB: * **Static site tree** (`docs/.vitepress/deploy-site`): HTML pages (44 locales), root files (`sitemap.xml`, `hashmap.json`, `robots.txt`, `favicon.*`, `llms.txt`), and shadcn registry files (`/r/*`) (~43k files). * **Asset bucket** (`docs/.vitepress/deploy-assets`): High-volume hashed client bundles (`/assets/*`) and demo assets (`/demos-assets/*`) (~95k files), synced via `docs/scripts/sync-assets.ts`. Only changed objects upload. * **Edge router** (`docs/edge/src/index.ts`, configured by `docs/edge/wrangler.jsonc`): Routes `/assets/*` and `/demos-assets/*` to the bucket with immutable caching headers (`public, max-age=31536000, immutable`) and falls back to the static site tree for HTML and root documents. Deploy needs the `CLOUDFLARE_API_TOKEN` and `CLOUDFLARE_ACCOUNT_ID` secrets; `DOCS_ASSETS_BUCKET` and `DOCS_ASSETS_UPLOAD_CONCURRENCY` are optional repo variables. **The deploy build is memory-bound.** Three settings keep it inside a 16 GB runner, and all three have a reason: * `themeConfig.search.options._render` skips `/api/**`. The local-search plugin runs a *second* full markdown-it pass over every page and holds one MiniSearch index per locale in memory for the whole build; indexing the translated API mirrors meant ~38k extra renders and ~77 MB of retained indexes, to produce per-locale indexes so large that opening search would have downloaded them. * `buildConcurrency` (`DOCS_BUILD_CONCURRENCY`) drops from VitePress's default of 64 to 12. Every in-flight page holds its rendered HTML and Vue SSR context alive. * The workflow swaps the runner's 4 GB swapfile for 24 GB. When peak RSS crosses physical memory the kernel OOM-killer takes out the runner *agent*, and the job reports only "the runner has received a shutdown signal" — no V8 heap error, no stack, no clue. Swap converts that into slow progress. **Why the full build stays in CI.** The localized API tree is ~875 pages per locale across 44 locales — roughly 35k generated files. Those are gitignored on purpose (they were committed once and removed), so building locally would mean either re-committing them or hand-pushing `dist`. Paying for one full render per release in CI keeps the generated output out of git and off your machine. The translation half is already incremental: `TranslationMemory` serves unchanged strings from `l10n//docs.api.po`, so a guide or API edit only re-translates what actually changed. `docs/` is a VitePress site (`@pantoken/docs`) with two locales — `root` (English, `/…`) and `hu` (Magyar, `/hu/…`) — a symmetric prefix swap that VitePress's default routing already handles (don't set a custom `i18nRouting`). * **Translation layer is `docs/.vitepress/i18n.ts`.** `LOCALE_THEMES[locale]` holds every localizable UI string (nav/sidebar labels, `editText`, the theme selector, VitePress chrome labels, and local search). `config.ts` expands these into per-locale `themeConfig` (search is the exception — it lives in the global `themeConfig.search.options.locales`). Add new UI strings here, never inline. * **Block-level API translation.** `build-api-locales.ts` doesn't translate whole `.md` files — it runs `segment-markdown.ts` to split each generated page into blocks: `prose` (descriptions, remarks, `@example` captions, visible text nodes in HTML example fences, cssdoc table Description cells), `glossary` (section headings, stability-badge pills, table column labels), and `preserve` (non-HTML code fences, signatures, breadcrumbs, token tables). HTML example markup, attributes, comments, and script/style/code contents remain verbatim. Only prose carries a content key, so a page's prose survives the scaffolding churn (badge flips, token-value changes, signature edits) that used to bust a whole-file key. `glossary` blocks always go through the deterministic `GlossaryTranslationAdapter` (keyless, never cached); `preserve` blocks are emitted verbatim. * **The committed cache carries the prose; CI serves it.** The translation memory (`docs/i18n-cache/hu.api.json`) is content-addressed and adapter-agnostic, so a claude-authored prose entry is served to a `glossary` build as a plain cache hit. The workflow: run `pnpm run docs:api:locales:claude` **locally** to author prose (a cold run is bounded to ~30–40 batched `claude -p` calls, resumable via the memory's autosave), then commit `hu.api.json`. The deploy build (`docs:build:deploy`) runs the `glossary` adapter, which serves that prose from cache and only ever fills structural headings/labels. Brand-new prose that isn't cached yet passes through as English — the glossary **never** caches its own prose passthrough (that would permanently mask the block from a later claude run), so it stays a miss until claude authors it. Never wire `:claude` into CI. * **Cognates are cached, echoes aren't.** Some translations are legitimately identical to English ("Interfaces" in French, "Classes" in Catalan). An identical value from a batch in which other units *did* change is cached and stamped `pantoken-verbatim` in the PO catalog, so it isn't re-flagged and re-paid for on every run. A batch that comes back wholly unchanged still fails the guard — that's the shape of a silently broken adapter. * **Approve known identity labels per locale.** `LOCALE_VERBATIM_API_PROSE` in `docs/scripts/refresh-api-pot.ts` records verified cognates and weekday abbreviations, so they don't depend on another string changing in the same batch. Never exempt all Latin-script locales: ordinary labels such as "Checkbox" still need translation. The targeted API catalog fixer retries fuzzy and unapproved source-identical entries, not just empty entries, and successful translations clear their fuzzy flag. * **Running the cold pass.** Each `claude -p` call cold-starts a full agent, and the dominant cost is the per-call bootstrap — loading MCP servers, plugins, and project settings — not the translation itself (it dwarfs even a small model's inference). So the `:claude` tasks pass `--model claude-haiku-4-5-20251001 --strict-mcp-config --setting-sources user`: a fast model, **no MCP**, and user settings only (keeps auth, drops project/local hooks). That cuts each call from minutes to a few seconds. Override by editing the task or exporting your own `DOCS_TRANSLATION_COMMAND_ARGS` before a direct `node scripts/…` run (`DOCS_TRANSLATION_COMMAND` overrides the `claude` binary itself). Either task logs progress (`… N/M labels + prose blocks translated`) and saves the memory after **each** chunk, so it's resumable — a kill or crash keeps completed chunks and a re-run serves them from cache. * **The cold pass is generation-bound, so it runs chunks concurrently.** Once MCP is stripped, the wall-clock cost is the model streaming translations, not startup — so `ClaudeCodeTranslationAdapter` runs up to `DOCS_TRANSLATION_CONCURRENCY` (default 5) `claude -p` calls at once. Prose is batched at `DOCS_TRANSLATION_BATCH_BUDGET` chars/request (default 4k) — small enough that each JSON response stays reliable and progress is fine-grained, with the pool hiding the per-call startup. A chunk that errors is logged and skipped (its blocks stay uncached and retry next run), never sinking the whole run. Raise concurrency for more speed if you're not rate-limited; lower the budget if a run trips the per-item fallback (the model dropping a key from a large response). * **Every call is bounded by `DOCS_TRANSLATION_TIMEOUT_MS` (default 120000).** A CLI that wedges without exiting used to stall a locale indefinitely (the run appears to stop mid-file list); the timed-out chunk is now killed, logged, and skipped, and its strings retry on the next run. Raise it for a slow model or big batch budget. * **Scope a run with `DOCS_TRANSLATION_LOCALE`.** A locale tag (`hu`), a tier name from `i18n.config.json`'s `locales.tiers` (`primary` → `en-AU en-CA en-GB hu`, `secondary` → the rest), or a comma/space-separated mix of both (`DOCS_TRANSLATION_LOCALE="primary,ga"`). A `-` prefix subtracts — `"-ga"` is every locale but Irish, `"primary,-hu"` is the primary tier without Hungarian. Unset means every locale. An unrecognized entry, or a selection that resolves to nothing (a tier with no docs locale like `source`, or `"hu,-hu"`), throws rather than building an empty directory. Honored by `build-api-locales.ts` and `translate-guide-po.ts`. * **Scope a guide translation with `DOCS_TRANSLATION_FILE`.** Pass one or more comma/space-separated guide paths relative to `docs/` (for example, `DOCS_TRANSLATION_FILE=guide/components.md`) when a force run should repair only selected hand-written guides. Paths are validated against the guide source file list, and unknown values throw rather than silently running every guide. Combine with `DOCS_TRANSLATION_LOCALE` and `DOCS_TRANSLATION_FORCE=1` for a single-file retry. ## Translation drift: what blocks a merge Every drift checker in the repo reports through one shared policy, embedded in `i18n.config.json` at the repo root. A checker no longer decides its own exit code — it hands findings to a `DriftReporter` (`tools/translation-adapters/src/drift-policy.ts`), which resolves a severity per finding and returns the exit code. * **`block`** fails the job, so `ci-gate` blocks the merge. * **`warn`** reports the finding as a GitHub annotation on the PR diff plus a job-summary table, and exits clean. * **`off`** drops the finding entirely. Severity is a `(surface, locale-tier)` matrix. Tiers are named locale groups matched in declaration order, so a specific tier must precede the `"*"` catch-all; patterns use the same syntax as a `VerbatimPolicy` (exact tag, `"prefix*"` glob, or `"*"`). This matters because a hard gate's cost scales with locale count — blocking every surface across ~90 locales means no English string lands until every translation does. | Surface | Checker | What it covers | | -------------- | -------------------------------------- | ------------------------------------------------------------------- | | `ui.strings` | `@pantoken/web-components#check:drift` | `src/i18n.json` → `l10n//ui.strings.po` (via `i18n-engine`) | | `cli.scaffold` | `@pantoken/scaffold#check:drift` | `l10n//cli.scaffold.po` (via `i18n-engine`) | | `cli.ai` | `@pantoken/ai#check:drift` | `l10n//cli.ai.po` (via `i18n-engine`) | | `docs.guides` | `docs:check:drift` | `docs/guide/*.md` → `l10n//docs.guides.po` whole-file units | | `docs.api` | `docs:check:drift` | generated API segments → `l10n//docs.api.po` | | `docs.home` | `docs:check:drift` | translatable `docs/index.md` frontmatter (hero, actions, features) | | `docs.chrome` | `docs:check:drift` | `.vitepress/i18n.json` → `l10n//docs.chrome.po` | | `docs.demos` | `docs:check:drift` | `demos/*/i18n.json` → `l10n//docs.demos.po` | | `docs.parity` | `docs:check:locales` | structural locale-tree parity | didn't run, not that a translator is behind. The committed default: **every configured locale blocks on translation drift.** A missing source key, translation, or structural parity gap fails the build. `docs.parity` blocks for every locale because `docs:build:all` runs `docs:api:locales` before it, so a gap there means a generator didn't run. A surface the config doesn't name inherits `fallback`, which is tier-aware — so a checker's brand-new surface id still blocks on English before anyone edits the policy. Two escape hatches: * `I18N_DRIFT_STRICT=1` still escalates an explicitly configured `warn` to `block` (it never resurrects an `off`). `vp run i18n:check:drift:strict` remains a compatibility alias for the complete blocking audit. * `I18N_CONFIG=/path/to/i18n.config.json` swaps the configuration file. CI wiring: the `i18n-drift` job runs `vp run gate:i18n` when catalog, source, or policy paths change; that gate generates the English API tree and checks every surface. `gate:i18n` also runs before npm publishing. AI translation is never wired into CI; fill drift locally with `vp run i18n:translate`. `translate` re-extracts a space's `l10n/.pot` from its source before merging, so a newly added key reaches the PO catalogs without a separate step, and `check` reports a stale template as drift in its own right. Run `vp run i18n:extract` when you want to refresh a template without spending translation credits. A messages space has no generic render step — the owning package's `generate` script rebuilds its locale bundles from the PO catalogs. `i18n translate` takes four narrowing options: `--locale ` and `--tier ` limit which locales run, `--provider ` picks a `provider.profiles` entry (`copilot`, `agy`, `claude`) for the command, model, and effort, `--concurrency ` bounds parallel provider calls, and `--force` retranslates entries that already have a translation. `I18N_TRANSLATION_COMMAND` and `I18N_TRANSLATION_COMMAND_ARGS` still override whatever `--provider` resolves. ## Publishing the create-pantoken-app skill `packages/scaffold/skills/create-pantoken-app/SKILL.md` is the one canonical source, staged into two places by tasks in `docs/.vitepress/config.ts`: * `stage-create-pantoken-app-skill.ts` → `docs/public/create-pantoken-app.md`, served by the main docs site at `pantoken.app/create-pantoken-app.md`. * `stage-create-pantoken-app-domain.ts` → `ai/create-pantoken-app-site/` (a git submodule pointing at [`thedannywahl/create-pantoken-app`](https://github.com/thedannywahl/create-pantoken-app)), whose static-site deployment serves the same content at the domain root `create.pantoken.app` — a URL-hack shortcut so an agent CLI can fetch the skill without the `/create-pantoken-app.md` path. Both tasks only stage the local working tree. The submodule is a separate GitHub repo, so publishing a skill update to `create.pantoken.app` needs a commit + push inside it. `.github/workflows/ publish-create-pantoken-app.yml` automates this: on every push to `main` that touches `packages/scaffold/skills/create-pantoken-app/**`, it re-runs the staging script, commits+pushes the submodule if changed, then opens a PR that bumps the submodule pointer in this repo. It needs a repo secret `CREATE_PANTOKEN_APP_PAT` (a fine-grained PAT scoped to just `thedannywahl/create-pantoken-app`, Contents: Read and write — the default `GITHUB_TOKEN` can't push to a different repo); without it the workflow warns and skips the push instead of failing. The pointer PR uses `RELEASE_PAT` when available so CI runs automatically; otherwise it uses `GITHUB_TOKEN`, and its checks must be run manually before merging. To do the same thing manually (e.g. testing a change before it's merged, or if the secret isn't set yet): ```sh vp exec node docs/scripts/stage-create-pantoken-app-domain.ts cd ai/create-pantoken-app-site git commit -am "sync create-pantoken-app skill" git push ``` Then, in the main repo, commit the submodule's new pinned commit (`git add ai/create-pantoken-app-site && git commit`) so the two stay in lockstep. ## The cssdoc integration pantoken consumes `@cssdoc/*` from **npm** (catalog entries; consumers use `catalog:`), not a local workspace link. `docs/scripts/build-css-api.ts` is a thin wrapper: it builds pantoken's `resolveToken` (syntax + value + local vars) and `resolveDemo` hooks, then calls `@cssdoc/typedoc`'s `emitCssApi`, and keeps the `unknownReferences` drift guard. All page/index/sidebar rendering lives in `@cssdoc/markdown`. The live `
` preview is pantoken's own `@pantoken/typedoc-plugin-live-example`, kept out of `@cssdoc/markdown` so the upstream stays generic. **Catalog gotcha:** cssdoc packages reference `catalog:` deps; if cssdoc adds a new one, pantoken's catalog must carry it too, or install fails with `ERR_PNPM_CATALOG_ENTRY_NOT_FOUND_FOR_SPEC`. ## The nested-`vp`-spawn limitation Under `vpr docs:dev`, VitePress runs *under* `vp`. Any `vp run …` / `vp pack` spawned from **inside** that process dies with `Failed to spawn process: Invalid argument (os error 22)` on a cache miss. A direct `node scripts/x.ts` is unaffected. So: * The docs orchestrator's `upstream[].build` uses `["node","scripts/generate.ts"]` (cwd = the package dir), not a nested `vp run`. * The CSS-API node runs `["node","scripts/build-css-api.ts"]` (cwd `docs`). * `@pantoken/web-components`' `register()` bundle genuinely needs `vp pack` (which also can't nest), so it's not in the live orchestrator — rebuild it in a separate top-level shell (`vpr @pantoken/web-components#build`); `outputWatchPaths` on its `dist` bridges the change into HMR. A `vp run X && vp run Y` chain *inside* a package.json script is fine — that's a top-level script-runner context, not a spawn from within the running VitePress process.