Refactor build and watch internals into cohesive subsystems by bcomnes · Pull Request #333 · bcomnes/domstack

bcomnes · 2026-09-17T01:31:42Z

Summary

Make build and watch internals easier to understand by separating orchestration from the state and behavior owned by each subsystem, while keeping the public API intact.

Keep DomStack as a small public facade, with watch lifecycle, event planning, dependency tracking, output ownership, and logging collected under lib/watch/.
Separate page preparation, vars, global data and subscriptions, layouts, generated pages, output writing, and worker transport under lib/build-pages/.
Remove redundant copying, iteration, module imports, and repeated renderer preparation. Preserve structured subscription-error metadata and improve Markdown plugin typing.
Restore test-cases/ to checked-in example sites with build/output assertions. Move detailed regression tests beside their subsystems and public type contracts into type-tests/, with guidelines for maintaining that distinction.
Remove make-array and the test:version-build verification stage. The actual versioning and publishing hooks remain unchanged.

Behavior and compatibility

Watch still distinguishes producer-input changes from output selection. Generated pages remain downstream of global data, partial writes remain tracked for recovery, and successful watch baselines advance only after rendering and cleanup succeed. Published nested data stays shared; non-enumerable-key behavior is unchanged.

Renderer preparation now happens once per initialized page. HTML and Markdown content is captured during initialization rather than reread on each render. Render calls still receive current inputs, output hooks remain lazy, and every watch page build prepares fresh renderers in a fresh worker. readMarkdownContent() remains an independent filesystem read. This is a per-page snapshot, not an atomic snapshot of the entire site.

The optimizations remove redundant work, but no end-to-end speedup is claimed. Retaining prepared renderers also retains their source content for the build lifetime; timing and peak-memory benchmarking remain follow-up work.

Review guide

Start with the watch and page-build subsystem boundaries, then review the prepared-renderer behavior and its regression tests. The later test-reorganization commit separates example-site acceptance coverage from detailed behavioral coverage without changing runtime implementation. Small fixes and follow-up cleanups are recorded in separate commits.

Validation

Latest full Node suite: 586 passed, 2 existing TODOs, 0 failures.
TypeScript and full ESLint checks passed.
Regression coverage includes edits during active builds, settings and imported-dependency changes, subscriber invalidation, lazy hooks, and partial-write recovery.
Reorganization audit confirmed every original named test remains and assertion counts did not decrease.
Package dry-run and declaration configuration exclude relocated test-only code; no declaration build was needed.
Diff whitespace checks passed. Existing intentionally invalid fixture and upstream dependency editor diagnostics remain.

…nership Extract the public facade's watch implementation into lib/watch, retain the pure planner, and add focused lifecycle and bookkeeping tests. Reduce repeated planning, dependency analysis, ownership scans, and disabled logging work.

…outputs Extract the direct coordinator and worker protocol, group producer/subscriber helpers and output writers, and isolate streaming factory expansion. Preserve API/type exports and worker error metadata; document module ownership and add protocol regressions.

Avoid source-array allocation on vars cache hits and merge into one target while preserving spread semantics and snapshot timing. Reduce subscriber bookkeeping allocations, skip unselected factory collision setup, and reuse full-build page selections. Add regression coverage for semantics and skipped work.

coveralls · 2026-09-17T01:36:41Z

Coverage Report for CI Build 35180175492

Warning

Build has drifted: This PR's base is out of sync with its target branch, so coverage data may include unrelated changes.
Quick fix: rebase this PR. Learn more →

Coverage increased (+0.4%) to 97.048%

Details

Coverage increased (+0.4%) from the base build.
Patch coverage: 18 uncovered changes across 3 files (2994 of 3012 lines covered, 99.4%).
No coverage regressions found.

Uncovered Changes

File	Changed	Covered	%
lib/watch/index.js	595	585	98.32%
lib/build-pages/generated-pages/index.js	201	195	97.01%
lib/build-pages/global-data/resolve-global-data.js	35	33	94.29%
Total (59 files)	3012	2994	99.4%

Coverage Regressions

No coverage regressions found.

Coverage Stats


Relevant Lines:	10940
Covered Lines:	10736
Line Coverage:	98.14%
Relevant Branches:	3357
Covered Branches:	3139
Branch Coverage:	93.51%
Branches in Coverage %:	Yes
Coverage Strength:	589.27 hits per line

💛 - Coveralls

socket-security · 2026-09-17T02:23:03Z

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff	Package	Supply Chain Security	Vulnerability	Quality	Maintenance	License
	npm/@types/markdown-it-emoji@3.0.1

View full report

Copilot

🔵 Needs a closer look

The change set is a large cross-cutting refactor across build, watch, worker transport, and public types, which warrants final human review despite the added tests.

Pull request overview

This pull request refactors DOMStack’s build and watch internals into more cohesive subsystems, with accompanying type-coverage updates, new/relocated tests, and documentation adjustments to reflect the new boundaries and behaviors.

Changes:

Reorganizes watch planning, logging, and output-ownership/cleanup into lib/watch/* subsystems with expanded regression coverage.
Extracts and groups page-build functionality (worker protocol, global-data handling, generated pages, vars caching, outputs collection/writing) under clearer lib/build-pages/* boundaries.
Updates TypeScript declarations/type-tests and repo configuration (tsconfig/declaration excludes, packaging excludes, docs) to align with the refactor.

File summaries

File	Description
types/thread-stream.d.ts	Clarifies worker_threads aliasing comment for Node versions.
types/markdown-it-plugins.d.ts	Adds missing typings for markdown-it plugins.
types.ts	Updates public type export paths after refactor.
type-tests/registry-subscriptions.test.ts	Adds type contract coverage for subscription metadata stripping.
type-tests/registry-required-vars.test.ts	Adds type contract coverage for required vars inference.
type-tests/registry-page-outputs.test.ts	Adds type contract coverage for pageOutputs typing and context.
type-tests/index.test.ts	Fixes type-test imports/paths for new layout.
type-tests/data-deps-type-checks.ts	Adds compile-time checks for `DataDeps` and producer contracts.
tsconfig.json	Expands include set; keeps error fixtures excluded.
test-fixtures/test-build/src/README.md	Adds minimal fixture README for testBuild output.
test-fixtures/test-build/copyfolder/copied.txt	Adds copy fixture for testBuild.
test-cases/README.md	Documents test-cases organization and expectations.
test-cases/page-outputs/src/root.layout.js	Adds page-outputs acceptance fixture layout.
test-cases/page-outputs/src/global.data.js	Adds page-outputs acceptance fixture global data.
test-cases/page-outputs/src/article/page.vars.js	Adds pageOutputs fixture via companion vars.
test-cases/page-outputs/src/article/page.md	Adds markdown content fixture for sidecar read.
test-cases/nested-layouts/type-checks.ts	Updates imports to moved layout/page-writer modules.
test-cases/nested-layouts/src/typed/page.ts	Adds typed page fixture.
test-cases/nested-layouts/src/source/style.css	Adds nested-layouts CSS fixture.
test-cases/nested-layouts/src/source/page.vars.js	Adds nested-layouts vars fixture.
test-cases/nested-layouts/src/source/page.md	Adds nested-layouts markdown fixture.
test-cases/nested-layouts/src/source/client.js	Adds nested-layouts client fixture.
test-cases/nested-layouts/src/root.layout.js	Adds nested-layouts root layout fixture.
test-cases/nested-layouts/src/root.layout.css	Adds nested-layouts root CSS fixture.
test-cases/nested-layouts/src/root.layout.client.js	Adds nested-layouts root client fixture.
test-cases/nested-layouts/src/post.layout.js	Adds nested-layouts child layout fixture.
test-cases/nested-layouts/src/post.layout.css	Adds nested-layouts child CSS fixture.
test-cases/nested-layouts/src/post.layout.client.js	Adds nested-layouts child client fixture.
test-cases/nested-layouts/src/plain/page.html	Adds nested-layouts plain HTML fixture.
test-cases/nested-layouts/src/other.layout.js	Adds nested-layouts alternate layout fixture.
test-cases/nested-layouts/src/other-label.js	Adds nested-layouts alternate helper fixture.
test-cases/nested-layouts/src/markup/page.vars.js	Adds nested-layouts markup vars fixture.
test-cases/nested-layouts/src/markup/page.html	Adds nested-layouts markup HTML fixture.
test-cases/nested-layouts/src/label.js	Adds nested-layouts shared helper fixture.
test-cases/nested-layouts/src/global.vars.js	Adds nested-layouts global vars fixture.
test-cases/nested-layouts/src/global.css	Adds nested-layouts global CSS fixture.
test-cases/nested-layouts/src/global.client.js	Adds nested-layouts global client fixture.
test-cases/nested-layouts/src/article.layout.js	Adds nested-layouts middle layout fixture.
test-cases/nested-layouts/src/article.layout.css	Adds nested-layouts middle CSS fixture.
test-cases/nested-layouts/src/article.layout.client.js	Adds nested-layouts middle client fixture.
test-cases/nested-layouts/src/archive.pages.js	Adds generated-pages fixture within nested layouts.
test-cases/generated-pages/redirects.test.js	Adds redirects validation coverage for page-owned metadata.
test-cases/general-features/src/worker-page/page.js	Updates import path to moved page-writer types.
test-cases/general-features/src/markdown-it.settings.js	Replaces ts-ignore with more specific ts-expect-error.
test-cases/general-features/src/global.data.js	Replaces ts-ignore with more specific ts-expect-error.
test-cases/general-features/src/feeds.template.js	Replaces ts-ignore with more specific ts-expect-error.
test-cases/build-errors/src/another-broken-page/page.js	Removes stray ts-ignore in intentional error fixture.
test-build.test.js	Updates testBuild import/pathing and fixture root.
site/layouts/docs/navigation-watch.test.js	Switches to watch test helpers under `lib/watch/`.
scripts/verify-version-build.js	Removes version-build verification script.
package.json	Updates files/devDeps and removes make-array; adjusts build ignore list.
lib/watch/test-helpers.js	Adds shared watch test helpers for real chokidar observation.
lib/watch/rebuilds.test.js	Updates fixture paths and uses new watch test helpers.
lib/watch/prepared-renderers.test.js	Adds coverage for renderer snapshotting across watch batches.
lib/watch/plan.test.js	Adds batch planning perf/ordering regression tests.
lib/watch/plan.js	Refactors batch unioning and avoids unnecessary source scans.
lib/watch/page-outputs.test.js	Adds watch coverage for cross-session output ownership cleanup.
lib/watch/page-output-ledger.js	Introduces retained output ownership/cache across watch sessions.
lib/watch/output-ownership.test.js	Strengthens assertions on page reports/ownership metadata.
lib/watch/output-cache.test.js	Updates imports to new test helpers and watch helpers.
lib/watch/logging.test.js	Adds coverage for new watch/build logging behaviors.
lib/watch/logging.js	Adds watch/build logging utilities with level guards.
lib/watch/lifecycle-tests/logging.test.js	Fixes DomStack import path for lifecycle tests.
lib/watch/lifecycle-tests/index.test.js	Adds watch lifecycle assertions and fixes import path.
lib/watch/incremental-global-data-tests/helpers.js	Updates imports to relocated watch helpers and DomStack entry.
lib/watch/incremental-global-data-tests/fixtures/producer-middle.js	Adds fixture to test prepared imports.
lib/watch/incremental-global-data-tests/fixtures/producer-leaf.js	Adds fixture leaf module.
lib/watch/incremental-global-data-tests/fixtures/global.data.js	Adds incremental global-data fixture producer.
lib/watch/incremental-global-data-tests/fixtures/data.json.template.js	Adds template fixture for incremental data.
lib/watch/generated-page-ownership.test.js	Adds watch coverage for generated page ownership cleanup.
lib/identify-pages.js	Moves computePageUrl import and tightens JSDoc option types.
lib/helpers/fs-path-to-url.test.js	Adds tests for filesystem-path URL conversion helper.
lib/helpers/fs-path-to-url.js	Adds helper to convert OS paths to URL paths.
lib/helpers/compute-page-url.test.js	Adds tests for computePageUrl behavior.
lib/helpers/compute-page-url.js	Updates computePageUrl to use new fs-path helper.
lib/domstack-manifest/settings.js	Updates resolveVars import path after refactor.
lib/domstack-manifest/records.js	Tightens JSDoc optional property typing.
lib/cli/tests/logging.test.js	Fixes bin.js path after refactor.
lib/cli/tests/index.test.js	Fixes bin.js path after refactor.
lib/cli/tests/eject.test.js	Fixes project root resolution after refactor.
lib/cli/tests/commands.test.js	Fixes bin/package.json path after refactor.
lib/build-pages/worker/worker.js	Splits build coordinator vs error protocol import.
lib/build-pages/worker/protocol.js	Adds worker error serialization/restore with domain metadata.
lib/build-pages/worker/page-outputs.test.js	Adds worker-transport tests for subscription errors in pageOutputs.
lib/build-pages/worker/index.js	Introduces worker runner coordinating buildPages in a Worker.
lib/build-pages/worker/generated-pages.test.js	Adds worker-boundary tests for generated pages and errors.
lib/build-pages/vars/resolve-vars.test.js	Updates fixture paths and improves TS expectation comment.
lib/build-pages/vars/resolve-vars.js	Introduces shared vars resolver with resolveVarsExport helper.
lib/build-pages/vars/page-vars.test.js	Adds tests for vars caching and retry semantics.
lib/build-pages/vars/page-vars.js	Adds PageVars cache to reduce merge/copy overhead.
lib/build-pages/templates/template-builder.test.js	Updates WatchDependencyTracker import path.
lib/build-pages/templates/template-builder.js	Updates data-deps and watch-deps import paths.
lib/build-pages/resolve-vars.js	Removes legacy resolve-vars module (moved/split).
lib/build-pages/page/resolve-page-companion.test.js	Adds tests for companion resolution and postVars ordering.
lib/build-pages/page/resolve-page-companion.js	Adds companion resolver keeping exports live but vars resolved.
lib/build-pages/page/page-data-vars-catch.test.js	Adds regression test for vars merge error wrapping/cause.
lib/build-pages/page/page-data-renderer.test.js	Adds tests for prepared renderers and live inputs semantics.
lib/build-pages/page/page-data-page-outputs.test.js	Adds initialization/provider-selection ordering tests.
lib/build-pages/page-builders/md/index.js	Updates PageBuilderType import path.
lib/build-pages/page-builders/md/get-md.test.js	Adds plugin registration + renderMd handlebars gating tests.
lib/build-pages/page-builders/md/get-md.js	Removes ts-ignores and tightens renderMd vars typing.
lib/build-pages/page-builders/md/get-md-types.test.ts	Adds compile-time plugin typing assertions.
lib/build-pages/page-builders/js/index.js	Updates validatePageOutputsHook import path.
lib/build-pages/page-builders/index.js	Re-exports template builder from new location.
lib/build-pages/page-builders/html/index.js	Removes ts-ignore and tightens handlebars gating check.
lib/build-pages/outputs/test-helpers.js	Fixes DomStack/builder imports for output tests.
lib/build-pages/outputs/resolve-page-output-provider.test.js	Adds tests for provider precedence and validation behavior.
lib/build-pages/outputs/resolve-page-output-provider.js	Adds provider resolver with duplicate-provider warning.
lib/build-pages/outputs/page-writer.js	Updates PageData/pageOutputs type imports and removes alias typedef.
lib/build-pages/outputs/page-outputs.test.js	Adds normalization/validation iterator behavior tests.
lib/build-pages/outputs/page-outputs.js	Preserves DomStackDataError subtype through normalization failures.
lib/build-pages/outputs/page-outputs-types.test.ts	Updates imports and output-writer paths for type checks.
lib/build-pages/outputs/page-output-writer.js	Updates PageData import path after refactor.
lib/build-pages/outputs/collect-page-outputs.js	Adds output collection with layout + page provider ordering.
lib/build-pages/layouts/subscriptions-build.test.js	Adds coverage for nested renderer subscription isolation.
lib/build-pages/layouts/resolve-layout.test.js	Adds resolveLayout vars export coverage.
lib/build-pages/layouts/resolve-layout.js	Adds layout module resolver + pageOutputs validation.
lib/build-pages/layouts/resolve-layout-name.test.js	Adds tests for layout name resolution precedence and errors.
lib/build-pages/layouts/resolve-layout-name.js	Adds layout-name resolver avoiding unrelated var reads.
lib/build-pages/layouts/resolve-layout-chain.test.js	Updates resolveLayout import path for chain tests.
lib/build-pages/layouts/resolve-layout-chain.js	Adds layout chain resolver with cycle detection.
lib/build-pages/layouts/nested-test-helpers.js	Adds shared nested-layouts test setup helpers.
lib/build-pages/global-data/watch-dependencies.test.js	Adds ordering/enumeration/immutability regression tests.
lib/build-pages/global-data/watch-dependencies.js	Refactors fingerprint diffing and avoids Object.values/entries overhead.
lib/build-pages/global-data/resolve-global-data.js	Extracts global.data resolver from legacy module.
lib/build-pages/global-data/page-subscriptions.test.js	Adds PageSubscriptions coverage for readiness + access narrowing.
lib/build-pages/global-data/page-subscriptions.js	Adds per-page subscription binding/access layer.
lib/build-pages/global-data/global-data-state.test.js	Updates imports for new watch plan + global data resolver.
lib/build-pages/global-data/global-data-state.js	Updates imports for PageData and WatchEvent paths.
lib/build-pages/global-data/global-data-state-types.test.ts	Updates types.ts import path for type checks.
lib/build-pages/global-data/data-deps.js	Fixes DomStackDataError import path after refactor.
lib/build-pages/generated-pages/test-helpers.js	Adds reusable temp-fixture helpers for generated pages tests.
lib/build-pages/generated-pages/streaming.test.js	Refactors test setup into streaming-test-helpers.
lib/build-pages/generated-pages/streaming-test-helpers.js	Adds shared helper for streaming generated pages tests.
lib/build-pages/generated-pages/index.test.js	Adds generated-pages selection/collision regression tests.
lib/build-pages/generated-pages/index.js	Adds generated page expansion, validation, and conflict detection.
lib/build-esbuild/settings.test.js	Adds esbuild settings regression coverage (metafile/defines).
lib/build-esbuild/index.js	Updates resolveVars import and tightens JSDoc types.
index.test.js	Fixes DomStack import path and adds ignore normalization tests.
examples/tailwind/src/layouts/root.layout.js	Removes unused ts-ignore in example.
docs/pages/README.md	Documents renderer preparation and watch rebuild behavior.
docs/implementation/README.md	Updates watch session description language.
docs/api/README.md	Updates testBuild reference link.
declaration.tsconfig.json	Excludes additional test/helpers/fixtures from declaration build.
CONTRIBUTING.md	Removes mention of removed version-build test script.
agents.md	Documents additional repo testing/organization guidelines.
.gitignore	Removes stale ignore entry for old streaming fixture dir.

Review details

Files reviewed: 147/171 changed files
Comments generated: 1
Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

bcomnes added 7 commits September 16, 2026 18:21

fix(page-outputs): preserve subscription error metadata

71fca07

fix(types): accept explicit undefined in optional build options

6b958c1

refactor: replace make-array with native ignore normalization

dc66aad

docs: keep implementation guide focused on high-level behavior

c1f8c01

bcomnes added 4 commits September 16, 2026 19:07

refactor(build-pages): collect remaining subsystem boundaries

cf59229

refactor(types): remove redundant internal aliases

68fc3a4

fix(types): type Markdown plugins without suppressions

f1a876d

chore(types): explain expected errors and remove stale suppressions

7ebf7d3

bcomnes added 8 commits September 16, 2026 19:28

refactor(build-pages): isolate per-page data subscriptions

b4c2d4e

refactor(build-pages): isolate page vars caching

35d7b5a

refactor(build-pages): extract lazy output hook collection

beb0843

Simplify page preparation and output reporting

6993b30

Reuse prepared page renderers within each build

d1e0f5d

Separate example-site acceptance tests from subsystem regressions

c2bec1f

Document example-site and subsystem test boundaries

de63ce5

Remove version-build verification test stage

8119151

bcomnes marked this pull request as ready for review September 17, 2026 03:48

bcomnes requested a lite review from Copilot September 17, 2026 03:49

Copilot started reviewing on behalf of bcomnes September 17, 2026 03:49 View session

Copilot AI reviewed Sep 17, 2026

View reviewed changes

Comment thread lib/identify-pages.js Outdated

Fix typo in JSDoc for buildDrafts option

0a8fcd2

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

bcomnes merged commit bca4086 into master Sep 17, 2026
6 checks passed

bcomnes deleted the refactor/build-watch-subsystems branch September 17, 2026 03:58

Navigation Menu

Uh oh!

Refactor build and watch internals into cohesive subsystems - #333

Refactor build and watch internals into cohesive subsystems#333
bcomnes merged 20 commits into
masterfrom
refactor/build-watch-subsystems

bcomnes commented Sep 17, 2026 •

edited

Loading

Uh oh!

coveralls commented Sep 17, 2026 •

edited

Loading

Uh oh!

socket-security Bot commented Sep 17, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

bcomnes commented Sep 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Behavior and compatibility

Review guide

Validation

Uh oh!

coveralls commented Sep 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Coverage Report for CI Build 35180175492

Coverage increased (+0.4%) to 97.048%

Details

Uncovered Changes

Coverage Regressions

Coverage Stats

💛 - Coveralls

Uh oh!

socket-security Bot commented Sep 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🔵 Needs a closer look

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

bcomnes commented Sep 17, 2026 •

edited

Loading

coveralls commented Sep 17, 2026 •

edited

Loading

socket-security Bot commented Sep 17, 2026 •

edited

Loading