Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 46 additions & 0 deletions test/perf/rendering/evidence/render-bundle-2026-09-27/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
# Native WebGPU render-bundle spike, 2026-09-27

Current decision: **production candidate accepted for conditional integration**; see [phase 2](phase2/README.md) for the larger scale, reconstruction and churn matrix. The phase-1 measurements and historical recommendation below are preserved as acquired.

Historical phase-1 decision: the evidence gate is met for a static state-churn fixture, and a native per-batch prototype has a repeatable but small CPU benefit. Keep production integration deferred: this result justifies a narrowly scoped follow-up, not general native-bundle support. This establishes a measured triple-digit-batch case for future evaluation.

Measured source: `07d5ea5496381a24b82d491edf763578448574c5`, tree `c57d4ada609cb74bf0a227ecd4e2b72cb6b9e94f`. This is the rendering-integrity implementation state, including the renderer consolidation. The result already includes those changes; it is not a measurement of the preceding architecture. Chromium 153.0.8010.12, Windows, headless, hardware adapter reports NVIDIA / Blackwell. The browser does not expose the exact device name. Three independent browser processes, two scene sizes per process. Raw readings and adapter metadata are in [results.json](results.json). Preliminary coarse-timer runs are excluded.

## Workload and method

The existing sprite fixture uses four fixed-function blend modes in runs of 64, following the repository's mixed-blend rationale. Sprites are static, 8x8, share one texture, and are scattered within a 1280x720 view. This is a representative state-churn mechanism, not a captured game or an exact reproduction of the published mixed-blend benchmark (which has different texture and nesting settings). No per-sprite forced flushes. Observed batches, rather than node counts, determine eligibility.

The probe requires at least 100 batches, actual retained replay, and at least 0.1 ms instrumented replay time before enabling the candidate. Instrumented replay times include two timer calls per batch and are diagnostic only; the A/B measurement removes that wrapper. COOP/COEP enable finer browser timers. After 150 warmup frames and 100 diagnostic frames, each arm receives 30 settling frames and 100 measured frames per block. Eight blocks alternate AB and BA. GPU completion is awaited after every frame, outside the timed region. The measurement covers `resetStats`, clear, root rendering, and flush; it is CPU submission time, not GPU time or display FPS. This serial acquisition does not represent a saturated pipelined game loop.

The disposable Vite transform changes only the final draw-encoding section of `WebGpuSpriteRenderer.replayRetainedBatch`. One native bundle per retained payload replaces pipeline/bind-group/vertex/index/draw commands with `executeBundles`. Existing replay, resource checks, UBO upkeep, pass handling, and stats remain live. Both arms use the same transformed source and toggle a flag. Core files on disk are unchanged. The fixed static workload permits a WeakMap payload cache without implementing general invalidation.

## Results

Times are milliseconds. Baseline and bundle columns are medians of the eight block medians (upper middle for an even count). Paired saving is the median of the eight within-block differences, so it need not equal the difference between the two aggregate columns.

| Fresh process | Nodes | Batches | Baseline | Native bundle | Paired saving | Positive pairs |
| ------------- | -----: | ------: | -------: | ------------: | ------------: | -------------: |
| 1 | 1,000 | 13 | 0.060 | Not eligible | - | - |
| 2 | 1,000 | 13 | 0.060 | Not eligible | - | - |
| 3 | 1,000 | 13 | 0.060 | Not eligible | - | - |
| 1 | 10,000 | 157 | 0.200 | 0.155 | 0.030 | 7/8 |
| 2 | 10,000 | 157 | 0.175 | 0.140 | 0.040 | 8/8 |
| 3 | 10,000 | 157 | 0.210 | 0.155 | 0.045 | 8/8 |

Instrumented replay for the 10,000-node scene: 0.204 / 0.202 / 0.358 ms per frame. These are not additive breakdowns of the uninstrumented frame times. Native bundles reduce the measured CPU frame region by approximately 20-26% when comparing aggregate arm medians, but the paired absolute saving is only 30-45 microseconds. This is not a 20-26% whole-game speedup.

All three candidate runs built exactly 157 native bundles across the measured blocks, with no subsequent rebuild. WebGPU validation scopes and uncaptured-error handlers reported no errors. The canvas comparisons in the historical timing records copied before explicitly awaiting GPU completion; those fields are retained as acquired, but are not the authoritative pixel-parity evidence.

A separate three-process correctness acquisition, [parity-results.json](parity-results.json), uses a persistent RenderTexture, awaits submitted GPU work, and reads via the backend's staging-buffer/mapAsync path. All three runs have 157 batches, exactly 157 parity bundle builds, zero byte differences for both initial bundle execution and subsequent cached replay, and no WebGPU validation errors. Each baseline contains 712,545 nonzero RGB bytes; a clear-only negative control differs in 800,955 bytes, rejecting empty or stale readback. The target switch explicitly resets the disposable cache, so this proves static offscreen sprite parity, not general target-format invalidation or canvas presentation. No performance samples were reacquired or replaced.

## Phase-1 limits and historical recommendation

The prototype is throwaway and must not ship unchanged. It does not handle texture/buffer/pipeline identity changes, device loss, changing render formats, stencil, custom materials, mixed renderers, or general pass-state invalidation. It does not combine multiple batches into a single bundle, measure rebuild amortization, measure GPU cost, or validate another backend/device. Bundles can invalidate pass bindings; the tested sprite path explicitly rebinds its complete draw state, but mixed-renderer behavior needs separate proof.

Do not drop native bundles as inherently useless, and do not treat the historical 4-16 batch argument as current evidence. Defer production integration given the small absolute saving and missing lifecycle coverage. Reopen only when a real CPU-bound consumer produces many retained batches and profiling identifies retained replay as a material contributor to its CPU frame budget. Require a meaningful absolute CPU saving against that consumer's budget, not just a relative percentage, then test a second hardware class and rebuild/invalidation amortization. Resource/pass invalidation and mixed-renderer parity remain acceptance requirements. Combining compatible batches into one group bundle is a future experiment under this same trigger, not v0.19 scope.

## Reproduce and validation

The [archived reproducer](reproduce.md) contains the complete disposable runner, browser probe, and summary script. Run it against the measured source revision above with the existing development dependencies installed. It does not activate native bundles in the runtime or add a maintained benchmark lane. Raw timing arrays are retained so the summaries can be checked independently.

Three independent browser acquisitions exited successfully, with pixel parity and WebGPU validation as reported above. This is experimental evidence rather than a published cross-library reference profile. General lifecycle and performance regression suites are outside this static probe.
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
{
"revision": "07d5ea5496381a24b82d491edf763578448574c5",
"date": "2026-09-27T03:38:23.521Z",
"results": [
{
"run": 0,
"browser": "153.0.8010.12",
"count": 10000,
"batches": 157,
"replayCallsPerFrame": 157,
"instrumentedReplayMs": 0.2616500008106232,
"isolated": true,
"adapter": {
"vendor": "nvidia",
"architecture": "blackwell",
"device": "",
"description": ""
},
"samples": [],
"bundleBuilds": 0,
"parityBatches": 157,
"parityBundleBuilds": 157,
"parityReadback": "RenderTexture/readPixels",
"pixelByteDifferences": 0,
"replayPixelByteDifferences": 0,
"nonzeroRgbBytes": 712545,
"clearControlDifferences": 800955,
"errors": []
},
{
"run": 1,
"browser": "153.0.8010.12",
"count": 10000,
"batches": 157,
"replayCallsPerFrame": 157,
"instrumentedReplayMs": 0.2639999997615814,
"isolated": true,
"adapter": {
"vendor": "nvidia",
"architecture": "blackwell",
"device": "",
"description": ""
},
"samples": [],
"bundleBuilds": 0,
"parityBatches": 157,
"parityBundleBuilds": 157,
"parityReadback": "RenderTexture/readPixels",
"pixelByteDifferences": 0,
"replayPixelByteDifferences": 0,
"nonzeroRgbBytes": 712545,
"clearControlDifferences": 800955,
"errors": []
},
{
"run": 2,
"browser": "153.0.8010.12",
"count": 10000,
"batches": 157,
"replayCallsPerFrame": 157,
"instrumentedReplayMs": 0.264350001513958,
"isolated": true,
"adapter": {
"vendor": "nvidia",
"architecture": "blackwell",
"device": "",
"description": ""
},
"samples": [],
"bundleBuilds": 0,
"parityBatches": 157,
"parityBundleBuilds": 157,
"parityReadback": "RenderTexture/readPixels",
"pixelByteDifferences": 0,
"replayPixelByteDifferences": 0,
"nonzeroRgbBytes": 712545,
"clearControlDifferences": 800955,
"errors": []
}
]
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
# Phase 2: scale, rebuild cost and churn

Decision: production candidate accepted for adaptive promotion of stable retained content. These measurements justify implementation work, not unconditional native replay or a universal profitability threshold. Lifecycle safety, promotion behavior and integrated overhead remain implementation gates. The proposed production path uses normal replay while observing stability, limited native construction after promotion, and immediate fallback after invalidation.

Source commit: `07d5ea5496381a24b82d491edf763578448574c5`, tree `c57d4ada609cb74bf0a227ecd4e2b72cb6b9e94f`. Windows, RTX 5070 Ti (OS adapter query), Chromium 153.0.8010.12; WebGPU identifies NVIDIA / Blackwell. No second physical GPU or iGPU is available on this machine. This is local diagnostic evidence, not a controlled cross-library reference profile.

![Scale and churn curves](scaling.svg)

The curves show medians of three independent process results, with min-max ranges rather than confidence intervals. The 13-batch break-even point is omitted because one process has no positive static saving. The forced 2,000-batch workload is intentionally separate from the ordinary scale curve.

## Scale and construction

Each cell uses a fresh browser process; three processes per cell. Sprites are 8x8. The ordinary scene retains the existing optimizer and cycles four blend modes in 64-sprite plateaus. The stress scene alternates Normal/Multiply and sets preserveDrawOrder to force exactly 2,000 batches. Its first attempted unordered construction collapsed to two batches, correctly failing the batch-count assertion; those failed attempts are excluded.

Times below are milliseconds. Savings are within-block differences of frame medians; ranges span the three process-level medians. Construction is the median paired cold-native minus warm-native frame time, with all group data already retained. Break-even is that incremental native reconstruction cost divided by static replay saving, calculated separately for each process. It is an estimate of needed reuse AFTER construction; waiting 30 frames before constructing a bundle does not pay for its future construction.

| Scene | Observed batches | Static saving | Full native reconstruction | Estimated static break-even |
| ----------- | ---------------: | --------------: | -------------------------: | --------------------------: |
| 1k | 13 | -0.010 to 0.010 | 0.030 to 0.085 | No stable positive result |
| 10k | 157 | 0.045 to 0.065 | 0.335 to 0.385 | 5.923 to 8.444 frames |
| 50k | 782 | 0.200 to 0.360 | 1.695 to 1.835 | 4.708 to 8.800 frames |
| 100k | 1563 | 0.170 to 0.345 | 3.205 to 3.325 | 9.435 to 18.853 frames |
| stress-2000 | 2000 | 0.285 to 0.340 | 4.055 to 5.630 | 12.618 to 19.754 frames |

## Synthetic native-cache rebuilds

The rotating eviction schedule removes 0%, 1%, 5% or 10% of native payload cache entries per frame, accumulating fractional entries rather than rounding every frame upward. This retains all engine recording data and isolates native reconstruction. Both arms run the same eviction bookkeeping; it is not a measurement of real engine resource invalidation. Lifetime in this deterministic schedule is approximately 1/rate, not an observed lifetime distribution of real content.

| Scene | 0% saving | 1% saving | 5% saving | 10% saving |
| ----------- | --------------: | --------------: | -------------: | ---------------: |
| 1k | -0.010 to 0.010 | 0.000 to 0.010 | 0.000 to 0.005 | -0.005 to 0.000 |
| 10k | 0.045 to 0.065 | 0.025 to 0.040 | 0.005 to 0.015 | 0.000 to 0.000 |
| 50k | 0.200 to 0.360 | 0.145 to 0.240 | 0.060 to 0.145 | -0.060 to 0.030 |
| 100k | 0.170 to 0.345 | -0.080 to 0.345 | 0.015 to 0.165 | -0.205 to -0.060 |
| stress-2000 | 0.285 to 0.340 | 0.295 to 0.335 | 0.075 to 0.405 | -0.100 to -0.070 |

The 100k scene loses at 10% eviction in all three processes. The 1% 100k range includes a negative run, and increasing from 50k to 100k does not double the benefit. These observations do not identify the cause of nonlinearity; browser/driver behavior, scheduling, garbage collection and other effects have not been separated. No fixed batch threshold or age guarantees profit.

## Dynamic and mixed cases

Camera moves the view by one pixel on alternating frames. Texture redraws the same canvas and calls updateSource each frame, preserving texture identity. Groups uses 100 actual RetainedContainers of 100 sprites each and calls invalidateContent on rotating 0/1/5/10% subsets every frame; unlike cache eviction, this exercises real recording replacement. Mixed has 40 ordered retained groups, each Sprite, Text, Tilemap, NineSlice geometry, Sprite: 200 retained batches from four renderer classes. All tested native renderers use the same final-encoding substitution and keep their existing resource/UBO upkeep live.

| Case | Batches | Static / 0% saving | 10% saving |
| ------- | ------: | -----------------: | ---------------: |
| camera | 157 | 0.040 to 0.055 | -0.035 to 0.005 |
| texture | 157 | -0.005 to 0.045 | -0.025 to -0.020 |
| groups | 100 | 0.030 to 0.035 | 0.025 to 0.165 |
| mixed | 200 | 0.370 to 0.465 | 0.250 to 0.320 |

Actual group invalidation drives the whole CPU frame into roughly 5-9 ms in this fixture; the small paired native differences are not evidence that native bundles solve the engine recording cost. The complete baseline/candidate timings and actual native build/group invalidation counts remain in the raw records. Those numbers must not be substituted for the synthetic-cache-only curve.

All 27 process/cell acquisitions completed without WebGPU validation errors. All final baseline/native/repeated-native RenderTexture readbacks match byte-for-byte; clear-only negative controls differ. Dynamic cases additionally compare baseline/native pixels at each churn rate. Mixed renderer counts verify that Sprite, Text, TileChunk and ScalableSprite all participated. These are bounded fixture checks, not general lifecycle coverage.

## Acquisition and remaining gates

120 warmup frames, one structural replay probe, 30 native warmup frames, 12 alternating cold/warm construction pairs, then six AB/BA blocks per eviction rate. Each arm gets 20 settling frames and 60 timed frames. Timed work includes scene updates, eviction or actual group invalidation, stats reset, clear, rendering and flush. GPU completion is awaited outside each timed region. Acquisition is serial, not a saturated pipelined game loop. Direct per-encoder build timers run in a separate diagnostic frame and are not used for break-even estimates. Summaries use upper-middle medians for even sample counts; sub-nanosecond floating-point residues in paired differences are normalized to zero.

No second hardware class, texture/view recreation, custom material uniforms, stencil/mask transitions, format-changing targets or device-generation changes are covered by this matrix. The stable offscreen target is not proof of target invalidation. Those remain correctness gates for production. Promotion age, workload floor, build-budget behavior and burst invalidation have not been measured by this eager per-batch prototype. No automatic eligibility policy is shipped here.

Use [reproduce.md](reproduce.md) for the complete disposable source snapshots. Raw files are `*-results.json`; derived process summaries are `*-summary.json`. The phase-1 timing acquisition remains unchanged. These new results refine the decision to pursue conditional integration while preserving normal retained replay as the correctness fallback.
Loading
Loading