Load-balance, active-box windowing, block-structured AMR - #1628
Open
sbryngelson wants to merge 724 commits into
Open
Load-balance, active-box windowing, block-structured AMR#1628sbryngelson wants to merge 724 commits into
sbryngelson wants to merge 724 commits into
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR introduces an opt-in (“default-off”) family of performance/diagnostic features (load-weight and SFC partition diagnostics, weighted init-time decomposition, rank timing), plus major simulation capabilities (active-box RHS windowing and block-structured AMR) and corresponding post-processing support and documentation/validation updates.
Changes:
- Adds new runtime parameters and toolchain metadata/validation hooks for the experimental performance/AMR feature family.
- Extends the simulation code with new modules for active-box restriction, load-weight diagnostics, SFC partition reporting, rank timing, and AMR integration points (including restart/output plumbing).
- Updates post_process to read/write AMR fine-block overlays and adds/updates golden metadata plus documentation/indexing.
Reviewed changes
Copilot reviewed 82 out of 94 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| toolchain/mfc/params/descriptions.py | Adds user-facing descriptions for new experimental/performance parameters. |
| toolchain/mfc/params/definitions.py | Registers new parameters (AMR, hybrid sensors, load-balance diagnostics) and target applicability. |
| toolchain/mfc/lint_docs.py | Treats new validator checks as non-physics doc checks. |
| tests/F980C769/golden-metadata.txt | Adds golden metadata for a new/updated test. |
| tests/ECABA006/golden-metadata.txt | Adds golden metadata for active-box test coverage. |
| tests/DD4CD8F3/golden-metadata.txt | Adds golden metadata for a new/updated test. |
| tests/CC4213FD/golden-metadata.txt | Adds golden metadata for a new/updated test. |
| tests/BD21A5C0/golden-metadata.txt | Adds golden metadata for a new/updated test. |
| tests/BCBA6E74/golden-metadata.txt | Adds golden metadata for a new/updated test. |
| tests/ACE05393/golden-metadata.txt | Adds golden metadata for a new/updated test. |
| tests/987D9025/golden-metadata.txt | Adds golden metadata for a new/updated test. |
| tests/852CCB81/golden-metadata.txt | Adds golden metadata for AMR-related golden tests. |
| tests/65C375B4/golden-metadata.txt | Adds golden metadata for a new/updated test. |
| tests/4DADE04B/golden-metadata.txt | Adds golden metadata for a new/updated test. |
| tests/454C565F/golden-metadata.txt | Adds golden metadata for a new/updated test. |
| tests/3A474BEE/golden-metadata.txt | Adds golden metadata for a new/updated test. |
| tests/2FC423D3/golden-metadata.txt | Adds golden metadata for a new/updated test. |
| tests/13945217/golden-metadata.txt | Adds golden metadata for moving-IB under AMR test coverage. |
| src/simulation/m_viscous.fpp | Clamps FD coefficient indexing to avoid ghost-region coefficient OOB in IB drag gradient evaluation. |
| src/simulation/m_time_steppers.fpp | Integrates active-box bounds into RK update loops and interleaves AMR fine-stage/subcycle operations. |
| src/simulation/m_start_up.fpp | Wires up new modules (rank timing, active-box, load-weight, SFC partition, AMR) into init/timestep/finalize and restart I/O. |
| src/simulation/m_sfc_partition.fpp | Adds analysis-only SFC tiling + weighted partition prediction and reporting. |
| src/simulation/m_rank_timing.fpp | Adds per-rank wall-time imbalance measurement helpers and reporting. |
| src/simulation/m_load_weight.fpp | Adds per-cell load-weight field construction and rank-level imbalance reporting. |
| src/simulation/m_hypoelastic.fpp | Refactors FD coefficient setup into a callable update routine (supporting AMR grid swaps). |
| src/simulation/m_global_parameters.fpp | Adds AMR working-state mirrors and slot selection helper plus defaults for new parameters. |
| src/simulation/m_data_output.fpp | Adds output/report hooks for load-weight, SFC partition, and rank-time diagnostics. |
| src/simulation/m_checker.fpp | Adds input validation/prohibits for active-box, hybrid sensors, load-balance, and AMR configurations. |
| src/simulation/m_active_box.fpp | Adds active-box initialization/growth and debug envelope checking. |
| src/simulation/m_acoustic_src.fpp | Adds AMR-aware handling of acoustic source support (bounding boxes and overlap abort). |
| src/post_process/m_start_up.fpp | Calls AMR fine-data reader and AMR overlay writer when amr is enabled. |
| src/post_process/m_global_parameters.fpp | Adds default-off amr flag for post_process overlay behavior. |
| src/post_process/m_data_output.fpp | Implements AMR fine-block overlay mesh/variables output (Silo/binary) and multimesh registration. |
| src/common/m_phase_change.fpp | Exposes per-cell Newton iteration count and threads it through relaxation to support load-weighting. |
| src/common/m_global_parameters_common.fpp | Adjusts start_idx lifecycle/allocation and makes load_weight_wrt visible to GPU macros. |
| src/common/m_derived_types.fpp | Introduces a simple t_box type used by new partitioning infrastructure. |
| src/common/m_box.fpp | Adds box/partition arithmetic helpers (equal/weighted splits, box-from-splits). |
| src/common/m_boundary_common.fpp | Skips BC buffer population during AMR fine advance to rely on coarse-driven ghost fill. |
| docs/module_categories.json | Registers new modules under documentation categories. |
| docs/documentation/readme.md | Adds AMR section link to the documentation index. |
| .typos.toml | Adds project-specific abbreviations to the spelling allowlist. |
| D = ((gs_min(lp) - 1.0_wp)*cvs(lp))/((gs_min(vp) - 1.0_wp)*cvs(vp)) | ||
|
|
||
| #ifdef MFC_SIMULATION | ||
| if (relax .and. load_weight_wrt) then |
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #1628 +/- ##
==========================================
+ Coverage 61.67% 62.39% +0.72%
==========================================
Files 84 96 +12
Lines 21619 29068 +7449
Branches 3196 4778 +1582
==========================================
+ Hits 13334 18138 +4804
- Misses 6093 7753 +1660
- Partials 2192 3177 +985 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Member
Author
|
Upstream latent gap found during the MHD+AMR investigation (independent of this PR): Also for the record: MHD+AMR was attempted and re-gated on measured evidence rather than assumption — the coarse/fine seam is a continuous O(1) div(B) source that cleaning spreads but cannot remove (details in the amr.md support matrix row and commit ac203b1). |
sbryngelson
changed the title
Member
Author
|
2 tasks
…d by device copies
…on advisories amr_max_blocks defaulted to 4, capping the GLOBAL box count at four: any real refinement bound immediately and silently truncated the refined region. It sizes replicated metadata only (slots are allocated lazily for owned blocks), so a large pool costs ~11 kB/box/rank. Default is now 1024; case.md corrected, since it documented the parameter as a slot count rather than metadata. Adds four measured advisories at init (rank 0, warn-only - every setting is legal and sometimes correct): amr_regrid_int=0 is static AMR and never adapts; amr_max_grid_size=0 derives a cap from the decomposition that SHRINKS as ranks are added, making the box set rank-dependent (pinning it measured 3.0x faster in 3D); lock-step steps the coarse level amr_ref_ratio**level times more often than its CFL needs (subcycling measured 1.55x per unit physical time); and amr_regrid_int<4 regrids more often than the per-cell tag sweep justifies (interval 8 measured 1.39x). Also reports per-slot memory DEMAND at init rather than guessing supply - there is no portable way to query free device memory across four compilers and three offload backends, so it prints what a slot costs and lets the user compare. Gated on the full suite: 706 passed, 0 failed. precheck clean.
…rebalancing does not pay s_l0_rebalance computed the post-cut load gap and then migrated without ever comparing it to the pre-cut gap. Because the SFC re-cut is restricted to CONTIGUOUS Morton ranges, its finest correction is one whole tile - 25% of a rank's load at 4 tiles/rank - which exceeds the 5% deadband it is chasing, so it can only overshoot. Measured: 5.738E-02 -> 3.898E-01, a 6.8x WORSE partition that never recovered. Now evaluated into a temporary and committed only if the gap does not worsen. Rejects a strict worsening rather than accepting only a strict improvement: with near-uniform tile costs the Morton and cartesian partitions have EQUAL gaps, and rejecting those would silently remove the migration that golden A0968B15 exists to cover - it would still pass, since migration is bit-neutral, so the coverage loss would be invisible. amr_owner_cut moves with newo or not at all: f_amr_owner resolves tile ownership against it, so refreshing the cut without migrating would leave it disagreeing with amr_block_owner. The rebalance print now includes the deadband. Without it, 'deadband skipped the re-cut' and 'the re-cut ran and was rejected' emit byte-identical output, and the guard cannot be shown to execute. Verified by forcing the deadband to zero: 27 evaluations (9 windows x 3 reps), re-cut reached every time (deadband 0.000E+00 in-log), rejected every time, no gap ever worsened. Docs record OPEN 1's answer. On a purpose-built imbalanced case (blob IC, refinement concentrated in one rank's half, fine_work 2097152 vs 206016) L0 tiling costs 35% and rebalancing recovers 0.4% of it, inside run-to-run spread, reproduced across three independent jobs. OPEN 2 is demoted as a consequence. Scope limit recorded: 2D, single refinement level, hydro with uniform per-cell coarse cost. Also reconciles amr_block_batching.md: two headlines still named the packed super-grid as the live lever after it was disproved, and the cost law is confirmed with AMR active (time tracks TOTAL block advances, 3% at the largest point). 12/12 L0/AMR goldens pass. precheck clean.
…ting it is blocked Attributing 337,431 device copies to 16,698 dispatches (rocprofv3, temporal attribution): kernels taking fields as DUMMY ARGUMENTS pay 33-54 argument maps per dispatch, while kernels reading module-level GPU_DECLARE'd state (m_amr_registers freg/creg) pay EXACTLY ZERO. Copy count matches the deep-member count (3*sys_size + q_T_sf + bc_type = 20 at sys_size=5; measured 20.2 per dispatch). Total copy time 2.0 s against ~1.5 s of kernel time. A 4-translation-unit reproducer isolates the penalty at 4.3-4.7x per region. This kills two plausible plans: flattening the scalar_field interface (3,840 sites across common/) buys nothing, since flat-array dummies measure the same as deep-allocatable ones; and batching attacks dispatch COUNT while the tax is per-dispatch argument mapping. Converting s_amr_fine_rk_update to module-level views was attempted and reverted at 74/76 goldens. A control experiment isolates the cause: keeping the attach calls but reverting the kernel to read its dummy arguments fails byte-identically, so the kernel is innocent and the ATTACH ITSELF corrupts the slot's device data. Neither available clause is a correct attach on this backend - map(always,to:) copies the stale host array over live device state, map(always,alloc:) forces a fresh uninitialised allocation. A device-side probe comparing view against dummy reads zero difference because both resolve to the same WRONG buffer. A 6-phase standalone reproducer (attach-only, attach+detach, slot recycling, varying extents, realloc-at-new-extent) passes every phase, so this is an integration property rather than a property of the technique. No source change: GPU_ENTER_DATA(attach=) has 8 existing users in m_rhs.fpp/m_igr.fpp that depend on its current expansion, so it cannot be changed without a validated replacement primitive. Records the next step - establish an attach that neither copies nor reallocates on present data, and validate against golden 00EB793A specifically. Docs only; src unchanged. precheck clean.
…ule scope The attach hunt was solving the wrong problem. The per-dispatch tax is not that pointers are slow, it is that fields arrive as DUMMY ARGUMENTS, so every target region re-maps their deep members. Nothing needs aliasing, attaching, or flattening - the fields need to BE module state. m_rhs already declares ~12 module-scope GPU_DECLARE'd derived-type arrays whose kernels measure zero maps per dispatch; only q_cons_vf/q_prim_vf/rhs_vf arrive as dummies, and they are exactly the taxed ones. Promoting a dummy to a module variable of the SAME NAME leaves every loop body untouched - only declarations and call sites change. That is what makes it tractable and what makes it safe under a bit-identical requirement, in contrast to the flat-store alternative which threads a slot index through every reference. The monolithic path then pays nothing (the module arrays are its arrays); the AMR path copies a block's fields in before the advance and rhs back after, device-to-device: ~1.3 us against the measured ~2.9 ms of argument mapping per s_compute_rhs invocation. Re-entrancy verified, the one risk that could have killed it: s_amr_advance_children is recursive, but the recursion sits OUTSIDE the stage loop - every level-clev block completes all three stages before the routine recurses into clev+1 - and the L0/fine advance loops are serial over blocks. No nesting, so no working-set stack is needed. Records four de-risking MWEs to be built as ONE matrix per the lesson that a 6-phase matrix compiled in ~20 s eliminated four hypotheses at once and predicted an MFC failure, three golden-gated phases, and the conditions that would falsify the plan. Docs only; src unchanged. precheck clean.
…5, timing not measurable today Stress matrix in amr-bench/attach/promote/: promotion through a 3-deep cross-TU call chain over a sub-range, poisoned halo verified untouched, 25 advances with host/device deliberately diverged plus present-table pressure, 48 blocks of varying extents through one shared working set, and mid-run reallocation. All five pass. Mechanism detail for implementation: a module-scope deep type needs BOTH the parent descriptor and its members mapped - map(to:) on the parent alone leaves %sf null and the first kernel faults at nil. With promotion that setup runs once at init and the working set is never re-pointed, which is why this design sidesteps the attach problem that killed the view-based one. No speedup is claimed: the node showed a ~134 us per-region launch floor uniform across all 8 GCDs with the GPUs idle, versus ~20 us for the same construct earlier the same day, so a ~53 us mapping delta is inside the floor. MFC's own profile implies ~2-3x per region; the earlier 4.4x is an upper bound pending an idle node.
… on this compiler The premise was an attribution artifact. 'Dummy kernels pay 33-54 maps, module-state kernels pay 0' came from temporal attribution (correlation ids do not match across the two traces). Copies cluster: 11.5% of inter-dispatch intervals hold zero copies and one holds 2893, so a kernel dispatched in a burst reads as 0 regardless of need. The median of 20 per interval matches the deep-member count, so the ~20 maps/dispatch average stands - the per-kernel split does not. Every mechanism then measured directly by copy count, one variant per process under rocprofv3, needing no attribution and immune to the launch-time floor: deep-type dummies 9.4, flat-array dummies 9.4, pointer dummies 9.1, defaultmap(present:aggregate) 9.1, map(present,alloc:) 9.1, declare mapper 39.1 (4x worse), module state in every form including the exact freg pattern 32.8 (3.5x worse). Per-region map traffic is irreducible by interface or clause on this compiler, and module scope makes it worse. That retires promotion, interface flattening, declare mapper, and the view/attach design together. What survives: the tax is real (2.0 s of copies against ~1.5 s of kernels), but it can only be attacked by fewer block advances (amr_max_grid_size, ~20x, already shipped) or fewer regions per advance (kernel fusion, untried). Docs only; src unchanged. precheck clean.
BASE_CFG hardcodes precision = 2 (double post_process output), which case_validator rightly prohibits on a --single build. Every post_process case on the single-precision CI lane has been failing this validation invisibly -- the suite discarded post's exit code until 68bcaf1 -- and surfaced as 'Case validation failed for post_process' across the lane once the check landed. get_post_process_mods now requests precision = 1 when the build is single. Post output files are not golden-compared, and the lane's post step never succeeded before, so no goldens move.
…it-identical The min-separation merge restarted an all-pairs scan after every fusion and removed the absorbed box by an O(n) shift; at np=128 the gathered list is ~12,400 leaves collapsing through ~11,300 fusions per call, and rg:clus grows to 75% of wall at np=512. Survivors now live on a doubly-linked list (O(1) unlink) and each pass finds its fusion through a uniform bin grid over the LIVE boxes' bounding box, width max(ext_max + thr, clamped so nb_tot stays O(n)) -- sound because a tooclose pair is within ext_max + thr - 1 per dimension and enlarging cellw only widens the candidate set; bit-identical because for each i in list order the minimum surviving candidate index equals the first tooclose j the linear walk meets. Also fixes: the nacc == 0 walk read an unwritten nxt(1); the [amr-merge] fusion counter was declared and printed but never incremented (its 'zero fusions' reading was an artifact); and this rebuild restores the truncation warning, amr_n_shaped accumulation and grideff 'shaped' field that the first version of this commit reverted through a stale-base file copy (caught in review). Measured on mi2104x pinned-binary A/B: rg:clus 138.6->83.8 (np32), 1074.9->335.7 (np64), 8462.1->1076.1 ms/call (np128).
s_amr_st_reserve chose device-native vs host-staged growth by COLUMN COUNT (32), but a column is ~33 MB at the default 64^3 block cap and ~1.8 GB on the 57.3 GiB run that motivated the guard -- the same count means wildly different bytes. The store-capacity ratchet pushes production runs to 43-81 columns, sending every later growth on the full-store host PCIe round trip (measured 4.4 s/regrid vs 0.55 under the guard). The guard now budgets the transient itself: stage on-device while the extra copy (oldcap * column bytes, which is exactly the store size regardless of factorization) stays under 4 GiB, so near-limit stores of ANY column shape still take the OOM-safe host path. Review-noted exposure on 16 GB cards is recorded at the constant; the registers site keeps its count guard (per-array transient, two orders smaller) with its stale cross-reference corrected.
The previous fix (4d5f6ee) patched get_post_process_mods, but only coverage_build.py calls that function -- the test suite embeds POST_PROCESS_OUTPUT_PARAMS directly into the generated case program, so the single-precision CI lane kept failing every post_process validation unchanged. The guard now lives in the generated program itself, keyed on ARGS['mfc']['single'], which run/input.py already serializes. Validated end-to-end by regenerating a case through the harness and executing it both ways: single=true yields precision 1, single=false yields 2. The earlier fix was committed validated only by construction; its local check had failed on environment and was not repeated -- the same never-observed-effect class as the dead counters.
…(47) glob_bytes counted 18 metadata ints per block (72 B) and ignored the amr_slots struct array, whose descriptors dominate at ~1 kB/slot -- a 10-20x under-report that silently propped up the metadata-distribution-deferred decision and would have steered the production-size rung. Ledger entry 47 records the landing, the toolchain-lock incident, and the forensic corrections.
… verdicts The [amr-mig] counters counted only blocks rank 0 itself sends and printed them unreduced -- the zero-cost audit read a frozen 141 MB startup transient at every P off what looked like machine-wide migration. Now SUM-reduced across ranks (ml_g, mirroring the [amr-halo] reduce just above; rank0_bytes kept for cross-run continuity). Ledger 48 records: the rhs growth above np128 is ranks-per-node density (memory-bandwidth contention), not P, pending the pre-registered np1024 confirmation; the dirty-box payoff is 4/np independent of bin occupancy (127x at np512); ladder migration is a frozen startup transient so keep-tol requires a moving-feature deck; escaped is 0 through np512; and the memory probe measured 201 MB/rank replicated metadata plus a ~3.3 GB/rank fixed footprint that OOMs dense packings.
After a fusion only the surviving box changed, so instead of restarting the whole pass the merge now (a) re-tests earlier survivors against the grown box (minimum index first, which is provably what the restart would find), else (b) re-tests later survivors, else resumes at the survivor's live successor. Bins are built once per cellw epoch and maintained incrementally (absorbed boxes unlinked, the survivor re-filed when its lo crosses a bin; extent growth doubles cellw and rebuilds, amortized log). Sequence identity was proven by a differential control (amr-bench/tools/merge_ctl.f90) against the retained restart implementation: identical fusion sequences on directed + random suites spanning 864 back-fusion chains; mutation skip-(a) diverges on 57 cases proving the control can fail; stale-resume is proven benign by containment + (b)-exhaustion; the survivor refile is kept as O(1) insurance with the Morton-order masking argument documented in the control. The [amr-merge] mean_outer_pos field is renamed mean_visit because its semantics change with the restart's death (per-pass scan depth -> monotone visit index) -- old cbar formulas must not be applied to new logs. Expected from the counter laws (F ~ 90 np, mop = np/2): pair tests fall ~mop/2-fold, two orders of magnitude at np512, growing with P. Gates: CPU 9/9 bit-identical with counters exactly matching the restart implementation (pair_tests 496/fusions 31 and 171/18 on the churn control); GPU 66/66.
… NaN) ab11150 put the same scratch arrays in both private() and map(alloc:) on one combined OMP construct in the HLLC and WENO kernels -- non-conforming OpenMP that its own commit message flagged as unverified on the other CI compilers. nvfortran resolves the conflict map-wins: cross-compiled PTX shows every scratch-array store with no thread index in the address chain, i.e. all threads share one vel_L etc., and the resulting races produce the deterministic first-step 'NaN(s) in timestep output' that has held the Phoenix NVHPC gpu-omp lane red on basic 1D tests since Aug 13 (lane green Aug 2; master unaffected -- it never carried the clause). amdflang resolves it private-wins, which is why the local gate could never see it, and an explicit conforming split (map on an enclosing target data region) breaks amdflang instead -- no conforming formulation reproduces its fast path. Measured tonight, interleaved 3x3 on the production case: the hack is worth 6.3 +/- 3% on amdflang (not the 26.4% recorded at introduction; the sequential-order measurement that suggested 12.6% was drift-confounded). Decision: pay the ~6% for conforming code on all four compilers; block-local scratch (option B) can recover it later as pure performance work.
…gesort The Morton sort ran an insertion sort over R concatenated per-rank runs -- O(n^2) in inversions, measured growing 5.1x over the first rank doubling (2.2x the next), of the same order as the merge residual beside it. Now a stable bottom-up mergesort on an index permutation with the payload gathered once at the end. Stability at key ties (fall back to acceptance order) is load-bearing for B1: proven by a differential control (amr-bench/tools/sort_ctl.f90, verdict saved in logs/sort_ctl_final.log -- identical orders on tie-heavy suites, an unstable mutation diverges on all 48 cases), and ties cannot occur in production anyway since accepted boxes have distinct lo corners. Sort-then-gather equals sort-with-payload because the stable order is unique (independently audited, including the gather-vs-scatter direction). Block-locals renamed msw/mslo/msmid/mshi to avoid shadowing the host's BR-stack arrays. Gates: CPU 9/9 with all 20 output files byte-identical to the pre-change binary; GPU gate follows.
Per-site XOR folds of mix(unordered pair id, per-channel sequence, block/dim/face key), BXOR-reduced at finalize and compared BY FAMILY (sends and receives of the same traffic live under paired audit sites -- a per-site compare mismatches structurally, found by exactly that false positive). Under the FIFO order contract the sender's and receiver's triples coincide message-for-message so the family hashes cancel; any cross-rank ordering divergence misaligns key against sequence and the finalize check aborts with the family named. The mixing hash is load-bearing: a raw packed-field XOR is provably blind to pairwise transpositions -- the property control (amr-bench/tools/oracle_ctl.f90, verdict saved in logs/oracle_ctl_final.log) demonstrates raw-XOR blind on 50/50 transposition trials and the splitmix64 fold catching 50/50. Wired at the eight F5 wave sites (faces + freg, lo/hi distinguished in the key, loop-scope of every peer/block variable verified); other sites opt in incrementally via the new optional peer/key arguments. A seeded canary (MFC_XA_SEED=1 corrupts exactly one fold, latched -- an XOR accumulator can return to zero, so testing it would allow self-cancelling double fires) proves the wiring can fail: it aborts fatally with family 5 named. The per-(site,dir,peer) sequence counters are O(P) per rank -- accepted for M0, replaced by plan-derived sequences at M1. Gates: healthy churn np=2 silent; canary aborts; 9/9 subset.
…ecv-plan scan The stage-fill wave rebuilt its coarse-owner receive plan by scanning every block in the machine at every RK stage -- at 1e5 ranks x 75 boxes that is ~360 M predicate evaluations per rank per step to find ~75 survivors. The surviving set is O(local) (level + not-mine + interior-overlap are all regrid-invariant), so it is now cached in amr_l1r_blk, rebuilt lazily when amr_mesh_epoch moves (NOT the owner-only dirty flag, which regrid and restart never set). List order is ascending block id, identical to the scan it replaces, so the wave's message order is unchanged -- verified live by the M0 order oracle (silent on the converted wave; the seeded canary still aborts). Resurrected from the never-landed stash pilot, adjudicated against the W1 conversion recipe: this site carries no safety calls between loop head and filters (the s_amr_check_lag_clear trap lives in other routines). Gates: 9/9 goldens, oracle silent at np=2, reldebug bounds arm clean. First of the ten per-stage scan conversions -- the 8.5-12% O(P)-growing family behind the weak-scaling gap.
Three more per-stage global scans converted (4 of 10 done). The stage-fill gather scan is the recipe's documented trap site: s_amr_check_lag_clear ran for EVERY level-1 block above the owner filter, so it keeps a dedicated scan gated on bubbles_lagrange (coverage unchanged, cost only in that configuration) while the gather loop walks a NEW padded list -- the same epoch-keyed walk now builds both flavors: raw region vs my interior (amr_l1r, the batch-1 list) and region +/- amr_cpat_mar vs my coarse range (amr_l1p, this loop's exact predicate); the body keeps its own intersection and empty-cycle as belt-and-braces. The two owner-side scans (stage-fill send prep, reflux-faces owner side) walk amr_my_blk with the level filter kept (the list carries all owned levels) and the ownership test retained defensively. List orders are ascending block id, identical to the scans replaced -- wave message order unchanged, verified by the M0 oracle (silent at np=2). Gates: 9/9 goldens, oracle silent, reldebug arm zero violations.
reflux-recv rides amr_l1p (the participates predicate is provably inside the padded-list predicate); freg recv, parent-fill send, and restrict-parent recv share the new amr_fch_blk list (level>=2, my parent, foreign child); freg send walks amr_my_blk; the parent-fill consume loop gets its own amr_own_blk list because it tests the MULTI-owner amr_owns_all notion, which amr_my_blk would silently narrow. The bigger find: f_amr_parent_block is itself an O(global blocks) scan and these waves called it per block per RK stage, making them quadratic in the global block count -- it is now cached per mesh epoch in the same builder walk, alongside a children adjacency (CSR) that replaces the identical hidden scan inside s_amr_sibling_face_weights. The lag guard keeps its full-coverage scan, gated on bubbles_lagrange. Survivor sets and ascending order are preserved at every site (exact predicates kept in bodies), so MPI posting order is unchanged by construction. Independently audited: the audit added the orphan-parent query guard (an unguarded amr_child_ptr(-1) would have turned a broken mesh into silent weight corruption), removed a re-entrant list refresh from inside a loop over that list, and keyed the cache on (epoch, num_blocks). Gates: 69/69 + 14/14 GPU goldens (isolated worktree), multi-level np=2 order oracle silent with all families balanced and the F2 traffic proven per-stage by 10-vs-20-step differencing (20 vs 51 msgs, 3 per marginal step = one per RK stage), seeded canary aborts as required, reldebug clean.
Every message in the reflux-faces and freg waves (payloads and the debug identity headers, both directions) now carries its own tag: base + band*2^16 + gen*2^12 + seq, with the band space starting at the next 65536 boundary above every legacy tag and asserted against MPI_TAG_UB at init. gen (mod 16) bumps at wave entry on every rank -- both call sites are rank-unconditional, and the subcycle path skips them on all ranks together -- separating successive waves that share a band. seq is the message's position in the pair's canonically ordered transfer list (ascending block id, then dim, lo before hi), derived independently by each end from the same replicated metadata, so message matching no longer depends on posting order or MPI non-overtaking; the in-wave per-peer counters reset touched entries only (O(active peers), never O(P)). The audit's xa_seq array -- the order oracle's per-peer O(P) interim state, 368 B x P per rank -- is deleted: call sites pass the plan-derived seq and the oracle folds it directly. New seeded gate MFC_XA_SEED=2 shifts one plan seq on rank 0's send side, modeling a sender deriving a different plan order; the finalize oracle must abort. Gates: family counts byte-identical to the pre-M1 baseline (130/51/6/32/120/16 on the multi-level np=2 oracle deck), seeds 1 and 2 both abort, reldebug clean, 16/16 goldens at np=1/2/4 including wide-L2 and force-migrated.
…eX, kernels are the per-GPU gap
…W1 wall-neutral at this scale; rs:rfp claim retracted
…at 1.01-1.03x per doubling
CCE 19.0.0 cannot codegen the HLL kernel for benchmarks/viscous_weno5_sgb_acoustic: llc asserts in StackSlotColoring (LiveInterval.cpp:391, "empty range"). That case runs HLLC, so the kernel it dies on is one the case can never call. The Case Opt | Frontier (CCE) (gpu-acc) lane has never passed on this branch. Case optimization now bakes riemann_solver in and guards each solver body and its dispatch arm on it, so an unreachable solver is never emitted. riemann_solver = -1 means the case did not pin one, and every solver is compiled as before. Non-case-optimized builds are unchanged: all four solvers, all four dispatch arms. The alternative, -Oipa0 on m_riemann_solver_hll.fpp.f90, also cleared the crash but cost 14.4% on HLL cases (hypo_hll, 3 reps, gbpp 4: 1.2174 -> 1.3928 ns/gp/eq/rhs, non-overlapping ranges) and applied to every Cray GPU build. Verified on Frontier: the crashing case builds clean, and both it and hypo_hll pass check_case_optimization_output.py. The large diff is ffmt reindenting the guarded bodies (sbryngelson/ffmt#9); git diff -w is far smaller. Claude-Session: https://claude.ai/code/session_01XZjDp6ch827LVJRrvVb8hC
… the O(P) term; controls and reorder
…k's coarse extent (silent NaN since a108dd3)
Lines of Code
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
An opt-in, default-off family of performance features and the measurement infrastructure they rest on. With all flags at their defaults the only touched production path is
s_mpi_decompose_computational_domain, refactored through the newm_boxmodule (byte-identical; covered by the existing suite).m_box(partition arithmetic),m_load_weight/load_weight_wrt(per-cell load-weight field + imbalance metric),m_sfc_partition/sfc_partition_wrt(Morton-SFC predicted-imbalance diagnostic),m_load_balance/load_balance(weighted static decomposition at init; AMR-fine-work-aware),m_rank_timing/rank_time_wrt(per-rank compute-time diagnostic).m_active_box/active_box: restricts reconstruction/Riemann/RK windows to a light-cone-grown box around non-ambient flow; strict-subset golden-tested.hybrid_wenoandhybrid_riemann(+hybrid_weno_eps,hybrid_smooth_flux): linear-optimal weights / central-or-Rusanov flux in smooth cells, full WENO/HLLC at flagged discontinuities (Jameson sensor, stencil-dilated, per-level under AMR).m_amr+m_amr_registers: two-level 2:1 refined block hierarchy; conservative restriction and conservative-linear prolongation with physics-specific closures; per-stage flux registers with Berger–Colella refluxing; Berger–Rigoutsos multi-block dynamic regrid; optional dt/2 subcycling; multi-rank (single-owner blocks assigned by Morton-SFC work balancing at each regrid, with migration; blocks may span rank seams via P2P coarse↔fine gather/scatter; same-level seam halo; distributed registers); restart (both IO modes, regridded-layout persistence); AMR-aware post-processing (fine blocks visualizable as Silo overlay domains); GPU-resident fine level on both OpenACC and OpenMP offload.Full algorithm and user documentation:
docs/documentation/amr.md(support matrix enforced at runtime by the checker — unsupported combinations abort with named messages, never silently).AMR physics support matrix (abridged; authoritative table in amr.md)
Supported and golden-tested: single- and multi-fluid (5-eq,
mpp_lim) · 6-eq with per-block pressure relaxation · viscous (refluxed) · phase change (relax) · chemistry incl. species diffusion · Euler–Euler bubbles (polytropic/non-polytropic, mono/polydisperse, QBMM incl. non-polytropic with per-blockpb/mvside-state; dynamic regrid + subcycle) · acoustic sources (coarse-grid support with regrid exclusion) · immersed boundaries (multi-body, static or prescribed-motion, incl. dynamic regrid with body-containment expansion and per-substage guards) · 2D axisymmetric (per-block WENO-coefficient recompute) · stretched grids (exact parent-bisection ghost coordinates + per-swap coefficient recompute) · hybrid WENO/Riemann sensors (per-level) · Lagrangian bubbles (cloud excluded from blocks; two-way coupling on the coarse grid; regrid clips around the moving cloud) ·active_box(blocks contained in the growing window; agrees with plain AMR to ~1e-14) · IGR (restriction-only coupling: fine sigma solve seeded/Dirichlet-bounded by the coarse solve; documented truncation-order seam, exact free-stream) · 1D MHD/RMHD (div(B)=0 by construction in 1D; HLL and HLLD, incl. relativistic).Gated with named aborts (documented rationale): surface tension (seam force imbalance is structural — three fixes attempted and diagnosed in amr.md) · 2D/3D MHD (attempted and measured: the c/f seam is a continuous O(1) div(B) source GLM cleaning cannot remove — needs constrained-transport-class B prolongation/reflux) · hyperelasticity · 3D cylindrical (global azimuthal filter) · force-driven IB (
moving_ibm=2) · STL bodies · Riemann-extrapolation BCs (bc=-4) ·amr_subcycleunder IGR · stretched grids with Lagrangian/IB-regrid (uniform-spacing index conversions).Validation evidence
Known issues (all non-gating or in progress)
continue-on-error): an intermittent post-detected NaN on the two Lagrangian+AMR goldens. Exhaustively unreproducible off GitHub's runners — the exact failing stack (NVHPC 24.3 SDK,-tp=px -Kieee, HPC-X MPI, and the CI docker image itself under apptainer) passes elsewhere, as do native/zen2 builds; 24.5+ green. Documented at the golden definitions.Review guide
The commit history is arc-ordered (active-box → load-weight → SFC → weighted decomposition → rank timing → hybrid → m_box → AMR rungs → physics envelope → CI/GPU hardening); reviewing by arc is much easier than by file. The AMR arc builds stepwise: static hierarchy → restriction/prolongation → fine advance → refluxing → regrid → subcycling → multi-rank → GPU → each physics rung with its own validation. Commit messages carry the validation evidence for their change (measured defects, golden UUIDs, repro details for CI fixes).
All parameters ship default-off with
case_validatorentries, runtime checker gates, andcase.md/amr.mddocumentation.