Skip to content

FAST.Farm slice output (terrain following, and arbitrary slice) - #3392

Draft
andrew-platt wants to merge 194 commits into
OpenFAST:devfrom
andrew-platt:f/FF_sliceOutput
Draft

FAST.Farm slice output (terrain following, and arbitrary slice)#3392
andrew-platt wants to merge 194 commits into
OpenFAST:devfrom
andrew-platt:f/FF_sliceOutput

Conversation

@andrew-platt

@andrew-platt andrew-platt commented Jul 17, 2026

Copy link
Copy Markdown
Collaborator

Not ready to merge. See merge order below

Co-authored-by: Regis Thedin registhedin@gmail.com - conceptual development and multi-round plan iteration

Feature or improvement description

Adds two new FAST.Farm VISUALIZATION output features requested by the "Large Wind Farm Task" and issue #2383. Both live in dedicated required sections (between VISUALIZATION and OUTPUT); set NumPlaneSlices=0 / NumTerrainSlices=0 to disable.

  1. NumPlaneSlices — axis-aligned, extent-controlled planar sampling (Snippet 6 grammar from the plan). Each slice is defined by an origin(m), a plane keyword (XY, YZ, or XZ), and two in-plane extent(m) values. The sample grid is fixed to the low-res spacing (dX_Low/dY_Low/dZ_Low), so no npoints column is required. Invalid plane keywords are a fatal input error.
  2. NumTerrainSlices — terrain-following point-cloud sampling from an STL surface or a plain-text point cloud, each swept along one or more Offsets(m) in an OffsetNormal direction (facet-normal by default for STL). Every sample point is trilinearly interpolated in the low-res disturbed wind field at each output step.

Both features write self-describing XML VTK output (.vts StructuredGrid for planar slices, .vtp PolyData for point clouds) plus a matching .series JSON sidecar for ParaView's built-in time-series loader. Out-of-domain sample nodes are masked with IEEE quiet NaN so ParaView clips them automatically.

Each feature has its own sampling period (WrPlaneDT, WrTerrainDT) that falls back to DT_Low via DEFAULT, so users can sample the two feature streams at rates independent of the classic disturbed-wind output.

Feature 1 — three axis-aligned planes (conceptual and real output) fig1_arbslice_concept

The LESinflow_ArbSlice reg-test deck plants three slices: a horizontal hubXY plane (red, normal (0,0,1)), a cross-stream crossYZ plane through the turbine (green, normal (1,0,0)), and a stream-wise crossXZ plane (blue, normal (0,1,0)). All three intersect at the WT1 turbine.

fig2_arbslice_real

Above: horizontal hubXY plane at $t = 24$ s parsed directly from FAST.Farm.Plane.hubXY.008.vts. The plane is discretised at $121 \times 61$ nodes on the low-res spacing (dX_Low = dY_Low = 10 m); the wake deficit downstream of WT1 shows up as the dark low-velocity patch centred at $(1000, 1000)$.

Input file syntax (from the reg-test deck):

--- AXIS-ALIGNED PLANE SLICES (extent-controlled) --- [Feature 1, Snippet 6 grammar]
3                  NumPlaneSlices     - Number of axis-aligned extent-controlled planar slices (-) [0 to 99]
DEFAULT            WrPlaneDT          - Feature 1 sampling period (s); DEFAULT falls back to DT_Low
SliceName  origin(m)      plane    extent1(m)  extent2(m)
(-)        (m,m,m)        (-)      (m)         (m)
"hubXY"    (700 700 95)   XY       1200        600           ! 1200x600 XY plane at hub height
"crossYZ"  (1000 700 5)   YZ       600         300           ! 600x300 YZ plane through the turbine at x=1000
"crossXZ"  (700 1000 5)   XZ       1200        300           ! 1200x300 XZ plane through the turbine at y=1000
Feature 2 — STL mountain + hub cloud (conceptual and real output) fig3_terrainslice_concept

The LESinflow_TerrainSlice reg-test deck plants two slices:

  • "terr" — a 4-facet pyramid mountain read from mountain.stl (12 vertices, one per STL facet corner), swept along the per-facet normal to three offset sheets at 50/100/150 m. Total sample count: $3 \times 12 = 36$ points.
  • "hub" — a 9-point hand-authored point cloud read from hub_points.txt, one sheet at offset 0 with an explicit vertical OffsetNormal (0 0 1). Exercises the CSV-with-header parser path in ReadPointCloudFile.
fig4_terrainslice_real

Above: sampled velocity magnitude at $t = 24$ s parsed from FAST.Farm.TerrSlice.terr.008.vtp (green circles, 36 pts across three offset sheets) and FAST.Farm.TerrSlice.hub.008.vtp (blue squares, 9 pts at hub height). The underlying pyramid is drawn faint for scale.

Input file syntax (from the reg-test deck):

--- TERRAIN-FOLLOWING SAMPLING --- [Feature 2, PDF §2.8 grammar with SourceType flag]
2                  NumTerrainSlices   - Number of terrain-following point-cloud slices (-) [0 to 99]
DEFAULT            WrTerrainDT        - Feature 2 sampling period (s); DEFAULT falls back to WrDisDT
SliceName   Offsets(m)     OffsetNormal   SourceType   FileName
(-)         (m,list)       (-,-,-|dflt)   (STL|Point)  (quoted)
"terr"      50 100 150     default        STL          "mountain.stl"       ! sample above the mountain surface on 3 offset sheets
"hub"       0.0            (0 0 1)        Point        "hub_points.txt"     ! single sheet at hub height (offset 0)

Related issue, if one exists

Closes / addresses #2383 (parts pt. 1 = axis-aligned planar sampling, pt. 2 = terrain-following point cloud). The full design plan lives in FF_output_features_plan.md on the branch and cross-references the same issue.

Impacted areas of the software

Area File(s) Change
AWAE core modules/awae/src/AWAE.f90 New: ExtractSliceSub (bounded variant of ExtractSlice), ValidatePlaneSlices, EmitPlaneSlices, ReadSTLFile, ReadPointCloudFile, ValidateTerrainSlices, EmitTerrainSlices. Init calls both Validate* after existing slice validation; End closes both feature's .series sidecars.
AWAE I/O helpers modules/awae/src/AWAE_IO.f90 New: WriteVTK_PolyData (.vtp), WriteVTK_StructuredGrid_2D (.vts), streaming VTK_Series_Open / _Append / _Close for .series JSON sidecars.
AWAE registry modules/awae/src/AWAE_Registry.txt Flat-array fields on InputFileType / ParameterType / MiscVarType for both features (flat rather than array-of-derived-type — see design notes below).
AWAE types (generated) modules/awae/src/AWAE_Types.f90 Regenerated.
FAST.Farm glue glue-codes/fast-farm/src/FAST_Farm_IO.f90 New: ReadPlaneSlicesBlock and ReadTerrainSlicesBlock (both sections are required; NumPlaneSlices=0 / NumTerrainSlices=0 disables output).
Reg-tests reg_tests/CTestList.cmake Two new ff_regression entries — LESinflow_ArbSlice and LESinflow_TerrainSlice.
Reg-test decks reg_tests/r-test submodule bump to 64feb0011 New decks with baselines (see r-test PR below).
Docs docs/source/user/fast.farm/InputFiles.rst, OutputFiles.rst, ModelGuidance.rst, FutureWork.rst, api_change.rst New sub-sections describing input syntax, output naming (.Plane.<name>.<t>.vts, .TerrSlice.<name>.<t>.vtp, matching .series sidecars), NaN masking, and the independent sampling-rate story. FutureWork links back to #2383 for the arbitrary-orientation extension.

Additional supporting information

Commit layout on this branch (8 logical commits on top of b/FF_wake_start_noise):

Commit Subject
12af71a69 FAST.Farm: foundation for terrain-following and axis-aligned planar output
eff99bcdd FAST.Farm: Feature - axis-aligned planar sampling (NumPlaneSlices)
e7ace15fd FAST.Farm: Feature - terrain-following point-cloud sampling (NumTerrainSlices)
cdfb61065 FAST.Farm docs: document the new NumPlaneSlices / NumTerrainSlices blocks
1218c9f01 FAST.Farm: fix variable shadowing warnings (nannan_val, namenameOut)
c7d4c9eba FAST.Farm: WrPlaneDT/WrTerrainDT DEFAULT falls back to DT_Low (not WrDisDT)
24cab6932 FAST.Farm: NumPlaneSlices normal column → plane keyword (XY/YZ/XZ)
5585b8229 FAST.Farm: make PlaneSlices and TerrainSlices input sections required

Design notes worth calling out at review:

  • Flat arrays instead of AWAE_PlaneSliceType/AWAE_TerrainSliceType. An earlier draft used derived types for the per-slice state, but the generated per-element pack/unpack in FAST_Farm_Types.f90 blew past the 7.7 GiB container's memory ceiling during f951 code-gen. The switched design uses PlaneSliceOrigin(3,:) / PlaneSliceExtent1(:) / etc. plus a prefix-sum TerrainSliceOffsetIdx(:) into a packed TerrainSliceOffsetsPacked(:) — the ragged offsets stay ragged without paying the derived-type registry cost.
  • Duplicate-emit guard for t = 0. Farm_InitialCO calls AWAE_CalcOutput twice at $t=0$ (once for validation and once for the actual initial output). Both features carry a scalar m%LastPlaneSliceN / m%LastTerrainSliceN initialised to -1; the Emit* routines early-return when m%Last*N == n so the identical .vts/.vtp file isn't rewritten and the .series sidecar doesn't get a duplicate t=0 entry.
  • NaN masking for out-of-domain nodes. EmitPlaneSlices pre-fills the write buffer with ieee_value(0.0_SiKi, ieee_quiet_nan), then calls the bounded ExtractSliceSub for only the in-domain window [iLoE, iHiE] × [jLoE, jHiE]. EmitTerrainSlices NaN-fills any point whose $(x, y, z)$ falls outside [oXYZ, oXYZ + (nXYZ-1)·dXYZ]. ParaView masks these automatically.
  • Plane keyword validation in Feature 1. The plane column accepts only XY, YZ, or XZ (case-insensitive); any other value is a fatal init error. The error message names the offending slice and value. The terrain-following feature (NumTerrainSlices) serves the arbitrary-orientation use case.

Conceptual overlap with the existing AWAE_vtk .vtk.series writer. During the rebase onto b/FF_wake_start_noise (see "Test results" below) it became clear that this branch's AWAE_IO.f90::VTK_Series_Open/_Append/_Close helpers and the base branch's AWAE_vtk.f90::Write_DisWind_Series/Write_WakePlane_Series/Write_WireFrame_Series both emit ParaView .series JSON sidecars, but with materially different flow patterns:

AWAE_vtk (base branch) AWAE_IO (this PR)
Trigger Called once from AWAE_End after the run Streamed: Open at Init, Append each CalcOutput, Close at End
File extension .vtk.series (legacy VTK) .vts.series / .vtp.series (XML VTK)
Requires known step count? Yes — iterates 0..NumDT/WrDisSkp1 No — accretes as steps happen
Behaviour on abnormal termination Nothing written Whatever steps completed are still valid ParaView-loadable

No literal duplicate code and both routines coexist without collision on the merged branch, but a future cleanup PR could consolidate them into a single AWAE_vtk interface (streaming API for both) — kept out of this PR to hold the rebase minimal.

Generative AI usage

Substantial portions of this PR — implementation across all 4 phases, rebase conflict resolution onto b/FF_wake_start_noise, and reg-test deck authoring — were generated with AI assistance. Co-author trailers are on every commit.

Co-authored-by: Claude noreply@anthropic.com
Co-authored-by: Claude noreply@anthropic.com
Co-authored-by: Google Gemini 3.5 Flash gemini@google.com
Co-authored-by: GitHub Copilot copilot@github.com

Test results, if applicable

  • r-test branch merging required — bumps reg_tests/r-test submodule pointer. All 9 FAST.Farm regression decks now include both required sections (set to 0 when inactive). See companion r-test PR.

Full FAST.Farm regression suite in build-docker-double (double-precision, AMREX_READER=ON, RelWithDebInfo):

100% tests passed, 0 tests failed out of 9
Total Test time (real) = 126.99 sec

    Start 84: AMReX .................. Passed
    Start 85: TSinflow ............... Passed
    Start 86: LESinflow .............. Passed
    Start 87: LESinflow_ArbSlice ..... Passed   ← Feature 1 deck (new)
    Start 88: LESinflow_TerrainSlice . Passed   ← Feature 2 deck (new)
    Start 89: TSinflow_curl .......... Passed   ← b/'s updated wake-decay baseline still matches
    Start 90: ModAmb_3 ............... Passed
    Start 91: TSinflowADskSED ........ Passed
    Start 92: MD_Shared .............. Passed

Output verification. For each new deck, the per-step .vts/.vtp files were spot-checked by parsing the XML and confirming (a) node coordinates match the input origin + i·d_axis / Pts + Offset·OffsetNormal arithmetic to machine precision, (b) .series sidecars have exactly one entry per CalcOutput invocation with monotonically increasing time, and (c) no duplicate t = 0 entry from the double-call inside Farm_InitialCO.

Merge order

  • final release of 5.0.1
  • cross merge of 5.0.1 to dev
  • perform a check of code redundancy after merging dev to this branch
  • merge

andrew-platt and others added 30 commits May 12, 2026 13:07
Add Write_Planes_WireFrame and PlaneCorners helper to emit each
active wake plane's outline in the global frame using vtk_lines.

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
This is not complete, but contains some of the info.
The .vtk.series file is not working as expected yet.
The edge points on the wake planes were not getting updated.  The loop limits were one point short.
New guidance for calculating with larger safety factors, and different calculation for catesian+curl than polar.

Co-authored-by: Regis Thedin <registhedin@gmail.com>
The third term in the calculation was incorrect.  This routine has not been used since it was added in 2014, so no results will be affected.

This was reported on the forum here: https://forums.nlr.gov/t/beamdyn-questions/9518/3
[BugFix] Error in `Quaternion_to_DCM` calculation
WD: fix smooth tapering from `numDFull` to `numDBuff` for curled wake
- Create modules/awae/src/AWAE_vtk.f90 containing module AWAE_vtk with
  all wake-plane VTK output routines: PlaneAxes, PlaneCorners,
  Write_Planes_WireFrame, Write_Planes_Data, Write_WakePlane_Data_File,
  Write_WakePlane_Series, and Write_NullPlane.

- Remove those routines from AWAE_CalcOutput's internal contains block
  and convert them to standalone module subroutines with explicit
  arguments (p, u, m, n, t, Tstr) replacing host association.

- Move the per-step Write_WakePlane_Series call from AWAE_CalcOutput to
  AWAE_End, writing series files once at shutdown instead of every output
  step.  Merge the former two-routine pattern (Write_WakePlane_Series +
  Write_AllWakePlane_Series) into a single Write_WakePlane_Series(p, m).

- Extract the inline null-plane block into Write_NullPlane subroutine.

- Remove unused variables: FmtStr argument, PlaneNum, fileExists,
  rootDir from the former Write_WakePlane_Series.

- Add AWAE_vtk.f90 to CMakeLists.txt and vs-build project; add
  'use AWAE_vtk' to module AWAE.

Co-authored-by: GitHub Copilot <175728472+Copilot@users.noreply.github.com>
…s files

Add a new routine Write_WireFrame_Series(p) to AWAE_vtk that writes one
ParaView .vtk.series index file per turbine for the WakePlanesWireFrame
VTK outputs. Called once at shutdown from AWAE_End alongside the existing
Write_WakePlane_Series call.

Co-authored-by: GitHub Copilot <175728472+Copilot@users.noreply.github.com>
Add Write_DisWind_Series to AWAE_vtk module to generate ParaView
.vtk.series index files for Low-resolution disturbed-wind XY, YZ,
and XZ output slices.  The routine is called from AWAE_End for each
valid slice, matching the existing pattern for wake-plane and
wireframe series files.

Co-authored-by: GitHub Copilot <noreply@github.com>
…mary file

Update AllFmt format strings in BeamDyn_IO.f90 for Init_Nodes and Init_QP
arrays to use proper spacing and trailing comma in Fortran format specs.
This was preventing writing of this information.

Remove redundant format statement from yaml_write_array2R[48]
I missed the usage of it right afterwards
Fix format issue preventing fields from appearing in BeamDyn YAML summary file
…dels

In WD_CalcOutput, the firstPass branch computed y%Vx_wake (polar) via
NearWakeCorrection but never populated y%Vx_wake2 for Cartesian/Curl models.
Additionally, the unconditional Cartesian/Curl taper block overwrote y%Vx_wake2
with xd%Vx_wake2 (still zeros) on firstPass, discarding any values set earlier.

Changes:
- Add Axisymmetric2CartesianVx conversion in the firstPass block of WD_CalcOutput
  to populate y%Vx_wake2 for Curl/Cartesian models.
- Guard the Cartesian/Curl taper block with (.not. OtherState%firstPass) so it
  does not overwrite the firstPass initialization.
- Add Mod_Wake guard and NearWakeCorrection call in InitStatesWithInputs for
  Cartesian/Curl state initialization.
- Remove unused GridTol parameter in AWAE.f90.

Co-authored-by: andrew-platt <andy.platt@nrel.gov>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
andrew-platt and others added 2 commits August 18, 2026 09:49
…sMerge

Fix GitHub Copilot identified issues in rc-5.0.1 cross-merge
@andrew-platt andrew-platt added ai assisted AI written with strong human guidance. and removed ai slop Evidence suggests it was entirely AI generated and submitted without oversite. labels Aug 25, 2026
andrew-platt and others added 26 commits August 25, 2026 17:51
OLAF: fix source-panel filename parsing (relative path)
The high-res search requested NumDT*n_high_low-1 time slices, but the largest
high-res index actually read by AWAE_UpdateStates is (NumDT-1)*n_high_low
(i_hl is forced to 0 on the final low-res step), i.e. (NumDT-1)*n_high_low+1
slices.

For n_high_low=4 this over-requested by 2, so a run whose LES data exactly
covered TMax was rejected for missing two directories it never reads. For
n_high_low=1 -- legal, since DT_High need only divide DT_Low -- it
under-requested by one, which is harmless today but becomes an out-of-bounds
read once the directory index is looked up from a table.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
amrex_find_subvols_c compared the directory index suffix against DirStartIndex
as a string. AMReX pads plotfile indices to a *minimum* width and widens beyond
it as needed, so once a run passes 99999 steps a six-digit index sorts before a
five-digit one: "100030" < "27150". Every such directory was silently skipped
and the run failed with a misleading count mismatch pointing at data that was
present on disk.

Parse the suffix to an integer and compare numerically. While here, harden the
extraction: anchor the prefix match at position 0, require the suffix be all
digits, and use stoll behind a try/catch. Previously a sibling directory such as
ffboxes_1_00010_bak threw std::invalid_argument across an extern "C" boundary,
terminating the process with no error message.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
ReadWindAMReX formatted the directory index with a fixed-width I<DirIndexLen>
edit descriptor, where DirIndexLen comes from len_trim(DirStartIndex). AMReX
treats that width as a minimum and widens it as the step counter grows, so a run
starting at "27150" and reaching index 100030 wrote '*****' and failed to open
the directory.

Format with I0 and left-pad to DirIndexLen, so the width matches
amrex::Concatenate for any index magnitude.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Add DirIndexLow(:) and DirIndexHigh(:) to AWAE_ParameterType, to hold the
plotfile directory index for each low- and high-resolution time step. These
replace the constant-stride assumption encoded in DirIndexDeltaLow/High, which
cannot represent LES output written with a varying solver time step.

Unused in this commit; populated and consumed in the following ones. The old
scalars are left in place so this commit builds standalone.

AWAE_Types.f90 regenerated with -DGENERATE_TYPES=ON.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… stride

amrex_find_subvols_c scanned the sub-volume directories, read each plotfile
header, validated that its time was a multiple of dt -- and then threw the
result away, returning only a first index and a single stride. That stride was
then required to be uniform across every directory.

The stride is an artifact of how the LES advanced its solver, not a property of
the data. AMR-Wind run with time.initial_dt = 0.025 until delay_time, then
fixed_dt = 0.05, writes boxes every 0.1 s throughout -- uniform in time to 1e-10
s -- but the index stride steps from 4 to 2 at the transition, and the run was
rejected.

Return the full time step -> directory index table instead. Each directory is
assigned the step round((header_time - start_time)/dt) and must land within
tolerance of it; a step claimed by no directory or by two is fatal. This
subsumes the old fmod filter, the count check and the stride check, and is
strictly stronger than all three: missing files and duplicates are now detected
exactly, and the residual test still skips deliberately decimated output.

The matching tolerance is max(1e-6, 1e-9*(|t0|+|t|)). The error being absorbed
is the drift a solver accumulates by summing its time step, which grows with
absolute simulated time -- a precursor restarted at t = 3e4 s carries far more
of it than one starting at zero -- so a fixed absolute tolerance is the wrong
shape. The floor preserves the previous behavior for runs starting near t = 0.

High-resolution sub-volumes are now compared to sub-volume 1 element by element
rather than by stride alone, which catches a sub-volume whose whole sequence is
offset from its siblings -- previously accepted, and silently supplying each
turbine a different instant in time.

ReadWindAMReX still builds names arithmetically in this commit; it is switched
to the table next.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ting it

ReadWindAMReX rebuilt the directory name at every read as
DirStartNum + Delta*n, which is only correct when the LES wrote its plotfiles at
a constant step index stride. Read the index out of the table built during
initialization instead.

This is what makes variable-time-step LES output work end to end: with the old
arithmetic, data whose stride changed part way through would resolve to a
directory holding a *different* simulation time -- for a 4->2 stride change 1000
steps in, step 1001 wants t+100.1 s but the formula lands on t+100.2 s, and the
error grows from there. Silently, with no failure.

Also guard the lookup: an out-of-range step is fatal rather than clamped, since
clamping would freeze the inflow for the tail of a run instead of reporting it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
DirStartNum, DirIndexDeltaLow and DirIndexDeltaHigh existed only to reconstruct
directory names arithmetically. With the index table in place they have no
remaining consumer.

AWAE_Types.f90 regenerated.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Five tests covering the behavior the preceding commits changed:

  variable_stride    stride 4 -> 2 with a uniform 0.1 s output interval, using
                     header times that carry the drift seen in real data
  wide_index         five-digit start index followed by six-digit indices
  missing_step       a gap must name the step, not just a count
  duplicate_step     two directories on the same time must be rejected
  out_of_tolerance   a near miss must be reported as such

variable_stride and wide_index fail on rc-5.0.1 and pass here; the other three
cover conditions the old code either mis-reported or accepted silently.

Fixtures are copies of the existing plotfile directories with only the Header
time and level_steps rewritten -- 130 KB total. The binary Level_0 data is never
opened by amrex_find_subvols.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two changes to the search, both cost-only -- the directories selected are
identical.

Walk candidates in ascending index order and stop at the first one past the
requested window. Simulation time rises with the step counter, so nothing after
it can qualify. Previously the search read a header for every directory matching
the prefix no matter how short the FAST.Farm run: a 100 s window over a
precursor that had written 2200 low-res boxes read all 2200.

Read the plotfile Header directly instead of constructing a PlotFileData, which
also opens Level_0/Cell_H and builds a DistributionMapping to obtain a number
already present in the text. Grid validation is unchanged and still runs on
every directory -- it just takes its inputs from a cheaper source. The text
parse is validated against amrex_read_header_c on the start directory before
being trusted, and falls back to it on any parse failure, so a plotfile whose
layout does not match is handled rather than mis-read.

Measured on a 65-turbine dataset: the low-res search for a 100 s window went
from ~7.6 minutes to 0.85 s.

No OpenMP. The plan allowed for parallelising the scan, but after these two
changes the measurement no longer justifies the complexity.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The AMReX section stated that the directory for time step n is
DirStartNum + n * Delta_index. That is no longer how the reader works, and
describing a constant stride as part of the file-naming contract is what led a
user to conclude their valid variable-time-step data was malformed.

Replace with the actual rule -- directories are matched by the simulation time
in their Header -- and state explicitly that the index stride may vary. Document
the matching tolerance, the exactly-one-directory-per-time-step requirement, and
the cross-sub-volume time consistency check.

No input file changes: DirStartIndex keeps its meaning and no field is added,
removed, or reinterpreted.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
amrex_read_data_c called PlotFileData::get(level, varname) once per velocity
component, which re-reads the whole level each time. The cost of a read is set by
the number of boxes rather than the volume of data, so three passes cost three
times as much.

Read every component at once with get(level) and index the component off the
Array4 instead.

Measured against a low-resolution sub-volume written with 93456 boxes (a
528x472x24 grid at chunk_size_vec = 8 8 1): 63 s -> 16.5 s. On the
high-resolution boxes, where per-box overhead dominates a 48 KB array, 55 ms ->
7 ms; that is the one that matters in aggregate, since FAST.Farm reads
NumTurbines * (n_high_low + 1) high-resolution boxes per low-resolution step
against a single low-resolution one.

Output is unchanged: dumping both readers over a real 68 MB low-resolution box
and a high-resolution box gives byte-identical results.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
ReadWindAMReX is where the 0-based directory index table is consumed and where
the index suffix is formatted, and neither had direct coverage. Four cases:

  lookup        step 1 of table [0, 6, 12] reads directory 00006, checked
                against a direct read of that directory rather than just the
                absence of an error
  out_of_range  a step past the table is fatal and names the available range;
                it must never be clamped, which would silently freeze the
                inflow for the rest of the run
  wide_index    DirIndexLen of five with index 100000 widens the suffix to six
                digits and reads the right box (the subvolwide fixtures)
  no_table      an unpopulated table is reported as such rather than looked
                up as zeros

Writing wide_index showed that pointing amrex_read_data at a directory that
does not exist aborts the process rather than returning an error: amrex::Error
calls abort() instead of throwing, so the try/catch in amrex_read_data_c never
fires. That is pre-existing and left alone here; the test was reworked to read
an existing fixture instead.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
With AMREX_READER=ON and BUILD_TESTING=ON the amrex_reader_utest target failed
to link:

    ld: crt1.o: in function `_start': undefined reference to `main'

Both of its sources are Fortran and its entry point is a Fortran program, but
awaelib transitively pulls in the C++ awaelib_c, so CMake resolves the link
language to CXX and the C++ driver's crt1.o then looks for a C main. Pin the
linker language to Fortran and pull in the C++ runtime explicitly with -cxxlib,
which is what the Fortran driver needs to resolve the AMReX and awaelib_c
symbols.

Every other unit test target in this file links with the Fortran driver
already; this one only differed because of the C++ dependency.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
-cxxlib is an Intel Fortran option; gfortran rejects it, so any GCC build with
AMREX_READER=ON and BUILD_TESTING=ON failed at link. It is also unnecessary:
with the link language pinned to Fortran the target links cleanly with ifx
without it, which is how glue-codes/fast-farm handles the same awaelib ->
awaelib_c C++ dependency.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…laimed

The scan walked candidates in ascending index order and stopped at the first
directory whose header time fell past the requested window, on the assumption
that time rises with the step counter. Within one run it does. But leftovers
from an earlier run with a different time step share the index base and carry
later times on lower indices, so a single stale directory could end the walk
before the valid data that sorts after it -- and the run then failed with a
spurious "no sub-volume directory was found for time step N" pointing the
user at data that was on disk. A killed run restarted with a smaller
initial_dt is exactly this case.

Keep the early stop, but only once every step has been claimed. While any
step is still missing, index order is not trusted to imply time order and the
walk continues; a missing step is reported only after the full scan. Clean
directories keep the same cost as before.

The new subvolstale fixture has such a leftover (index 00002 at t = 5.0)
ahead of the directories holding steps 1 and 2. It fails on the previous
commit with the message above and passes here.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…range

The text-parse fast path accepted any ncomp >= 1, and amrex_read_data_c had no
component check at all, so on the fast path only the starting directory was
ever validated for ncomp == 3. A directory with fewer components then reached
fab(i, j, k, ivar) with ivar past the FAB's component count: an out-of-bounds
read yielding garbage velocities instead of an error. The text parser now
returns false for anything but three components (which sends that directory
to amrex_read_header_c, where it is reported), and amrex_read_data_c checks
nComp() before reading.

Also:

- The parser cross-checks level_steps against the directory index it was
  asked about; a mismatch means the name does not describe the contents, so
  fall back to the authoritative reader rather than trust the text.
- parse_dir_index deliberately uses stoll to cope with wide indices, but the
  result was narrowed to int unchecked. Both narrowing sites now fail with a
  clear message past the 32-bit range instead of wrapping.
- The comment claiming the parser is safe to call from several threads is
  dropped; the AMReX read path is serial and nothing here relies on it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
amrex_find_subvols set ErrStat = ErrID_None only inside the ENABLE_AMREX_LIB
branch, so the #else branch called SetErrStat on undefined intent(out) dummies
(SetErrStat does ErrStat = MAX(ErrStat, ...)). The two older shims never
initialized them in either branch. Initialize in all three before the #ifdef.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The text-parse fast path only turns on when it agrees with amrex_read_header_c
on the starting directory, and none of the existing fixtures satisfy that:
they are hand-built subsets whose Header domain box (256x256x128) is much
larger than the box array in Cell_H (3x4x5), so every test took the slow path
and the parser had no coverage. Real Kynema/AMR-Wind sub-volume output is
tiled -- one FAB spanning the domain box -- and the fast path does engage on
it in production, which is where the measured speedup came from.

Add a fixture with that real layout: three consecutive high-resolution
snapshots as written by the sampler (headers, single FAB, three masked
velocity components, 0.1 s apart), with the FAB payload replaced by a known
field u = 1+i, v = 2+j, w = 3+k so the read can be checked exactly rather than
for plausibility. Expose the parser to Fortran as amrex_parse_header_text and
add four tests: the search and the read on the tiled fixture, agreement
between the text parser and amrex_read_header on it, and the disagreement on
the hand-built fixture that explains why those fall back.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The AMReX section said every step 0..NumDT-1 needs a directory. That is the
low-resolution count; each high-resolution domain needs steps 0 through
(NumDT-1)*(DT_Low/DT_High). Also state that directories between required
steps are ignored, so the LES may write sub-volumes more often than FAST.Farm
reads them, provided the FAST.Farm step is a multiple of the output interval.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
When OffsetNormal = 'default', terrain-following sample points are now
displaced purely in Z (vertical translation) rather than along the
per-facet STL normal. This matches the physical intent of sampling at
a fixed height above terrain.

An explicit OffsetNormal vector still displaces along that direction.

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: GitHub Copilot <copilot@github.com>
Fixes for AMReX data handling for FAST.Farm
Follow-up to #33. Four issues found reviewing that work.

Match the sub-volume prefix on the entry name, not the whole path. The scan
was anchored with rfind(prefix, 0), but directory_iterator over "." hands
back entries as "./name", so a WindDirPrefix carrying no directory of its
own matched nothing beyond the starting directory and the run died with
"no sub-volume directory was found for time step 1" with every directory
present. Not reachable through FAST.Farm, which prepends PriPath (at least
"./") to a relative WindDirPrefix, but it is reachable for any direct
caller of amrex_find_subvols -- the unit tests among them.

Check the grid in amrex_read_data against the caller's array. The reader
writes the destination by grid index and had no idea how big it was, so a
plotfile describing a larger grid wrote past the end of it. The sub-volume
search does check this at initialization, but it may have taken the Header
text fast path, which reads the domain box while the read takes the union
of the box array -- the two agree on real sampler output and are only
verified to agree on the starting directory. Also require the boxes to
tile the grid: they are disjoint, so summing their volumes settles it, and
anything less leaves part of the destination holding the previous step.

Count claimed steps instead of rescanning the table for each directory
past the window. The scan stops at the first such directory once the table
is complete, so this cost nothing on a good dataset -- but when a step is
genuinely missing it never stops, making the walk O(directories x steps)
on exactly the path that was supposed to report the problem quickly.

Say which tolerance a near miss was measured against. The tolerance grows
with simulated time, so the value quoted in the message was not quite the
one applied to the directory being reported.

Also soften the "exactly one directory per step" claim in the docs and the
routine comment: the walk stops early once the table is complete, so a
stale duplicate sorting after the first directory past the window is never
read. The check is best effort, not exhaustive.

Tests: two added, both fail without the corresponding fix. All 21 AMReX
reader unit tests pass, as do the other six utest suites, and FAST.Farm
links.

Assisted-by: Claude Code:claude-opus-5[1m] [Bash] [WebFetch]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants