Skip to content

fix(recomp): order PS2 floats as sign-magnitude for compares, MIN/MAX and VU MAX/MINI - #321

Open
abilliontokens wants to merge 1 commit into
ran-j:mainfrom
abilliontokens:fix/fpu-ordering
Open

abilliontokens wants to merge 1 commit into
ran-j:mainfrom
abilliontokens:fix/fpu-ordering

Conversation

@abilliontokens

@abilliontokens abilliontokens commented Oct 16, 2026 •

Copy link
Copy Markdown

Defect

The PS2 has no NaN or Inf: DIV.S by zero (and every overflow) yields an exponent-255 number pattern — 0x7FFFFFFF is what #272's ps2_fdiv returns for x / 0 — and any exponent-0 pattern is zero (the FPU flushes denormals). Three instruction groups were still translated to host IEEE semantics:

  • MIN.S/MAX.S emitted std::min/std::max. Fed a ±max pattern (a saturated division result), the host sees a NaN: std::max(NaN, 2) returns its second argument and a clamp like max(min(x/0, 1), 0) lets the ±max through instead of clamping to 1/0. PCSX2's MAX_S/MIN_S order PS2 floats as sign-magnitude and flush an exponent-0 selection to signed zero.
  • C.EQ/C.LT/C.LE (and their decoding aliases) compared with ==, <, <=, and the "unordered" forms ORed in isnan(). With a ±max operand every compare answers false on the host (NaN), so branches keyed on them take the wrong side. PCSX2 compares fpuDouble keys.
  • VU MAX/MINI emitted _mm_max_ps/_mm_min_ps. SSE returns the second operand when either is NaN, and PS2 ±max lanes are NaN patterns on the host. PCSX2's fp_max/fp_min compare the raw lanes as integers, reversed for two negatives.

Symptom

Stuntman (SLUS-20250): the car physics divides by a wheel constant that is 0 for the first mission's car; ±max → [0,1] clamp produced garbage instead of 1/0, the driven-wheel integrator span at 2.65e9 and the speedometer read 999 forever. A rigid-body c.lt.s with |ω| = 0 took the wrong branch ~9×/frame from frame 0. Both confirmed fixed in-title by an A/B diagnostic that logged every compare/min/max whose answer changed.

Fix

  • ps2_fkey maps a float to its sign-magnitude order key (exponent-0 ⇒ 0, exponent-255 ordered by magnitude); FPU_C_* compare keys and the "unordered" forms reduce to the four the R5900 implements (the decoder names a title's c.lt.s/c.le.s C_OLT/C_OLE — noted in a comment).
  • ps2_fmax/ps2_fmin select by key and flush an exponent-0 result to signed zero (ties keep the first operand, like std::max); MAX.S/MIN.S emit FPU_MAX_S/FPU_MIN_S.
  • ps2_vu_max_ps/ps2_vu_min_ps do the integer lane compare with the both-negative reversal; all six VU MAX/MINI emitter forms (plain, broadcast, immediate) call them.

Generated code must be regenerated to pick this up. The new test file compiles with /fp:precise (like generated code) so the tests cannot be folded away by /fp:fast.

Evidence

  • New PS2FpuOrder suite (6 cases): exponent-255 ordered largest (by mantissa), exponent-0 equal to zero (±0 equal, denormals), MIN/MAX clamping ±max and flushing denormals, the title's own clamp sequence (0x001A94FC–0x001A9508 raw words, both value and emitter checks), VU lane ordering with operands swapped so no SSE NaN rule can pass, and the six emitters. All fail on main.
  • ps2x_tests: Correction (2026-10-16): an earlier version of this description said EE scheduler selects absolute priority then FIFO also fails on pristine main. That was wrong. Our local MSVC build was configured with -DCMAKE_CXX_FLAGS=-msse4.1, which replaces CMake's default flags and so drops /EHsc, and that test failed only in that build. With default flags the suite passes on main: upstream CI is green on eb8688b (run 38574762065). The ps2x_tests totals measured in that build are withdrawn until re-run; every comparison above used the same build on both sides. TestSuiteRepair found the misconfiguration.
  • Builds on Emulate PS2 float semantics for EE FPU results and DIV.S/RSQRT.S #272's helpers; independent of fix(runtime): truncate and saturate FPU CVT.W.S like the EE FPU #316/fix(recomp): give VU0 DIV/SQRT/RSQRT the hardware FDIV-unit rules #317.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant