Skip to content

Emulate PS2 float semantics for EE FPU results and DIV.S/RSQRT.S - #272

Merged
ran-j merged 2 commits into
ran-j:mainfrom
drakolordx7:fix/ps2-float-semantics
Oct 14, 2026
Merged

ran-j merged 2 commits into
ran-j:mainfrom
drakolordx7:fix/ps2-float-semantics

Conversation

@drakolordx7

@drakolordx7 drakolordx7 commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

The PS2 EE FPU and the VUs have no Inf, NaN or denormals. Exponent 255 is an ordinary exponent (0x7F800000 is 2^128, the largest value is 0x7FFFFFFF), exponent 0 is zero, a result of 2^129 or more saturates to ±0x7FFFFFFF, a result below 2^-126 becomes ±0, and x/0 gives ±0x7FFFFFFF. The generated code followed IEEE, so Inf/NaN leaked into later maths (for example normalising a zero-length vector).

Changes

  • ps2_runtime_macros.h: new ps2_fadd / ps2_fsub / ps2_fmul / ps2_fdiv / ps2_fsqrt / ps2_frsqrt and ps2_vadd / ps2_vsub / ps2_vmul. Each runs the host operation and checks the result range; values with exponent 255 are handled in double on an out-of-line path. DIV.S by zero gives ±max with the sign of fs xor ft, RSQRT.S with a zero ft gives ±max with the sign of fs, SQRT.S and RSQRT.S use |ft|.
  • fpu_translator.cpp: DIV.S no longer emits copysignf(INFINITY, ...). RSQRT.S was emitted as 1/sqrt(fs); the EE computes fs / sqrt(ft).
  • Tests (ps2_float_semantics_tests.cpp, 8 tests): the 174 hardware results for add, sub, mul, div, sqrt and rsqrt from ps2autotests (tests/cpu/ee_fpu), plus edge values for each macro and the translated DIV.S/RSQRT.S code with distinct fd/fs/ft registers. 163 rows match bit for bit; 10 differ in the last mantissa bit and 1 is the exponent-0 operand case below, all marked in the table.

Found while recompiling Killzone (SCUS-97402): RSQRT.S is used in 382 functions there (vector normalisation), and with the wrong operands normals and lighting came out wrong.

Not emulated: the hardware's rounding of the last mantissa bit, and an ADD.S/SUB.S/MUL.S or vector operand with exponent 0 and a non-zero mantissa (it keeps its tiny host value instead of reading as zero).

Not included, because open PRs cover them: CVT.W.S truncation/saturation (#215) and the VU0 macro VDIV/VSQRT/VRSQRT Q operations (#198). Expect small textual conflicts with those two in ps2_runtime_macros.h and the test registration.

Cost (micro-benchmark, MSVC /O2 /Ob1, ns per instruction, main → this PR): ADD.S/SUB.S/MUL.S 0.14 → 0.34, DIV.S/RSQRT.S 0.92 → 1.21, VADD/VSUB/VMUL 0.16 → 0.59. Generated code has to be regenerated and rebuilt.

Tests: ps2x_tests run from the repository root: 444/444 (436/436 on main).

Made by drakolord and assisted with Claude Code.

The EE FPU and the VUs have no Inf/NaN. A value with exponent 255 acts
like a huge normal number, an overflowing result saturates to +-max
(0x7F7FFFFF) and x/0 gives +-max with the sign of (fs xor ft). The
generated code followed IEEE instead, so Inf/NaN leaked into later maths
(a normalisation of a zero-length vector, for example).

- ps2_runtime_macros.h: add ps2_fclamp/ps2_fdiv/ps2_frsqrt/ps2_vclamp.
  FPU_ADD_S/SUB_S/MUL_S clamp their inputs and result, FPU_DIV_S returns
  +-max on a zero divisor, FPU_SQRT_S takes sqrt(|x|), and PS2_VADD/VSUB/
  VMUL clamp every lane. FPU_RSQRT_S is new.
- fpu_translator.cpp: DIV.S no longer emits copysignf(INFINITY, ...) and
  wraps the DZ flag update and the division in one block. RSQRT.S was
  emitted as 1/sqrt(fs); the EE computes fs / sqrt(ft).

Found while recompiling Killzone (SCUS-97402): RSQRT.S appears in 382
functions there (vector normalisation), and with the wrong operands the
normals and lighting came out wrong. Verified with the new unit tests
(edge values for each macro, and the translated DIV.S/RSQRT.S code with
distinct fd/fs/ft), and by running Killzone with the change. Generated
code must be regenerated and rebuilt to pick it up.

CVT.W.S and the VU0 macro VDIV/VSQRT/VRSQRT Q-register operations are
left to ran-j#215 and ran-j#198, which cover them.

Made by drakolord and assisted with Claude Code.
#define PS2_PNOR(a, b) _mm_xor_si128(_mm_or_si128((__m128i)(a), (__m128i)(b)), _mm_set1_epi32(0xFFFFFFFF))

// PS2 float semantics (EE FPU and VU). The PS2 has no Inf/NaN: a value with exponent 255 behaves like a huge
// normal number, an overflowing result saturates to +-max (0x7F7FFFFF) and x/0 gives +-max with the sign of

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

0x7F7FFFFF is a float IEEE of 32 bits and not FPU from PS2.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right. 0x7F7FFFFF is the largest IEEE float; on the PS2 exponent 255 is an ordinary exponent, so the largest value is 0x7FFFFFFF. Fixed in 27b2749: saturation and x/0 now give ±0x7FFFFFFF. I checked the results against the hardware output in ps2autotests (tests/cpu/ee_fpu/*.expected) and added those 174 rows to the test suite.

}

// fs / sqrt(|ft|); the radicand's sign is ignored.
inline float ps2_frsqrt(float a, float b)

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cheking this bettter you use a zero-denominator branch that reuses the sign rule for ordinary division, but RSQRT.S does not have the same behavior when ft is negative zero.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct, that was the DIV.S rule. The hardware ignores the sign of ft everywhere in RSQRT.S, so with a zero ft the result takes the sign of fs alone: rsqrt(+0, -0) = 0x7FFFFFFF and rsqrt(-0, +0) = 0xFFFFFFFF in the ps2autotests output. Fixed in 27b2749, with tests for all four sign combinations. SQRT.S of -0 now gives +0 as well.

// normal number, an overflowing result saturates to +-max (0x7F7FFFFF) and x/0 gives +-max with the sign of
// (numerator xor denominator). Hosts that follow IEEE produce Inf/NaN here, which then poisons later maths
// (e.g. a normalisation of a zero-length vector).
inline float ps2_fclamp(float x)

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As I said early the clamping policy here conflates the host IEEE float range with the guest PS2 float range.

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And thats cary to ps2_fmax_signed and ps2_vclamp

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, and fixed in 27b2749. ps2_fclamp, ps2_fmax_signed and ps2_vclamp are gone. Operands with exponent 255 are no longer touched: 0x7F800000 is treated as 2^128, and anything involving such a value is computed in double and converted back. So 2^127 * 2 gives 0x7F800000, 2^128 + 2^128 saturates to 0x7FFFFFFF, and sqrt(0x7F800000) gives 0x5F800000, all matching the hardware table. The vector macros follow the same rules per lane.

Two things are still not emulated, and both are marked in the test table: the hardware's rounding of the last mantissa bit (10 of the 174 rows differ by one bit), and an ADD.S/SUB.S/MUL.S operand with exponent 0 and a non-zero mantissa, which the hardware reads as zero (1 row). Setting DAZ/FTZ on the guest threads would cover the second one for free, if you want that as a follow-up.

#define PS2_VADD(a, b) _mm_add_ps((__m128)(a), (__m128)(b))
#define PS2_VSUB(a, b) _mm_sub_ps((__m128)(a), (__m128)(b))
#define PS2_VMUL(a, b) _mm_mul_ps((__m128)(a), (__m128)(b))
#define PS2_VADD(a, b) ps2_vclamp(_mm_add_ps(ps2_vclamp((__m128)(a)), ps2_vclamp((__m128)(b))))

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we benchmark the vector-clamping changes separately from the DIV.S/RSQRT.S fixes?

Becasue each of these macros now applies the vector clamp to both inputs and the result.

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not claiming a measured performance regression but just to check dont need to push a bench code just do this test please.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Measured separately. Blocks written the way the recompiler emits code (one statement per instruction, operands loaded from and stored to the context through a pointer), MSVC /O2 /Ob1, i9-12900K, best of 9 runs, ns per instruction:

main first revision 27b2749
VADD/VSUB/VMUL 0.16 1.15 0.59
ADD.S/SUB.S/MUL.S 0.14 0.80 0.34
DIV.S/RSQRT.S 0.92 2.2 1.21

So the first revision's vector clamp cost about 1 ns per vector instruction. The new version does not clamp the operands at all. It runs the host operation, tests the result once for exponent 255, and only then goes to an out-of-line double path, which brings the vector cost down to about 0.4 ns over main. Of that, about 0.17 ns is flushing exponent-0 results to zero; I can drop the flush from the vector path if you would rather have the speed. This is a micro-benchmark with independent instructions, so it shows the worst case; I have not measured a whole game with it.

Review feedback on the first revision: it treated exponent 255 as an
overflow and clamped operands and results to 0x7F7FFFFF, which is the
largest IEEE float, not the largest PS2 float. On the PS2 exponent 255
is an ordinary exponent: 0x7F800000 is 2^128 and the maximum is
0x7FFFFFFF.

- Operands with exponent 255 are no longer clamped. Whatever touches
  them is computed in double and converted back, so 2^127 * 2 is 2^128
  and only a result of 2^129 or more saturates, to +-0x7FFFFFFF.
- A zero divisor gives +-0x7FFFFFFF. An exponent-0 divisor counts as
  zero whatever its mantissa holds.
- RSQRT.S with a zero ft takes the sign of fs alone; the first revision
  used fs xor ft, which is the DIV.S rule. SQRT.S of -0 is +0.
- Results below 2^-126 become +-0.
- ps2_fclamp, ps2_fmax_signed and ps2_vclamp are gone. The inline part
  of each helper is the host operation plus one range check on the
  result; the double path is out of line. PS2_VADD/VSUB/VMUL test the
  result once per vector instead of clamping both operands and the
  result.

Not emulated: the hardware's rounding of the last mantissa bit, and an
ADD.S/SUB.S/MUL.S or vector operand with exponent 0 and a non-zero
mantissa (it keeps its tiny host value instead of reading as zero).

Tests: the 174 hardware results for add, sub, mul, div, sqrt and rsqrt
from ps2autotests (tests/cpu/ee_fpu) are now a table in the test suite.
163 match bit for bit, 10 differ in the last mantissa bit and are marked
as such, 1 is the exponent-0 operand case above. ps2x_tests: 444/444.

Made by drakolord and assisted with Claude Code.
@ran-j
ran-j merged commit eb8688b into ran-j:main Oct 14, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants