Skip to content

AVX-VNNI-INT(8/16) support - #6058

Open
pthariensflame wants to merge 2 commits into
FEX-Emu:mainfrom
pthariensflame:avx-vnni-int8-int16
Open

pthariensflame wants to merge 2 commits into
FEX-Emu:mainfrom
pthariensflame:avx-vnni-int8-int16

Conversation

@pthariensflame

@pthariensflame pthariensflame commented Oct 23, 2026 •

Copy link
Copy Markdown

This builds on the recently-added AVX-VNNI support and adds the rest of the combinations of signednesses; there are no tests yet because we were unable to build FEX locally or get our IDE to cooperate on understanding the codebase, so this code is only maybe correct (or even compilable). Advice for how to proceed would be wonderful.

No LLMs were used in any capacity during any part of creating this code; this is good old-fashioned copy-pasting and modifying the copies, with a sprinkle of basic understanding of how the A64 ISA works and a fair amount of staring at the Intel ASDM.

EDIT: We can now build and test FEX, and have updated the tests!

@pthariensflame
pthariensflame marked this pull request as draft October 23, 2026 06:56
@Sonicadvance1

Copy link
Copy Markdown
Member

Oh my.

@pthariensflame
pthariensflame force-pushed the avx-vnni-int8-int16 branch 3 times, most recently from 6559052 to d3efefe Compare October 23, 2026 07:10
@pthariensflame

Copy link
Copy Markdown
Author

Should be "ready" for review at this point, inasmuch as untested code can be.

@Sonicadvance1

Copy link
Copy Markdown
Member

I'm contemplating what I need to do to get this unittested on real hardware. I want to write a bunch of unittests before this merges directly. Trying to find the best way forward.

@pthariensflame

Copy link
Copy Markdown
Author

Trying to find the best way forward.

There is no rush here, and your efforts are extremely appreciated no matter how long it takes or what needs to happen to test this. Thank you so much for taking our contribution-for-the-sake-of-a-fun-challenge seriously; we are willing to wait until cores supporting these extensions are much more accessible if that helps take pressure off.

@Sonicadvance1

Copy link
Copy Markdown
Member

I ordered an Intel Core 7 350 device which supports these extensions so I should be able to get to writing unittests next week.

@pthariensflame

pthariensflame commented Oct 23, 2026 •

Copy link
Copy Markdown
Author

an Intel Core 7 350 device

Oooh! Should be good for testing the AVX-IFMA and SHA512 extensions too.

@pthariensflame
pthariensflame force-pushed the avx-vnni-int8-int16 branch 4 times, most recently from 5229366 to 24673d9 Compare October 23, 2026 23:04
@pthariensflame

Copy link
Copy Markdown
Author

It compiles! And it appears to produce reasonable code (at least by inspection)! 🎉

@Sonicadvance1

Copy link
Copy Markdown
Member

Looks like the main thing missing in CI is that you're unconditionally using a FEAT_DotProd instruction so it faults on the "fallback" path.

@pthariensflame

Copy link
Copy Markdown
Author

Looks like the main thing missing in CI is that you're unconditionally using a FEAT_DotProd instruction so it faults on the "fallback" path.

Should be fixed now; we have no idea how to update the unit tests so that they reflect the intended codegen.

@Sonicadvance1

Copy link
Copy Markdown
Member

For instcountci it's best to compile with cmake options -DENABLE_VIXL_DISASSEMBLER=True -DENABLE_VIXL_SIMULATOR=True
Modify FEXCore/Source/Interface/Core/DiskCache.h to update the FormatVersion variable by incrementing it by one.
Then running ninja && ninja instcountci_tests ; ninja instcountci_update_tests
Then committing with git add -u ; git commit -m "InstcountCI: Update"

@pthariensflame
pthariensflame force-pushed the avx-vnni-int8-int16 branch 2 times, most recently from f162082 to 7fa486a Compare October 25, 2026 21:53
@pthariensflame

pthariensflame commented Oct 25, 2026 •

Copy link
Copy Markdown
Author

InstcountCI has been updated! Seems like we may want DOTPROD and I8MM versions of the VEX_map2 VNNI-INT(8/16) tests, since those aren't there right now. Should we try to add them?

EDIT: We will try to add them!

@pthariensflame

pthariensflame commented Oct 26, 2026 •

Copy link
Copy Markdown
Author

How exactly does InstCountCI succeed on x86_64 but fail on AArch64 when our local testing took place on AArch64 Linux???

@pthariensflame
pthariensflame marked this pull request as ready for review October 26, 2026 00:42
@pthariensflame
pthariensflame force-pushed the avx-vnni-int8-int16 branch 5 times, most recently from 308e70a to bafb664 Compare October 31, 2026 09:27
@pthariensflame
pthariensflame force-pushed the avx-vnni-int8-int16 branch 6 times, most recently from 88df3dd to 1629ff9 Compare November 3, 2026 04:14
@pthariensflame

Copy link
Copy Markdown
Author

Hooray! Only one specific failure left in the tests; investigating.

@pthariensflame
pthariensflame force-pushed the avx-vnni-int8-int16 branch 3 times, most recently from 726e9df to 110a33d Compare November 3, 2026 05:07
Comment on lines +5556 to +5562
"zip1 z2.s, z0.s, z1.s",
"umullb z0.s, z17.h, z18.h",
"umullt z1.s, z17.h, z18.h",
"zip2 z3.s, z0.s, z1.s",
"addp z2.s, p7/m, z2.s, z3.s",
"uzp2 z1.s, z2.s, z2.s",
"uzp1 z2.s, z2.s, z2.s",

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This kind of thing, with the repeated zipping and unzipping, should really be able to be optimized away somehow. We have no idea how to go about that though.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants