x86: check OSXSAVE and XCR0 before enabling AVX2 (fixes #475) - #490
Open
Alb3e3 wants to merge 1 commit into
Open
x86: check OSXSAVE and XCR0 before enabling AVX2 (fixes #475)#490Alb3e3 wants to merge 1 commit into
Alb3e3 wants to merge 1 commit into
Conversation
opus_cpu_feature_check() enabled the AVX2 code path whenever CPUID reported AVX, FMA and AVX2 support, without checking that the operating system has enabled the YMM register state. On a CPU where AVX is disabled by the OS or hypervisor, CPUID still advertises the features but executing AVX instructions faults, so opus could crash (issue xiph#475). Follow the standard detection sequence: require CPUID.1:ECX.OSXSAVE and, only then, that XCR0 reports both the SSE (XMM) and AVX (YMM) states as enabled before keeping HW_AVX2 set. A xgetbv helper is added alongside cpuid for the MSVC and inline-asm paths. Assisted-by: Claude Code (Claude Opus 5)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #475.
The problem
opus_cpu_feature_check()setsHW_AVX2from CPUID leaf 1 (AVX,FMA) and leaf 7 (AVX2) alone:CPUID advertising AVX/AVX2 is not sufficient to use them. AVX instructions fault with #UD unless the OS has enabled the YMM register state — that is,
CR4.OSXSAVEis set (mirrored inCPUID.1:ECX.27) andXCR0reports both the SSE (bit 1) and AVX (bit 2) states as enabled. When AVX is disabled in the OS or hypervisor, CPUID still reports the features, opus selects the AVX2 code path, and the first AVX2 instruction crashes — exactly the report in #475.The fix
Add the standard
XGETBVstep to the detection sequence: keepHW_AVX2only ifOSXSAVEis set andXCR0 & 0x6 == 0x6.OSXSAVEis checked first, because executingxgetbvis only safe once it is set. A smallxgetbv_low()helper is added next tocpuid()—_xgetbv()on MSVC, an inline-xgetbvasm on the other paths — returning the low 32 bits of the register, which is all the SSE/AVX state bits need.The link target and the SSE/SSE2/SSE4.1 paths are unchanged.
Verification
Built with
-DOPUS_X86_MAY_HAVE_AVX2=ON(runtime dispatch). The x86cpu translation unit compiles without warnings, andopus_select_arch()still returns the AVX2 tier (4) on this machine, whereOSXSAVEis set andXCR0 = 0x2e7(YMM enabled):I was able to confirm the positive path (AVX2 stays enabled when the OS has enabled YMM state) but not the negative path directly, since I do not have a host where AVX is CPUID-visible yet OS-disabled. The
OSXSAVE/XCR0sequence is the architecturally defined check Intel/AMD document for exactly that case; I'd appreciate a review from anyone who can exercise an AVX-disabled configuration. The MSVC_xgetbvpath is compile-checked by inspection only — I built with GCC.AI tool disclosure
Prepared with AI assistance; I confirmed the detection gap, wrote and built the change, and verified the positive path locally, and can answer questions during review. The commit carries an
Assisted-by:trailer.