Skip to content

x86: check OSXSAVE and XCR0 before enabling AVX2 (fixes #475) - #490

Open
Alb3e3 wants to merge 1 commit into
xiph:mainfrom
Alb3e3:avx-osxsave-check
Open

x86: check OSXSAVE and XCR0 before enabling AVX2 (fixes #475)#490
Alb3e3 wants to merge 1 commit into
xiph:mainfrom
Alb3e3:avx-osxsave-check

Conversation

@Alb3e3

@Alb3e3 Alb3e3 commented Jul 28, 2026

Copy link
Copy Markdown

Fixes #475.

The problem

opus_cpu_feature_check() sets HW_AVX2 from CPUID leaf 1 (AVX, FMA) and leaf 7 (AVX2) alone:

cpu_feature->HW_AVX2 = (info[2] & (1 << 28)) != 0 && (info[2] & (1 << 12)) != 0;
if (cpu_feature->HW_AVX2 && nIds >= 7) {
    cpuid(info, 7);
    cpu_feature->HW_AVX2 = cpu_feature->HW_AVX2 && (info[1] & (1 << 5)) != 0;
}

CPUID advertising AVX/AVX2 is not sufficient to use them. AVX instructions fault with #UD unless the OS has enabled the YMM register state — that is, CR4.OSXSAVE is set (mirrored in CPUID.1:ECX.27) and XCR0 reports both the SSE (bit 1) and AVX (bit 2) states as enabled. When AVX is disabled in the OS or hypervisor, CPUID still reports the features, opus selects the AVX2 code path, and the first AVX2 instruction crashes — exactly the report in #475.

The fix

Add the standard XGETBV step to the detection sequence: keep HW_AVX2 only if OSXSAVE is set and XCR0 & 0x6 == 0x6. OSXSAVE is checked first, because executing xgetbv is only safe once it is set. A small xgetbv_low() helper is added next to cpuid()_xgetbv() on MSVC, an inline-xgetbv asm on the other paths — returning the low 32 bits of the register, which is all the SSE/AVX state bits need.

The link target and the SSE/SSE2/SSE4.1 paths are unchanged.

Verification

Built with -DOPUS_X86_MAY_HAVE_AVX2=ON (runtime dispatch). The x86cpu translation unit compiles without warnings, and opus_select_arch() still returns the AVX2 tier (4) on this machine, where OSXSAVE is set and XCR0 = 0x2e7 (YMM enabled):

AVX=1 FMA=1 OSXSAVE=1
XCR0=0x2e7 YMM_enabled=1
opus_select_arch = 4

I was able to confirm the positive path (AVX2 stays enabled when the OS has enabled YMM state) but not the negative path directly, since I do not have a host where AVX is CPUID-visible yet OS-disabled. The OSXSAVE/XCR0 sequence is the architecturally defined check Intel/AMD document for exactly that case; I'd appreciate a review from anyone who can exercise an AVX-disabled configuration. The MSVC _xgetbv path is compile-checked by inspection only — I built with GCC.

AI tool disclosure

Prepared with AI assistance; I confirmed the detection gap, wrote and built the change, and verified the positive path locally, and can answer questions during review. The commit carries an Assisted-by: trailer.

opus_cpu_feature_check() enabled the AVX2 code path whenever CPUID
reported AVX, FMA and AVX2 support, without checking that the operating
system has enabled the YMM register state. On a CPU where AVX is
disabled by the OS or hypervisor, CPUID still advertises the features
but executing AVX instructions faults, so opus could crash (issue xiph#475).

Follow the standard detection sequence: require CPUID.1:ECX.OSXSAVE and,
only then, that XCR0 reports both the SSE (XMM) and AVX (YMM) states as
enabled before keeping HW_AVX2 set. A xgetbv helper is added alongside
cpuid for the MSVC and inline-asm paths.

Assisted-by: Claude Code (Claude Opus 5)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

opus_cpu_feature_check doesn't check XState

1 participant