Fall back to software when hardware decoder open fails

[Blackspirits]

Summary

The hardware decoder setup tries VideoToolbox on macOS and D3D11VA/DXVA2 on Windows. Device-creation and transfer-format failures already move to the next candidate, but an avcodec_open2 failure after attaching a hardware device previously aborted the media open.

The first revision made that fail-soft by retrying directly in software. The final revision preserves the full intended Windows fallback chain too:

• D3D11VA open failure → try DXVA2;
• DXVA2 open failure → try software;
• VideoToolbox open failure → try software;
• software avcodec_open2 failure still throws the original decode-open error.

Implementation details:

• OpenDecoder carries the next hardware-array index when retrying;
• each failed codec context is freed before trying the next candidate;
• if no later hardware device can be attached, the same path naturally opens a software codec;
• recursion is bounded by HardwareDeviceTypes.Length.

Validation

• based on upstream c77c4b428649ce7030e6cf218b4d7c384b955232
• diff is limited to FfmpegPlayer.cs
• revised full solution restore passed
• revised full solution build passed
• revised full solution test suite passed on the first run
• UI suite: 5,151 passed, 9 skipped, 0 failed (5,160 total)
• retry step was not used
• validation run: #34780354221

AI assistance: ChatGPT was used to adversarially review the D3D11VA/DXVA2/VideoToolbox decoder-open fallback chain.

[Blackspirits]

Independent adversarial re-check: upstream promises fail-soft hardware decoding, but an avcodec_open2 failure after attaching a hardware device still aborted the decoder. The retry is bounded by hardware:false, frees the failed hardware codec context first, and preserves the original exception if software open also fails. FFmpeg documents hw_device_ctx as owned/freed by libavcodec after assignment, so the cleanup path is correct. Full CI #34778793466 passed restore, build and the complete suite on the first run; UITests reported 5,151 passed, 9 skipped, 0 failed. Retry was not used. No blocker identified. Keep draft; no merge performed.

[Blackspirits]

Independent adversarial re-check of the revised fallback chain: a hardware avcodec_open2 failure now advances to the next platform hardware candidate before software. On Windows this preserves D3D11VA -> DXVA2 -> software; on macOS VideoToolbox -> software. Each failed codec context is freed before recursion, and hardwareStartIndex strictly increases, bounding recursion by HardwareDeviceTypes.Length. If no later hardware device attaches, the same invocation opens software; a software open failure still throws. Full CI #34780354221 passed restore, build and the complete suite on the first run; UITests reported 5,151 passed, 9 skipped, 0 failed. Retry was not used. No blocker identified. Keep draft; no merge performed.