[Bug]: HEVC/AV1 decode crashes driver

[KulkovVadim]

Describe the bug
AMF SimpleDecoder has driver timeout for some HEVC/H.265 and AV1 stream.

So I have h265 stream (Main, yuv420p(tv), 1920x1072 [SAR 1:1 DAR 120:67], 23.98 fps, 23.98 tbr, 1200k tbn), it works well on OneVPL, NVDec, ffmpeg, plays correctly on VLC, etc. When i trying to decode straem with SimpleDecoder at 15-th frame on call decoder->SubmitInput(data), i got exception thrown at amfrtdrv64.dll. (AMF 1.4.33, runtime 1.4.31.0)

Testing on AMD Ryzen 5 4500U shows that the problem starts with driver version 23.7.1, in previous driver versions the decoding process was completed without any exceptions, but frame defects were observed as a result of data decoding.

In my personal application, I received the same exception in AV1 stream on AMD Ryzen 7 7700X in multiple files.

To Reproduce
Steps to reproduce the behavior:

1. Build SimpleDecoder
2. Run with H.265/HEVC file
3. Submit input frame and see exception (amfrtdrv64.dll/amfrtdrv32.dll)

Setup (please complete the following information):

• OS: Windows 11 23H2;
• Driver Version 24.3.1;
• AMD Ryzen 5 4500U with Radeon Graphics, AMD Ryzen 7 7700X;
• Decoder.

Debug Log (please upload or paste):

358C [AMFDecoderUVDImpl]   Debug: AMFDecoderUVDImpl::Init(NV12, 1920, 1072)
358C [AMFDecoderUVDImpl]    Info: InitDecoderFormat() - Decoder stream format NV12
358C [AMFVideoStreamParserImpl]   Debug: AMFVideoStreamParserImpl::Init(NV12, 1920, 1072)
358C [AMFVideoStreamParserImpl]   Debug: AMFVideoStreamParserImpl::Terminate()
358C [AMFDecodeEngineImplDX]    Info: Init() OK! Video DX HW DECODER (1920x1072) for input stream 
358C [AMFDecodeEngineImplDX12]    Info: CreateDecoder() Init OK! Video HW VIDEO DECODER DX12 (1920x1072) for input stream; profile:UVD H265 Unencrypted Config Profile
3EC0 [AMFDecoderUVDImpl]   Debug: AMFDecoderUVDImpl::QueryOutput()
358C [AMFDecoderUVDImpl]   Debug: AMFDecoderUVDImpl::SubmitInput()
358C [AMFVideoStreamParserImpl]   Debug: AMFVideoStreamParserImpl::SubmitInput()
3EC0 [AMFDecoderUVDImpl]   Debug: AMFDecoderUVDImpl::QueryOutput()
358C [AMFVideoStreamParserImpl]   Debug: AMFVideoStreamParserImpl::QueryOutput()
...
358C [AMFVideoStreamParserImpl]   Debug: AMFVideoStreamParserImpl::SubmitInput()
358C [AMFVideoStreamParserImpl]   Debug: AMFVideoStreamParserImpl::QueryOutput()
Exception thrown at 0x00007FFA33A16AAA (amfrtdrv64.dll) в SimpleDecoder.exe: 0xC0000005: Access violation writing location 0x0000000F86472340.

Expected behavior
Receive an error message about the reasons why the stream cannot decode, so that you can process the error at the top level and prevent the application from crashing.

If you have any list of criteria or features of files that cause a driver error, please describe them in order to independently limit the ability to decode such files.

Screenshots
If applicable, add screenshots to help explain your problem.

Additional context
H.265/HEVC stream google drive

[DimkaTsv]

That is some pretty weird source file (don't misunderstand, there is issue with some rare occurence when decoding of the specific corrupted sources crashing driver on RDNA3 GPU's, but your problem isn't corrupted source, neither you use RDNA3).

I tried your source, and for me driver doesn't crash. And it does decode in players (aka it... plays, with proper image output and so on.). But it works differently for each player i tried.
On MPC-HC is unable to seek and skip, as well as fast forward and slowdown do not change playback speed but corrupt stream instead.
On Pot Player i can seek, but image will freeze for few seconds until it hits next keyframe. Windows player just denied me from opening this file as unsupported.
Heck, even ffmpeg was unable to carve out fragment from this stream due to inability to perform seek.

Just checked. This is not just normal H265 stream, it is one likely created with RTP stream properties. It has particular slice tags (IDR_W_RADL [IDR-slice] // TRAIL_R [P-slice]). Meaning it was supposed to be streamed directly to something.

But, if it helps with further debug, here is transcode error from VCEEnc
vceencc_log.txt

[hevc @ 000002757d76c640] nal_unit_type: 33(SPS), nuh_layer_id: 0, temporal_id: 0
[AMFVideoStreamParserImpl] 2024-05-29 03:13:27.031     69B8 [AMFVideoStreamParserImpl]   Debug: AMFVideoStreamParserImpl::QueryOutput()
[hevc @ 000002757d76c640] nal_unit_type: 34(PPS), nuh_layer_id: 0, temporal_id: 0
[AMFDecoderUVDImpl] 2024-05-29 03:13:27.031     69B8 [AMFDecoderUVDImpl]   Debug: AMFDecoderUVDImpl::SubmitInput()
[hevc @ 000002757d76c640] nal_unit_type: 19(IDR_W_RADL), nuh_layer_id: 0, temporal_id: 0
[AMFVideoStreamParserImpl] 2024-05-29 03:13:27.031     69B8 [AMFVideoStreamParserImpl]   Debug: AMFVideoStreamParserImpl::SubmitInput()
[AMFDecoderUVDImpl] 2024-05-29 03:13:27.032     225C [AMFDecoderUVDImpl]   Debug: AMFDecoderUVDImpl::QueryOutput()
[AMFVideoStreamParserImpl] 2024-05-29 03:13:27.032     69B8 [AMFVideoStreamParserImpl]   Debug: AMFVideoStreamParserImpl::QueryOutput()
AMFDEC: ERROR: Unexpected error while submitting bitstream to decoder.
[hevc @ 000002757d76c640] Decoding VPS
[hevc @ 000002757d76c640] Main profile bitstream
AMFDEC: Failed to load input frame: unknown error..
[hevc @ 000002757d76c640] Decoding SPS
Break in task AMFDEC: unknown error..
[hevc @ 000002757d76c640] Main profile bitstream
Clear vpp filters...
[hevc @ 000002757d76c640] Decoding VUI
Closing m_pmfxDEC/ENC/VPP...
[hevc @ 000002757d76c640] Decoding PPS
Closing Encoder...
[NULL @ 000002757d72ecc0] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0, temporal_id: 0
[AMFEncoderCoreHevc] 2024-05-29 03:13:27.037     225C [AMFEncoderCoreHevc]   Debug: AMFEncoderCoreHevcImpl::Terminate()

I suppose there should be something with those slice types?

[rhutsAMD]

I have reproduced the crash and created an internal ticket to the decoder folks to address this issue.

[KulkovVadim]

Thank for feedback. I am attaching AV1 files with the same problem with driver error for sending frames.
The Beach-Yoga-AV1.webm file opens and decodes the frames, however, seek by frame thrown exception in amdxx32.dll. AMD Ryzen 7 7700X.

Exception thrown at 0x2383F4B4 (amdxx32.dll) in app.exe: 0xC0000094: Integer division by zero.
Unhandled exception at 0x2383F4B4 (amdxx32.dll) in app.exe: 0xC0000094: Integer division by zero.

AV1 files (google drive)

[rhutsAMD]

The issue has been fixed internally and the fix will be available in a future public driver release.
We will notify when the release is made publicly available.