887A0006 The GPU will not respond to more commands, most likely because of an invalid command passed by the calling application.

[elephantpanda]

GPU: Quadro P5000

pip install onnxruntime-genai-directml

Model here.

This code fails every time on my computer:

import onnxruntime_genai as ai

print("Version = "+str(ai.__version__))
print("DML available="+str(ai.is_dml_available()))

model = ai.Model("D:\Phi3Onnx")

N=0 #change this to N=1 to pass test
print("Doing Test "+str(N))


generatorParams = ai.GeneratorParams(model)
generatorParams.input_ids = [N]*500
generator = ai.Generator(model,generatorParams)

for n in range(5):
    print(n)
    generator.compute_logits()
    generator.generate_next_token()

print("TEST "+str(N)+" PASSED")

With output:

Version = 0.4.0
DML available=True
Doing Test 0
0
1
Traceback (most recent call last):
  File "test.py", line 18, in <module>
    generator.compute_logits()
RuntimeError: D:\a\_work\1\onnxruntime-genai\src\dml\dml_command_recorder.cpp(143)\onnxruntime_genai.cp38-win_amd64.pyd!00007FFA45F309E3: (caller: 00007FFA45F24A25) Exception(1) tid(2384) 887A0006 The GPU will not respond to more commands, most likely because of an invalid command passed by the calling application.

Any hope of this being fixed by Christmas? If not can you just add this code to your suite of tests so that you can catch this bug in the future. Thanks.

(This bug may have already been mentioned but I am writing it much more clearly here so it is easy to reproduce.)