[BUG] Dropped messages when large numbers of messages are received.

[m-komo]

Describe the bug
Dropped messages were observed when monitoring thousands of messages from a USB High-speed device.
It could not be reproduced with the ProtoZOA (Full-speed device).

To Reproduce

1. Attach the USB high-speed MIDI 2.0 device which just sends back all received messages.
2. Update the driver to the USB MIDI 2.0 driver (UsbMidi2.sys).
3. Open midi.exe in two windows.
4. From one, monitor the attached device.
5. From the other, run 'midi ep send 0x40123456 0x00000000 -p 0 -i -c 10000'.

Sending messages completed in a short period, but some dropped messages were observed on the monitor.

Expected behavior
All messages are received and displayed on the monitor.

Screenshots

I have attached the captured USB log (USBLog.zip).

Installer Name or Version

• Windows.MIDI.Services.In-Box.Service.-.Developer.Preview.6.1.0.24194.2233-x64.exe

Desktop (please complete the following information):

• OS: Windows 11 24H2
• OS Build: 26100.1150

Device information, if this is with an external MIDI device:

• Any USB high-speed MIDI 2.0 device with the echo back functionality.
• Driver: [interim preview build] 240713-USBMidi2-x86-TESTSIGNED.zip (DriverVer = 07/13/2024,23.1.50.504)

Application Information

• 'midi.exe' console app.

[Psychlist1972]

Am I understanding your report correctly that all 10,000 messages were sent to the hardware and then received back from the hardware? Or are you showing that message number 10,000 was received? If you are just looking to see the last message was sent/received, it's possible that messages in the middle were not sent or received. We'd need to see an actual count to be sure.

[Psychlist1972]

Note that I cannot currently test this, as I do not have any MIDI 2.0 UMP USB High Speed devices and there are none on the market, so I'll need to rely on @m-komo for any further testing.

[m-komo]

Am I understanding your report correctly that all 10,000 messages were sent to the hardware and then received back from the hardware? Or are you showing that message number 10,000 was received? If you are just looking to see the last message was sent/received, it's possible that messages in the middle were not sent or received. We'd need to see an actual count to be sure.

I confirmed that there are no dropped messages on the USB transaction.
It means that all 10,000 messages were sent to the hardware and then received back from the hardware.
So, it may cause on the receive path.

I'll do further investigation.
Is there any way to know an occurrence of overflow on cross process buffers between the driver and service, and between service and app?

[AmeNote-Michael]

Note that new model implemented, if there is no room in the receive buffer (ie not serviced fast enough) we drop message.

I am looking into how to give useful messages in logs to get more information. Cannot put log message for every dropped message, that would just cause too many messages and not be useful.

I will also look to trying on a STM32 device I have on hand which gives me High Speed.

[dmiura]

@Psychlist1972

I do not have any MIDI 2.0 UMP USB High Speed devices

Montage M v2.0 is USB MIDI 2.0 High Speed device, but I'm not sure if Montage M can loop-back from In to Out...

[m-komo]

@Psychlist1972
I am not sure if it is exactly the same issue, but I was able to reproduce the same symptom with the combination of UM-ONE and USB MIDI 1.0 driver (USBAUDIO.sys).

To Reproduce:

1. Set the Roland UM-ONE mk2 to class-compliant mode and short INPUPT and OUTPUT to loop back messages.
2. Attach the UM-ONE to PC.
3. Make sure the device is working with the USB MIDI 1.0 driver (USBAUDIO.sys).
4. Open midi.exe in two windows.
5. From one, monitor UM-ONE.
6. From the other, run 'midi ep send 0x20904040 -p 0 -c 10000'.

I confirmed that there are no dropped messages on the USB transaction.

In addition, if UM-ONE is working with the USB MIDI 2.0 driver (UsbMidi2.sys), I could not see any dropped messages.

[Psychlist1972]

FWIW, this is what I see UM-One,

and with 50k messages

Ahh. But I see now that even though it is in class-compliant mode, the Roland driver was picked up. I'll need to try with the in-box driver.

So I just tried with the iConnectivity mio, which is using our older class driver (all the service code is the same for talking with the roland driver and with the older class driver. The only way to run different service code is to use the MIDI 2 driver) and I see it did drop a bunch of messages.

It stops almost immediately after the first console (the sender) completes. I've run it several times now, and the same thing happens each time.

In case your numbers look different from mine, note that I've made some changes recently that may impact that:

• Removed MMCSS for testing (so incoming messages take slightly longer to receive)
• Increased incoming message buffer sizes from PAGE_SIZE to PAGE_SIZE * 2

This appears to be a new failure. I'll investigate further. I'm running some traces now.

Thanks.

[Psychlist1972]

Here are my findings so far:

• Testing with UM-One with Roland driver: test sending 500 messages as fast as possible works (success, and is a bit faster than our in-box driver as well)
• Testing with mio with in-box MIDI 1.0 driver: test sending 500 messages as fast as possible results in receiving only 150 or so (fail)
• Testing with mio with in-box MIDI 1.0 driver, but add 1ms delay between outgoing messages: test works. (qualified success)

I've verified that it's not a problem with a device being closed too early by the service, or anything similar to that. I've also verified that all 500 messages are sent to the driver.

So from this initial testing, it appears that some devices, or our in-box MIDI 1.0 driver, may just not be able to handle the messages at the rate we're sending. I'm looking to see if there's any way to detect that.

I'm still investigating, as some things here don't quite make sense.

[MusicMaker]

FWIW, for example using a MIDIMAN 2x2 and DIN loop back OUT A to IN A by a cable, sending lot of messages, causes drops at the (USB) receiving end . When connecting OUT A to IN B, no drops occur. so that is definitely an issue with the interface and it's code structure. i tested it also with ALSA and QSYNTH (GM2 soundfount) loopback playing a big MIDI file. (is there any windows GM sound generator equivalent ?) . Do wonder if the UM-1 can handle large and fast data. Assume this shorting pin is the pin from MIDI In/OUT? For single DIN interfaces, used a Focusrite Audio interface by connecting OUT to IN and no drops occurred sending data to and receiving data back via USB.

[m-komo]

@MusicMaker Thank you for your comment.

Do wonder if the UM-1 can handle large and fast data.

I captured USB log and confirmed that there are no dropped messages on the USB transaction.
It means that all 10,000 messages were sent to the UM-ONE and then received back from the UM-ONE.

[sat-okada]

I have been able to reproduce this issue on my USB High-speed device with USB MIDI 2.0 driver(UsbMidi2.sys).

I couldn't reproduce 'midi ep send 0x40123456 0x00000000 -p 0 -i -c 10000' with my equipment,
so I used 'midi ep send 0x40123456 0x00000000 -p 0 -i -c 40000'.

I have confirmed that no messages are dropped on USB transaction.

My Windows version is
Windows 11 Pro Insider Preview Build 26257.ge_prerelease.240719-1747

My driver version is as follows.
Driver Provider : USBMidi2
Driver Date : 2024-07-19
Driver Version : 23.22.46.475

[sat-okada]

FYI, I ran a similar test on MacOS Sonoma, but it didn't reproduce.

[MusicMaker]

FWIW . Configured a high speed MPU based legacy device running a Linux kernel 6.1 and MIDI as a device. Connected it via USB to a PC and loaded MIDI2 (DP 2) driver in Windows. Sending messages with 0 delay goes fine but up to around 1200 and start to fail at around 1500. Adding a delay of 1 msec for MIDI2.0 the result is good even up to 10000 . That transfer took 2 min 40 seconds. is, Used arecordmidi -n 10000 . This tool will stop receiving once the legacy MIDI event count reaches 10000 so if it did not, should point to missed messages or other issues. I don't have a newer USB driver to test as others seem to have. Using Dev preview 6.
https://www.youtube.com/watch?v=KABgVAecUhU&list=PLroX0v1Yee0nQW28Ju4chUXSjK9tvagIH&index=15&pp=gAQBiAQB
https://www.youtube.com/watch?v=v4eZ0EMHOO4&list=PLroX0v1Yee0nQW28Ju4chUXSjK9tvagIH&index=16&pp=gAQBiAQB