Feature request: Gyro sensor

[nb-programmer]

Is your feature request related to a problem? Please describe.
I want the client to forward Gyro (or IMU) sensor data to the controller server to allow the use of motion and orientation (Just like DualShock).

Describe the solution you'd like
XInput doesn't support Motion input, but I think it can be mapped to an analog stick channel, or simulate a second controller for gyro (like DS4Windows)

Describe alternatives you've considered
DS4Windows with a DualShock controller. Not feasible with VirtualGamePad currently.

Additional context
N/A

[pschatzmann]

I second this — this would be a really cool feature!

[mickaelistria]

FWIW, kitswas/VirtualGamePad#16 was opened with an attempt to explain a user-story for this feature.

[mickaelistria]

kitswas/VGP_Data_Exchange#4 is a first PR towards forwarding pitch/roll from phone to games.

[mickaelistria]

#22 lets the client application capture gyro and send it to server.
kitswas/VirtualGamePad-PC#32 makes the server read the orientation to set extra axes.

[kitswas]

@mickaelistria I appreciate your enthusiasm. Let's not be hasty and think this through first.

Please see the guiding principles: https://kitswas.github.io/VirtualGamePad/roadmap/#:~:text=Every,performance

The changes you made are unlikely to be accepted as is. You are free to keep a modified fork of the project for your own uses, if you urgently need it.

[kitswas]

The data exchange format should not be modified unless absolutely necessary.
Reasons:

1. Performance (it lies on the hot-code path)
2. Compatibility (both server and client must be updated. do the new features work on both Windows and Linux?)

[mickaelistria]

You are free to keep a modified fork of the project for your own uses, if you urgently need it.

I don't mind using only my fork, I just need something to work for my cases (helping kids design specialized game controlers from real world objects) and don't have a strong interest in this code being merged upstream, although I believe it would in the best interest of this project; but I and am happy to share that there is an existing solution for it that anyone is free to pick.

Compatibility

About that data exchange format, I realized that the underlying marshaling API (colfer) is poor when it comes to backward compatibility in kitswas/VirtualGamePad#17 . This could relatively easily be avoided by removing the check for if (header != 127) return 0 and make the unmarshaler much more tolerant and thus more compatible and the protocol more capable of supporting new stories.

both server and client must be updated

That's already done, and I tested it using SNES' Mario Kart (under Snes9x emulator) on the phone similarly to using a wiimote as a wheel with good results.

do the new features work on both Windows and Linux?

As mentioned on the PR, orientation is only added to Linux with gamepads/evdev as 2 new axes. I have not worked on keyboard support at this moment as I have little need for it, and found it much less straightforward to use a keymapping than to "just" define Evdev axes.
Maybe I'll have a use-case for keyboard one day and will work on it...

Performance

Note that adding new elements to the protocol and supporting them on server-side has no impact on performance: they only make the server capable of receiving the data, but as long no such data is sent, the behavior is not changed. Only adding them on client side without server supporting cause compatibility breakage for the reason described above and can result in (IMO neglectable) 64 more bits transferred every 80ms.

So looking at the doc you shared, and sticking to the technical bits, I think that adding support inside the protocol and server meets your technical requirements already. Only changing the Mobile client has an impact on compatibility and performance.

[kitswas]

Adding a setting to the client to allow either thumbstick to be controlled by the sensor would be a lot cleaner instead of trying to add new axes. No changes would be required to the server or the exchange format.

This is in line with the initial suggestion in the issue.

XInput doesn't support Motion input, but I think it can be mapped to an analog stick channel

Does it address your use case? @mickaelistria

[mickaelistria]

Yes, I think this would work for my use-case.