getUserMedia documentation

[edolix]

Description

I'd like to open the discussion and share some info about device permissions and other browser-related things.

Here a list of thoughts we can discuss to build the proper UX workflow:

• Device picker
• Let the user access the App without permissions
• Let the user upgrade/downgrade devices while in call
  • In call with only mic permission and wants to upgrade to video-call so have to ask him video perms
• Default device
  • What happens if the user's choice has been unplugged. Cantina should fallback to the browser default (?)
  • What happens in case of no devices. The user could still join but without video/audio with a notification (?)

Please - add whatever idea in the list above!

Fingerprinting

All browsers are following the specification that give us the list of devices without deviceId, label and groupId if the user has never allowed (or denied) the App to access his devices.
This means we must have the perms to display a picker to choice the camera/microphone or speaker.

For camera and microphone devices, if the browsing context did not capture (i.e. getUserMedia() was not called or never resolved successfully), the MediaDeviceInfo object will contain a valid value for kind but empty strings for deviceId, label, and groupId. Additionally, at most one device of each kind will be listed in enumerateDevices() result.

To be clear: we can ask for audio and video separately so the user is able to allow "Microphone" but not "Camera". In that case Cantina will not be able to list the webcams.
Instead if we ask for Microphone and Camera (together), the user will see a single popup but denying that will make our App "not usable" until the user update his browser settings manually.

Each browser session generates a random value for deviceId

This is for privacy reason too but - each browser session generates random value for deviceId. Especially Safari presents 3 different behaviours for Mac, iPad or iPhone. It seems iPad/iPhone randomize them on window.location changes while Chrome/Firefox after closing/reopening the browser. 🤷‍♂️
Anyway - the random value means the App cannot trust the deviceId to re-use the same device across user's visits so - the workaround is:

• Look for deviceId if present or..
• Look for device label if present.

Both the above checks work only if the App already have the permissions.

Safari always alert the user with a popup

While Chrome and other normal browsers remember the user's choice asking only the first time - Safari display the popup the first time on every visit (including refresh!).

Only Chromium-based browsers support speaker selection

Firefox and Safari should support these APIs in the future...

No direct API to check for permissions

There is no API to check whether the App has or not the permissions. We have some workaround like filtering devices and see if both deviceId and label are not empty strings but we're always in the browser-hands.

Additional Resources

• https://webkit.org/blog/7763/a-closer-look-into-webrtc/
• https://webrtchacks.com/guide-to-safari-webrtc/
• https://w3c.github.io/mediacapture-main/#access-control-model
• https://bloggeek.me/webrtc-browser-support/

I'm happy to discuss these stuff anytime 👍

/cc @johnmcdowall

[natural411]

this is also interesting read https://www.digitalocean.com/community/tutorials/js-permissions-api

[edolix]

Tracked in https://github.com/signalwire/cloud-product/issues/1652