Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mac-avcapture: Use list-based format selector #10895

Open
wants to merge 5 commits into
base: master
Choose a base branch
from

Conversation

jcm93
Copy link

@jcm93 jcm93 commented Jun 19, 2024

Description

Updates the properties window for macOS mac-avcapture sources to use a unified list-based format selector when in the non-preset mode. Also adds a contextual warning in the properties window to show when a device has macOS system effects active.

Screenshot 2024-06-19 at 11 07 00 AM Screenshot 2024-06-19 at 11 07 27 AM

The frame rate selector is retained, so that we can continue to have explicit control over the device frame rate. The list selectors for resolution, color space, pixel format and video range, however, have all been consolidated inside the single format selector.

Motivation and Context

#9344 provided a much-needed update to the Video Capture Device source on macOS and added a lower-latency Capture Card Device source. A number of factors, however, have conspired to make the new version of the source harder to use, for both novice and adept users:

  • The new version does not reimplement the 'auto-populate parameters' feature of the source on other platforms. When a property is updated, the user must check all other properties to ensure that the combination of selected parameters is still a valid configuration, which is not always immediately apparent.
  • Because of "quirks" with OBS's frame rate selector and the default size for the source properties window, it is not apparent to many users that the list selectors for pixel format, video range and color space exist 'below the fold' in the source properties window.

This PR aims to simplify and improve the UI for the source while also reducing the complexity of the property selection code.

  • All parameters are now visible in the source properties window at its default size. The only parameter that can be invalid after changing the selected format is the frame rate, and its status should be more apparent within the context of the window.
  • It is easier to understand at a glance what combinations of parameters are valid for a given device. Making an appropriate selection should now be a quicker process.

Additionally, the new effects warning in the properties window should help to alleviate some user confusion. On macOS, enabling an effect such as Studio Light or Portrait mode for one video input will often enable them on all inputs (filed as FB13302833). Users are often confused to see webcam effects on e.g. capture card inputs. With this PR, the source properties window displays a warning for active system effects, which should make this issue more apparent.

How Has This Been Tested?

Tested locally on an M1 Max machine running the macOS Sequoia beta with all video capture devices on hand, including the built-in webcam, Continuity Camera from an iPhone, a virtual camera, and a single capture card from a popular brand, to ensure both OBS source types remained in good working order.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • Tweak (non-breaking change to improve existing functionality)
  • Performance enhancement (non-breaking change which improves efficiency)
  • Code cleanup (non-breaking change which makes code smaller or more readable)

Checklist:

  • My code has been run through clang-format.
  • I have read the contributing document.
  • My code is not on the master branch.
  • The code has been tested.
  • All commit messages are properly formatted and commits squashed where appropriate.
  • I have included updates to all appropriate documentation.

@WizardCM WizardCM added Bug Fix Non-breaking change which fixes an issue Enhancement Improvement to existing functionality macOS Affects macOS labels Jun 20, 2024
@derrod derrod self-requested a review June 22, 2024 17:53
/// Compare two `AVCaptureDeviceFormat`s for purposes of sorting in the properties window.
static NSComparator const OBSAVCaptureDeviceFormatCompare =
^NSComparisonResult(AVCaptureDeviceFormat *a, AVCaptureDeviceFormat *b) {
CMVideoDimensions aDimensions = CMVideoFormatDescriptionGetDimensions(a.formatDescription);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you tried calculating a "bandwidth" for each format (which would necessarily include resolution, number of planes, size per plane, colour depth, etc.) so you get a nice single numerical value representing a format and then you can just compare the value of A and B.

Also the default names for custom comparators in Swift is lhs and rhs, which I'd prefer here as well.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you tried calculating a "bandwidth" for each format (which would necessarily include resolution, number of planes, size per plane, colour depth, etc.) so you get a nice single numerical value representing a format and then you can just compare the value of A and B.

This seems like a nice idea, I'll just have to look into the specifics. Adjusted argument names for now.

Copy link
Author

@jcm93 jcm93 Jun 28, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a thought here so I don't forget; the sort order may also want to cleanly delineate aspect ratio in some way at the top level; i.e. all 16:9 formats separated from all 4:3 formats, etc; all landscape ratios separate from all portrait and square ratios that might exist too.

Copy link
Member

@PatTheMav PatTheMav left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some code-specific comments, overall I'd like to see more discussion about:

  1. Presentation of the formats in the dropdown (@Warchamp7 got some ideas there?)
  2. How to "upgrade" existing source configurations

@jcm93 jcm93 force-pushed the avcapturelist branch 4 times, most recently from de521d2 to af5e538 Compare June 27, 2024 22:16
@jcm93
Copy link
Author

jcm93 commented Jun 28, 2024

Couple misc changes:

  • Fixed the reaction effects property issue
  • We were checking if the received buffers differed from the configured format by checking the video range; it seemed simpler to check the media subtype (fourCC) instead, since that will detect if (for whatever reason) the pixel format changes but not the range.

Secondly, added a migration routine in configureSession to attempt to configure with the legacy properties if there is no value for the new supported_format key. This seems to me like the only thing we need to do to cleanly migrate from the old properties. All new sources will contain just the supported_format key-value, and supported_format will start superseding the old properties for existing sources once the properties window for an existing source is opened. Tested with a couple sources to make sure migration worked.

Hopefully this should be in a pretty good state now.

Was intending to squash at the end, just let know if you want me to squash whenever.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Fix Non-breaking change which fixes an issue Enhancement Improvement to existing functionality macOS Affects macOS
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants