Media Providers Change Proposal

[mihar-22]

👋

I wanted to put something that I've been considering today up for discussion. I've been looking at the boilerplate for our media setup and I realized it looks a little strange now since we lifted up most media state to <vds-media>. In particular, src and a few provider specific APIs are left on the provider element.

<vds-media controls poster="https://media-files.vidstack.io/poster.png" view="video">
  <!-- what do we need this element for anymore? -->
  <vds-video>
    <!-- `src` is the only state left hanging -->
    <video preload="none" src="https://media-files.vidstack.io/720p.mp4"></video>
  </vds-video>
</vds-media>

I was wondering how most would feel if we instead pulled src up to <vds-media> and possibly dropped provider elements like so:

<vds-media 
  src="https://media-files.vidstack.io/720p.mp4" 
  poster="https://media-files.vidstack.io/poster.png"
  controls  
  view="video"
  aspect-ratio="16/9"
>
  <!-- UI here -->
</vds-media>

Why?

• Less declarative boilerplate and minimal difference regardless of chosen provider.
• In this case providers would be more like "tech" in video.js as mentioned by Tom in this issue.
• We could reduce bundle size as there's some element-related code (if we decide to also drop provider elements).
• We could create a simple interface for implementing your own providers (if desired).
• All media state will be managed on <vds-media> - makes it simpler as there's no disconnect between media and provider on the surface.
• We could automatically detect based on the src which provider to dynamically load and set the provider up for you. One less step with respect to considering a provider and how to handle switching on source change (I've been asked many times). Devs could even pass in multiple media sources via an array and we could detect based on various options which we can call a source selection strategy (this could also be customized).

How?

Without Breaking Changes

A new base interface for providers that's not an element called MediaProvider, then we'll have stuff like AudioProvider, VideoProvider, and so on.

const media = document.createElement('vds-media');

media.onProviderAttach(provider => {
  if (isHLSProvider(media.provider)) {
    provider.config =  { lowLatencyMode: true };
    provider.on(...);
  }

  if (isHLSVideoElement(media.provider)) {
    // ...
  }
});

The hard part is figuring out how to approach the docs but we could move current "Usage" sections on provider pages down and call it "Element".

With Breaking Changes

We drop all provider elements and simply have new provider types:

import {
  type MediaProvider,
  type AudioProvider,
  type VideoProvider,
  type HLSProvider,
  isAudioProvider,
  isVideoProvider,
  isHLSProvider
} from 'vidstack';

media.provider; // base MediaProvider type

The MediaProvider interface might look something like this:

export interface MediaProvider {
  select(sources: MediaSrc[]): MediaSrc | null;
  load: () => Promise<MediaAdapter>;
  render: ($media: MediaStore) => Element;
}

// Adapter allows us to manage state on a provider.
export interface MediaAdapter {
  paused: boolean;
  muted: boolean;
  currentTime: number;
  // ...
  play(): Promise<void>;
  pause(): Promise<void>;
  canFullscreen(): Promise<void>;
  // ...
}

Options Summary

You can vote with a reaction like so:

• 👎 -> do nothing and leave the library as-is.
• 😕 -> implement this proposal without breaking changes. Have the new tech-like approach and also keep existing provider elements as they are today.
• 👍 -> drop provider elements completely and move towards a tech-like approach.

[josh-tt]

Not sure about changes/breaking changes implications, but I definitely found that having some things on provider and some on media to be confusing. Would rather have it all in one place if possible and without breaking changes, but that's just because I don't want to refactor again :)

[mihar-22]

In terms of updating your old markup it would be:

 <vds-media 
+ src="..."
  controls 
  poster="https://media-files.vidstack.io/poster.png" 
  view="video">
-  <vds-video>
-    <video preload="none" src="..."></video>
-  </vds-video>
</vds-media>

I don't think there's anything else? It'd be slightly more painful only in the case you're using <vds-hls-video> and listening to events or something. The other two providers <vds-audio> and <vds-video> don't have any API anymore.

[josh-tt]

Sounds easy enough and definitely would help imo

[thehydrogen]

How would vds-media-sync work with these changes? Would those options be brought up to the vds-media element?

[mihar-22]

That would still be kept as a separate element since it has a specific set of media responsibilities. It'll wrap <vds-media> because it's responsible for managing multiple instances. We could consider an alternative approach for stuff like that too though, maybe a way to register plugins. Not sure yet but leaning towards an element at the moment.

<vds-media-sync>
  <vds-media> 
    <!-- ... -->
  </vds-media>
</vds-media-sync>

[thehydrogen]

We implement a lot of the events and apply some styling to our vds-hls element uniquely. I think an approach of supporting both would be best, since having to squash all of these would probably be worse for readability and usability.

[QuentinGab]

On the Dx point of view only, I did find the current api with both <video /> and <vds-video /> confusing as the attributes of video and vds-video can be redondant.

What would be the implication for the bundle size, If I don't need the audioProvider would it still be imported as it is part of vds-media ?

[mihar-22]

All providers would be lazy loaded (dynamically) so you wouldn't pay the cost for others if never used :)

[mokipedia]

How would you control preload strategy? Having an option to load only metadata for example is crucial to us because we do not have posters (hoster limitation) and a lot of video players on one site.

How would you handle different sources or quality (e.g. 720p, 1080p,4k) woth your approach?

[mihar-22]

Hey @mokipedia!

How would you control preload strategy?

We'll expose a preload option on <vds-media> which will forward that setting to the underlying media element.

<vds-media preload="metadata"></vds-media>

How would you handle different sources or quality (e.g. 720p, 1080p,4k) woth your approach?

We have two options. The first is we allow the src property to accept either a string or array of sources.

media.src = [{src: '...', type: '...'}, {src: '...', type: '...'}]

We can then test these against each provider and pick the first one that can be played. The second option is we could allow <source /> to be provided to <vds-media>:

<vds-media>
  <source src="..." />
  <source src="..." />
</vds-media>

Probably just go with option 1 for now but we'll see, the second isn't hard to implement at all.

In terms of quality options, I think that should perferably be loaded the right way with an adaptive format like HLS/DASH. However, we could consider exposing an option for quality settings and load the right source based on the current player size. Maybe even a media query option. These types of things can be explored in time - we can open issues/discussions for these.

[nodgear]

This would be great for multi-media-type projects, where different types of media could be loaded at any time.
I also think breaking changes are not that big of a problem considering Vidstack is still in BETA (and we just went through a whole rework/refactoring anyway).

But how will the detection work?
Detecting by src file extension alone would be a problem for projects that uses stuff like BackBlaze to host it's content.
On BB and other hosting alternatives the public link is generated without the extension at the end of the url.

+1 for this, great idea.

[mihar-22]

Detecting by src file extension alone would be a problem for projects that uses stuff like BackBlaze to host it's content.
On BB and other hosting alternatives the public link is generated without the extension at the end of the url.

Yep so we'll need a type hint in those cases:

media.src = [{ src: '...', type: 'video/mp4' } ]