Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[wpe-2.38][mse] an error reported on early EOS #1366

Closed
emutavchi opened this issue Jul 16, 2024 · 4 comments
Closed

[wpe-2.38][mse] an error reported on early EOS #1366

emutavchi opened this issue Jul 16, 2024 · 4 comments
Assignees
@eocanha @modeveci
Labels
wpe-2.38

Comments

@emutavchi
Copy link

WPEWebKit MSE reports a playback error when app ends stream before appending any samples. Reproducible with attached test
mse_early_eos.html.gz

Jul 16 12:09:39 pioneer-v2-2k HtmlApp-0[2103]: [HtmlApp-0]:87 onSourceOpen()
Jul 16 12:09:39 pioneer-v2-2k HtmlApp-0[2103]: [HtmlApp-0]:35 fetch: https://dash.akamaized.net/akamai/bbb_30fps/bbb_a64k/bbb_a64k_0.m4a
Jul 16 12:09:39 pioneer-v2-2k HtmlApp-0[2103]: [HtmlApp-0]:35 fetch: https://dash.akamaized.net/akamai/bbb_30fps/bbb_30fps_1280x720_4000k/bbb_30fps_1280x720_4000k_0.m4v
Jul 16 12:09:39 pioneer-v2-2k HtmlApp-0[2103]: [HtmlApp-0]:54 appendToBuffer: https://dash.akamaized.net/akamai/bbb_30fps/bbb_a64k/bbb_a64k_0.m4a
Jul 16 12:09:39 pioneer-v2-2k HtmlApp-0[2103]: [HtmlApp-0]:54 appendToBuffer: https://dash.akamaized.net/akamai/bbb_30fps/bbb_30fps_1280x720_4000k/bbb_30fps_1280x720_4000k_0.m4v
Jul 16 12:09:39 pioneer-v2-2k HtmlApp-0[2103]: [WPEWebKit:Media:-] HTMLMediaElement::mediaLoadingFailed(A5B4F295) error = 4
Jul 16 12:09:43 pioneer-v2-2k HtmlApp-0[2103]: [HtmlApp-0]:79 Test failed. Got an error: 4, GStreamer encountered a general stream error
@eocanha
Copy link
Member

eocanha commented Aug 23, 2024

I've developed a preliminary solution in 29c0ec7, in the https://github.com/WebPlatformForEmbedded/WPEWebKit/commits/eocanha/eocanha-debug-150 branch. I still have to port it upstream, check the layout tests and have it reviewed, but at least it's a start.

eocanha added a commit to eocanha/WebKit that referenced this issue Aug 27, 2024
@eocanha
[MSE][GStreamer] Support markEndOfStream() before appendBuffer()
cb7b6f3 
https://bugs.webkit.org/show_bug.cgi?id=278726

Reviewed by NOBODY (OOPS!).

MediaSource::markEndOfStream() causes the SourceBuffer TrackBuffers to push
EOS to playback pipeline through WebKitMediaSrc. When that happens without
any SourceBuffer::appendBuffer() call, the pipeline can't finish autoplugging
and parsebin triggers an unrecoverable error. That's on GStreamer 1.18.6 at
least. On GStreamer 1.24 no error is triggered, but still, playback (of no
data) never finishes.

While it's certainly possible to change parsebin to not trigger the error,
the truth is that this seems legitimate GStreamer behaviour that shouldn't be
altered (and also, doesn't fix the problem by itself, I've checked it,
playback doesn't finish, see previous paragraph).

This commit adds support for temporarily ignoring the error when using an
affected GStreamer version, and asking the pipeline to change to READY state.
It also notifies HTMLMediaElement about timeChanged, so the "ended" event can
be triggered (after all, there's no other position to go beyond 0, as there
are no samples and 0 is already the duration, so technically playback is ended).

See: WebPlatformForEmbedded/WPEWebKit#1366

* Source/WebCore/html/HTMLMediaElement.cpp:
(WebCore::HTMLMediaElement::mediaPlayerTimeChanged): Allow time change processing when duration is zero and current time is zero.
* Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp:
(WebCore::MediaPlayerPrivateGStreamer::handleMessage): Ignore errors when the m_ignoreErrors flag is enabled.
* Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h: Add m_ignoreErrors flag.
* Source/WebCore/platform/graphics/gstreamer/mse/MediaPlayerPrivateGStreamerMSE.cpp:
(WebCore::MediaPlayerPrivateGStreamerMSE::updateStates): Report time changed on EOS with no buffer.
(WebCore::MediaPlayerPrivateGStreamerMSE::setEosWithNoBuffers): Set the m_isEosWithNoBuffer flag. Change the pipeline to READY when enabled (with errors disabled on older GStreamer versions).
* Source/WebCore/platform/graphics/gstreamer/mse/MediaPlayerPrivateGStreamerMSE.h: Added m_isEosWithNoBuffer flag.
* Source/WebCore/platform/graphics/gstreamer/mse/MediaSourcePrivateGStreamer.cpp:
(WebCore::MediaSourcePrivateGStreamer::markEndOfStream): Detect the "EOS with no buffers" condition and report it to the player private.
(WebCore::MediaSourcePrivateGStreamer::unmarkEndOfStream): Disable the "EOS with no buffers" condition.
* Source/WebCore/platform/graphics/gstreamer/mse/MediaSourcePrivateGStreamer.h: Added unmarkEndOfStream().
@eocanha eocanha mentioned this issue Aug 27, 2024
Merged
eocanha added a commit to eocanha/WebKit that referenced this issue Aug 27, 2024
@eocanha
[MSE][GStreamer] Support markEndOfStream() before appendBuffer()
9ebb270 
https://bugs.webkit.org/show_bug.cgi?id=278726

Reviewed by NOBODY (OOPS!).

MediaSource::markEndOfStream() causes the SourceBuffer TrackBuffers to push
EOS to playback pipeline through WebKitMediaSrc. When that happens without
any SourceBuffer::appendBuffer() call, the pipeline can't finish autoplugging
and parsebin triggers an unrecoverable error. That's on GStreamer 1.18.6 at
least. On GStreamer 1.24 no error is triggered, but still, playback (of no
data) never finishes.

While it's certainly possible to change parsebin to not trigger the error,
the truth is that this seems legitimate GStreamer behaviour that shouldn't be
altered (and also, doesn't fix the problem by itself, I've checked it,
playback doesn't finish, see previous paragraph).

This commit adds support for temporarily ignoring the error when using an
affected GStreamer version, and asking the pipeline to change to READY state.
It also notifies HTMLMediaElement about timeChanged, so the "ended" event can
be triggered (after all, there's no other position to go beyond 0, as there
are no samples and 0 is already the duration, so technically playback is ended).

See: WebPlatformForEmbedded/WPEWebKit#1366

* Source/WebCore/html/HTMLMediaElement.cpp:
(WebCore::HTMLMediaElement::mediaPlayerTimeChanged): Allow time change processing when duration is zero and current time is zero.
* Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp:
(WebCore::MediaPlayerPrivateGStreamer::handleMessage): Ignore errors when the m_ignoreErrors flag is enabled.
* Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h: Add m_ignoreErrors flag.
* Source/WebCore/platform/graphics/gstreamer/mse/MediaPlayerPrivateGStreamerMSE.cpp:
(WebCore::MediaPlayerPrivateGStreamerMSE::updateStates): Report time changed on EOS with no buffer.
(WebCore::MediaPlayerPrivateGStreamerMSE::setEosWithNoBuffers): Set the m_isEosWithNoBuffer flag. Change the pipeline to READY when enabled (with errors disabled on older GStreamer versions).
* Source/WebCore/platform/graphics/gstreamer/mse/MediaPlayerPrivateGStreamerMSE.h: Added m_isEosWithNoBuffer flag.
* Source/WebCore/platform/graphics/gstreamer/mse/MediaSourcePrivateGStreamer.cpp:
(WebCore::MediaSourcePrivateGStreamer::markEndOfStream): Detect the "EOS with no buffers" condition and report it to the player private.
(WebCore::MediaSourcePrivateGStreamer::unmarkEndOfStream): Disable the "EOS with no buffers" condition.
* Source/WebCore/platform/graphics/gstreamer/mse/MediaSourcePrivateGStreamer.h: Added unmarkEndOfStream().
webkit-commit-queue pushed a commit to eocanha/WebKit that referenced this issue Aug 30, 2024
@eocanha
[MSE][GStreamer] Support markEndOfStream() before appendBuffer()
a1d1ac6 
https://bugs.webkit.org/show_bug.cgi?id=278726

Reviewed by Xabier Rodriguez-Calvar.

MediaSource::markEndOfStream() causes the SourceBuffer TrackBuffers to push
EOS to playback pipeline through WebKitMediaSrc. When that happens without
any SourceBuffer::appendBuffer() call, the pipeline can't finish autoplugging
and parsebin triggers an unrecoverable error. That's on GStreamer 1.18.6 at
least. On GStreamer 1.24 no error is triggered, but still, playback (of no
data) never finishes.

While it's certainly possible to change parsebin to not trigger the error,
the truth is that this seems legitimate GStreamer behaviour that shouldn't be
altered (and also, doesn't fix the problem by itself, I've checked it,
playback doesn't finish, see previous paragraph).

This commit adds support for temporarily ignoring the error when using an
affected GStreamer version, and asking the pipeline to change to READY state.
It also notifies HTMLMediaElement about timeChanged, so the "ended" event can
be triggered (after all, there's no other position to go beyond 0, as there
are no samples and 0 is already the duration, so technically playback is ended).

See: WebPlatformForEmbedded/WPEWebKit#1366

* Source/WebCore/html/HTMLMediaElement.cpp:
(WebCore::HTMLMediaElement::mediaPlayerTimeChanged): Allow time change processing when duration is zero and current time is zero.
* Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp:
(WebCore::MediaPlayerPrivateGStreamer::handleMessage): Ignore errors when the m_ignoreErrors flag is enabled.
* Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h: Add m_ignoreErrors flag.
* Source/WebCore/platform/graphics/gstreamer/mse/MediaPlayerPrivateGStreamerMSE.cpp:
(WebCore::MediaPlayerPrivateGStreamerMSE::updateStates): Report time changed on EOS with no buffer.
(WebCore::MediaPlayerPrivateGStreamerMSE::setEosWithNoBuffers): Set the m_isEosWithNoBuffer flag. Change the pipeline to READY when enabled (with errors disabled on older GStreamer versions).
* Source/WebCore/platform/graphics/gstreamer/mse/MediaPlayerPrivateGStreamerMSE.h: Added m_isEosWithNoBuffer flag.
* Source/WebCore/platform/graphics/gstreamer/mse/MediaSourcePrivateGStreamer.cpp:
(WebCore::MediaSourcePrivateGStreamer::markEndOfStream): Detect the "EOS with no buffers" condition and report it to the player private.
(WebCore::MediaSourcePrivateGStreamer::unmarkEndOfStream): Disable the "EOS with no buffers" condition.
* Source/WebCore/platform/graphics/gstreamer/mse/MediaSourcePrivateGStreamer.h: Added unmarkEndOfStream().

Canonical link: https://commits.webkit.org/282958@main
eocanha added a commit that referenced this issue Aug 30, 2024
@eocanha
[MSE][GStreamer] Support markEndOfStream() before appendBuffer()
a584664 
https://bugs.webkit.org/show_bug.cgi?id=278726

Reviewed by Xabier Rodriguez-Calvar.

MediaSource::markEndOfStream() causes the SourceBuffer TrackBuffers to push
EOS to playback pipeline through WebKitMediaSrc. When that happens without
any SourceBuffer::appendBuffer() call, the pipeline can't finish autoplugging
and parsebin triggers an unrecoverable error. That's on GStreamer 1.18.6 at
least. On GStreamer 1.24 no error is triggered, but still, playback (of no
data) never finishes.

While it's certainly possible to change parsebin to not trigger the error,
the truth is that this seems legitimate GStreamer behaviour that shouldn't be
altered (and also, doesn't fix the problem by itself, I've checked it,
playback doesn't finish, see previous paragraph).

This commit adds support for temporarily ignoring the error when using an
affected GStreamer version, and asking the pipeline to change to READY state.
It also notifies HTMLMediaElement about timeChanged, so the "ended" event can
be triggered (after all, there's no other position to go beyond 0, as there
are no samples and 0 is already the duration, so technically playback is ended).

See: #1366

* Source/WebCore/html/HTMLMediaElement.cpp:
(WebCore::HTMLMediaElement::mediaPlayerTimeChanged): Allow time change processing when duration is zero and current time is zero.
* Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp:
(WebCore::MediaPlayerPrivateGStreamer::handleMessage): Ignore errors when the m_ignoreErrors flag is enabled.
* Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h: Add m_ignoreErrors flag.
* Source/WebCore/platform/graphics/gstreamer/mse/MediaPlayerPrivateGStreamerMSE.cpp:
(WebCore::MediaPlayerPrivateGStreamerMSE::updateStates): Report time changed on EOS with no buffer.
(WebCore::MediaPlayerPrivateGStreamerMSE::setEosWithNoBuffers): Set the m_isEosWithNoBuffer flag. Change the pipeline to READY when enabled (with errors disabled on older GStreamer versions).
* Source/WebCore/platform/graphics/gstreamer/mse/MediaPlayerPrivateGStreamerMSE.h: Added m_isEosWithNoBuffer flag.
* Source/WebCore/platform/graphics/gstreamer/mse/MediaSourcePrivateGStreamer.cpp:
(WebCore::MediaSourcePrivateGStreamer::markEndOfStream): Detect the "EOS with no buffers" condition and report it to the player private.
(WebCore::MediaSourcePrivateGStreamer::unmarkEndOfStream): Disable the "EOS with no buffers" condition.
* Source/WebCore/platform/graphics/gstreamer/mse/MediaSourcePrivateGStreamer.h: Added unmarkEndOfStream().

Canonical link: https://commits.webkit.org/282958@main
@eocanha
Copy link
Member

eocanha commented Aug 30, 2024 •
edited
Loading

The patch has landed upstream as https://commits.webkit.org/282958@main and was backported to wpe-2.38 as a584664.

Closing this Issue.

@eocanha eocanha closed this as completed Aug 30, 2024
@eocanha
Copy link
Member

eocanha commented Aug 30, 2024

I must have screwed my local environment and the commit had some merge leftovers. I've undone it (force-pushed the previous wpe-2.38 commit before mine) and I'm pushing again mine in a moment.

@eocanha eocanha reopened this Aug 30, 2024
eocanha added a commit that referenced this issue Aug 30, 2024
@eocanha
[MSE][GStreamer] Support markEndOfStream() before appendBuffer()
cf98ff8 
https://bugs.webkit.org/show_bug.cgi?id=278726

Reviewed by Xabier Rodriguez-Calvar.

MediaSource::markEndOfStream() causes the SourceBuffer TrackBuffers to push
EOS to playback pipeline through WebKitMediaSrc. When that happens without
any SourceBuffer::appendBuffer() call, the pipeline can't finish autoplugging
and parsebin triggers an unrecoverable error. That's on GStreamer 1.18.6 at
least. On GStreamer 1.24 no error is triggered, but still, playback (of no
data) never finishes.

While it's certainly possible to change parsebin to not trigger the error,
the truth is that this seems legitimate GStreamer behaviour that shouldn't be
altered (and also, doesn't fix the problem by itself, I've checked it,
playback doesn't finish, see previous paragraph).

This commit adds support for temporarily ignoring the error when using an
affected GStreamer version, and asking the pipeline to change to READY state.
It also notifies HTMLMediaElement about timeChanged, so the "ended" event can
be triggered (after all, there's no other position to go beyond 0, as there
are no samples and 0 is already the duration, so technically playback is ended).

See: #1366

* Source/WebCore/html/HTMLMediaElement.cpp:
(WebCore::HTMLMediaElement::mediaPlayerTimeChanged): Allow time change processing when duration is zero and current time is zero.
* Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.cpp:
(WebCore::MediaPlayerPrivateGStreamer::handleMessage): Ignore errors when the m_ignoreErrors flag is enabled.
* Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamer.h: Add m_ignoreErrors flag.
* Source/WebCore/platform/graphics/gstreamer/mse/MediaPlayerPrivateGStreamerMSE.cpp:
(WebCore::MediaPlayerPrivateGStreamerMSE::updateStates): Report time changed on EOS with no buffer.
(WebCore::MediaPlayerPrivateGStreamerMSE::setEosWithNoBuffers): Set the m_isEosWithNoBuffer flag. Change the pipeline to READY when enabled (with errors disabled on older GStreamer versions).
* Source/WebCore/platform/graphics/gstreamer/mse/MediaPlayerPrivateGStreamerMSE.h: Added m_isEosWithNoBuffer flag.
* Source/WebCore/platform/graphics/gstreamer/mse/MediaSourcePrivateGStreamer.cpp:
(WebCore::MediaSourcePrivateGStreamer::markEndOfStream): Detect the "EOS with no buffers" condition and report it to the player private.
(WebCore::MediaSourcePrivateGStreamer::unmarkEndOfStream): Disable the "EOS with no buffers" condition.
* Source/WebCore/platform/graphics/gstreamer/mse/MediaSourcePrivateGStreamer.h: Added unmarkEndOfStream().

Canonical link: https://commits.webkit.org/282958@main
@eocanha
Copy link
Member

eocanha commented Aug 30, 2024

Now for good, backported to wpe-2.38 as cf98ff8

@eocanha eocanha closed this as completed Aug 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@eocanha eocanha

@modeveci modeveci

Labels
wpe-2.38
Milestone
No milestone
Development

No branches or pull requests
3 participants
@emutavchi @eocanha @modeveci