Keyframe cut position accuracy problem (in both modes)

[bouscram]

I have noticed a problem which seems to be in ffmpeg, so I don't know if you can do anything, but here it is... (I used version 3.21.0 on macOS 10.15.4 for the tests)

I have a file I want to cut. So I have set the cutting point to a keyframe, as can be seen here:

When I use the "normal" cutting mode, the sound starts at the right time, but the video is blank until the next keyframe.
When I use the "keyframe" cutting mode, everything starts at the previous keyframe, adding a part of the commercial I want to remove, as if I had selected at this point:

In both cases, the command uses the same time as a stating point:

• normal mode:

ffmpeg -hide_banner -i '/Users/cram/Movies/Handbrake/Opération jupons.mp4' -ss '603.19755' -t '6922.98914' -c copy -map '0:0' -map '0:1' -map_metadata 0 -movflags use_metadata_tags -ignore_unknown -f mp4 -y '/Users/cram/Movies/Handbrake/Opération jupons-00.10.03.197-02.05.26.186.mp4'

• keyframe mode:

ffmpeg -hide_banner -ss '603.19755' -i '/Users/cram/Movies/Handbrake/Opération jupons.mp4' -t '6922.98914' -avoid_negative_ts make_zero -c copy -map '0:0' -map '0:1' -map_metadata 0 -movflags use_metadata_tags -ignore_unknown -f mp4 -y '/Users/cram/Movies/Handbrake/Opération jupons-00.10.03.197-02.05.26.186.mp4'

To support your analysis, I have prepared a zip file including the video from the beginning and both cut files: https://www.jottacloud.com/s/13733169b83ae2749a2a2924a699121bf92

[IsabellLehmann]

I have the same problem using version 3.21.0 on Windows. Instead of a blank frame, I still see the last frame from the last cut. Another interesting thing is, if I pause the video when the new frame should be there but is not and wait for some time, suddently the new frame appears. At least in some cases

[dbogdanov]

Same problem here. MacOS, Lossless Cut 2.4.0.

[bouscram]

I have found a sort of workaround. In "keyframe" cut mode, I have set the cut point about 2 second after the keyframe intended for the cut, as shown here:

With that setting, the video what cut at the keyframe before, as I wanted it. But it has worked only when the cut point was set far enough from the intended keyframe (I tried with 1 second and it didn't work). Based on other trials, it seems that the cut is performed at the previous, not the nearest keyframe as indicated in the settings panel

(You can also get something similar by using the "normal" cut mode, and setting the cut point some frames before the keyframe where you want to cut. It will add a short blank period before the video starts)

[mifi]

Yea, I've spent a lot of time trying to find a formula that works consistently, when it comes to where to cut to achieve an accurate cut, but I have not yet succeeded. Lossless cutting is not an exact science and will depend on the codec, keyframe types and a lot of factors. This is because of how ffmpeg works. So unfortunately for some formats and codecs you just have to trial and error because I don't know how to universally solve this. I will update the readme instructions.

[jberkus]

FWIW, I've just been bitten by this. I'm trying to remove a single keyframe from the beginning of a short video (that keyframe contains me coughing), and there's simply no way to do it with LLC. Unless I'm willing to trim off multiple keyframes (and lose content) I end up with a video that still includes the cough.

[mifi]

@jberkus one thing you could try is to download the latest GIT version from http://ffmpeg.org/download.html and then run it from the command line on your file

ffmpeg -ss "some time right after your first keyframe" -i your_file -c copy -o outfile.mkv

see if that works with the newest ffmpeg. If it works, then there is a chance that it will just start working in a future losslesscut version.

[cpiber]

One solution would be to insert keyframes as per https://superuser.com/a/908325, though I don't know how feasible this is for LLC.
It definitely works to re-encode a video (tried it with x264), haven't tried adding just that one keyframe where you want to cut.

[parasiteoflife]

I'm having a similar problem, and with several video files so far.
What I normally do for cutting is use Alt + LeftArrow/RightArrow so I end up in a keyframe then I go one (1) frame to the right using . (for the start) and one to the left using , (for the end) that way LossLessCut works as intended, if I go to the keyframe and set the start time there LossLessCut goes to the previous/next frame for the start and end of the segment. Then there are these video files that would be cut to a previous/next keyframe no matter what I do, and I don't know why.

I think LossLessCut should do what I first pointed out by default when the user is cutting using keyframe cut.

[mifi]

I get a lot of questions about this. I will refer other duplicates to this issue.

TLDR: With keyframe cut, ffmpeg will always cut at the keyframe before your selected from-cutpoint. Sometimes it will be the keyframe the cursor is currently sitting at, sometimes it will be the keyframe before that. Depends on codec, format, framerate etc. I've spent a lot of time trying to find a formula that works consistently, when it comes to where to cut to achieve an accurate cut, but I have not yet succeeded.

Lossless cutting is not an exact science and will depend on the codec, keyframe types and a lot of factors. This is because of how ffmpeg works. So unfortunately for some formats and codecs you just have to trial and error because I don't know how to universally solve this without breaking it for some codecs.

Potential fixes/workarounds:

• When calling ffmpeg, add a small time value (certain amount of frames) after the current cutpoint
  • add 1 frame (sometimes doesn't work)
  • add 2 frames (sometimes doesn't work)
  • cut in the middle between the keyframes (maybe works)
  • con: risk cutting at the next keyframe instead if keyframes are too close together, e.g. 1 keyframe at every frame
• Possibly some of these issues are caused by [Bugfix] Keyframe seeking rounds wrongly #585

See also discussion in #546 #516 #887

[fernandoherreradelasheras]

This can be solved with the minimum possible transcoding by:

1. transcode from the starting point to the frame before the next keyframe
2. extract the remaining part of the video
3. concatenate both videos

I have created a simple proof of concept in bash here:

smart-video-cutter.sh

[pldavid2]

Great work on this @fernandoherreradelasheras and @mifi
One question, wouldn't it be better to use show_packets instead of show_frames to find the correct keyframe? Remeber reading that on this comment, as not every I frame is an IDR frame:
#13 (comment)

[mifi]

@pldavid2 my code is using show_packets, see:

lossless-cut/src/ffmpeg.js

Line 104 in 11e1f81

const { stdout } = await runFfprobe(['-v', 'error', ...intervalsArgs, '-show_packets', '-select_streams', streamIndex, '-show_entries', 'packet=pts_time,flags', '-of', 'json', filePath]);

Let me know if it can be improved further. This is the function losslesscut has always used to read keyframes (also the ones shown in the timeline)

[pldavid2]

sorry my bad @mifi ! I only saw the proof of concept 🤦

I'll wait for the official release to test it 😄

[cyzs233]

Personally, I used a similar approach as bouscram's "keyframe cut" method. Except you don't have to cut 2 secs after the keyframe, 5 frames (press next_frame 5 times)will do the trick. I have cut 1000+ films using this tool with this "5 frame diviation principle" on both cut_begin and cut_end, and never ever encounter this "cut at the wrong keyframe" problem. (Of course, you have to press pre_frame on cut_ends)

But to cover every user case, I suggest we make it an option to allow users to tweak "frames of diviation from cutpoint" to their own needs. 4 frames of deviation is also fine, but I do get wrong cuts from time to time, thus increased it to 5 frames.

[mifi]

@cyzs233 interesting. I wonder where the "5 frames" comes from. Sometimes keyframes are also spaced less than 5 frames apart, e.g. in high action footage.

[cyzs233]

@cyzs233 interesting. I wonder where the "5 frames" comes from. Sometimes keyframes are also spaced less than 5 frames apart, e.g. in high action footage.

Indeed, tigher keyframe margins exist. IMO, from a user's perspective, the deterioration in those extreme cases usually can't be noticed by a user(merely a handful of frames differ from the desired position after all). If they want absolutely precise frame control, they would have used AviSynth/vapoursynth at the beginning instead of lossless-cut.

[Hark0nnen]

I have been cutting some HEVC MKVs, and after some trial and error found that to achieve a precise cut i have to use "normal cut" mode and set segment start point on a frame before the desired start key-frame, and for segment end i need to set it to 2 frames before the desired end frame

[mifi]

I just tried cutting a HEVC mkv and from my experience, 4 frames after the keyframe time seems to be correct. Doesn't matter if I disable all other tracks or not. If only we could find a way to determine this number...

[mifi]

next version will have a new function that allows auto-aligning all segments to keyframes with the press of a menu button.

[mifi]

next nightly build will have a new Export Option called "Shift all start times", it can be used to automatically shift all segment start times forward by one or more frames before cutting (up to 10 frames). This can be useful if the output video starts from the wrong (preceding) keyframe.

[ac615223s5]

why not set it to one frame before the next keyframe?

[ac615223s5]

even a 20 frame offset is insufficient for video at 240fps