Can't extract vectors from MP4 file

[saulspatz]

I really appreciate your sharing your software, and the help you've given me in getting it running, but I'm still having trouble, I'm afraid.

I built mpegflow and it works fine with mpi_sintel_final_alley_1.avi. Now, I made a little MP4 file to test with, and I have problems extracting the motion vectors. If I run in raw mode, the program seems to run correctly -- at least it produces output that doesn't look suspicious. It took about 1.5 seconds. I attach the output file as clip0mpf.txt.

When I tried producing arranged vectors, however, the program ran for over 11 minutes, until the system killed it. At that point, the output file was over 600 MB.

The first few headers look like:

pts=0 frame_index=1 pict_type=I output_type=arranged shape=90x80 origin=video
pts=1 frame_index=-1 pict_type=? output_type=arranged shape=90x80 origin=dummy
pts=2 frame_index=-1 pict_type=? output_type=arranged shape=90x80 origin=dummy
pts=3 frame_index=-1 pict_type=? output_type=arranged shape=90x80 origin=dummy
pts=4 frame_index=-1 pict_type=? output_type=arranged shape=90x80 origin=dummy

The dummy frames continue until we get:

pts=28027 frame_index=-1 pict_type=? output_type=arranged shape=90x80 origin=dummy
pts=28028 frame_index=29 pict_type=P output_type=arranged shape=90x80 origin=video

This was the last header printed before the program was killed by the system.

I have also tried extracting the raw vectors with extract_mvs.c that I downloaded from https://github.com/FFmpeg/FFmpeg/blob/master/doc/examples/extract_mvs.c.
The output of this program seems very different from that of mpegflow. The file is too big to attach in its entirety, but I've attached a large chunk from the beginning as clip0exp2.txt. I recognize that the formats are different, but I've spent a lot of time trying to reconcile the outputs of the two programs, with no success at all.

As to the video file itself, I've superimposed motion vectors with:
ffmpeg -flags2 +export_mvs -i clip0.mp4 -vf codecview=mv=pf+bf+bb clip1.mp4
and the results look reasonable. The video file is far too big to attach here, but if you'd like to see it, I can put it on Dropbox or Google Drive. If another place would be more convenient, just tell me where.

I'm running on OS X 10.10.5 (Yosemite) using ffmpeg 3.2.4.

Do you know what's happening here?

clip0exp2.txt
clip0mpf.txt

[jishnujayakumar]

Hey @saulspatz, can u help me with a compiled version of extract_mvs.c or command to compile it?

[saulspatz]

Sorry, but I can't. I posted that a year ago (to the day!) and the project I was working on terminated, and I deleted all my files, per my agreement with my employer.

…

On Tue, Apr 24, 2018 at 11:03 AM, Jishnu P ***@***.***> wrote: Hey @saulspatz <https://github.com/saulspatz>, can u help me with a compiled version of extract_mvs.c or command to compile it? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#17 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AB6gvWeQQjE0XN5eqPxhHIIykugB7kYOks5tr0y3gaJpZM4NGhNe> .

[winsieutoc]

Same my problem,
When I analyze rtsp stream, can not get AVFrameSideData data:
...
AVFrameSideData* sd;
sd = av_frame_get_side_data(pFrame, AV_FRAME_DATA_MOTION_VECTORS);
if(sd)
{
//Extract motion vectors
}

But sd always return NULL.
Pls support me.

My ffmpeg ver.2.8.15

[vadimkantorov]

Sorry, can't really do anything about that. You may try updating your ffmpeg. It can be that many ffmpeg codecs forget to make motion vectors accessible, then you'll need to hack into ffmpeg and implement that functionality

[jishnujayakumar]

Try https://github.com/jishnujayakumar/MV-Tractus