Currently, all the frames are stored sequentially in the fsv file. Since we already have frames metadata in the manifest, we could theoretically store them in any order in the fsv file and resolve the correct order when loading the file. Thus when loading from stream, we would ideally be able to progressively increase the density of frames (or fps) as the stream is read.
Yet only keyframes can be correctly decoded without preceding keyframes or delta frames. So I guess the keyframes should be progressively loaded first, then the delta frames but in order relative to their closest prior keyframe.
It would also be useful to interleave color and alpha frames for the fsv alpha format, so we could also load alpha videos from stream. This would dramatically change the current format where the two tracks are clearly separated, each with their own manifest. If we changed the format this way, we might as well abandon the 4 leading bytes used to discriminate between alpha and non-alpha videos, and store this metadata in a single manifest common to both the alpha and color track. Also see #12 about how we could handle multi tracks more generally.
Currently, all the frames are stored sequentially in the fsv file. Since we already have frames metadata in the manifest, we could theoretically store them in any order in the fsv file and resolve the correct order when loading the file. Thus when loading from stream, we would ideally be able to progressively increase the density of frames (or fps) as the stream is read.
Yet only keyframes can be correctly decoded without preceding keyframes or delta frames. So I guess the keyframes should be progressively loaded first, then the delta frames but in order relative to their closest prior keyframe.
It would also be useful to interleave color and alpha frames for the fsv alpha format, so we could also load alpha videos from stream. This would dramatically change the current format where the two tracks are clearly separated, each with their own manifest. If we changed the format this way, we might as well abandon the 4 leading bytes used to discriminate between alpha and non-alpha videos, and store this metadata in a single manifest common to both the alpha and color track. Also see #12 about how we could handle multi tracks more generally.