detecting motion vectors / handling scrolling better
Scrolling can make us use a video region for the area being scrolled, we should be able to supply the motion vectors to the encoder: we know that the whole region moves as one unit so this should save a lot of CPU and bandwidth.
- add optional codec API to pass motion vectors to a test video codec (either x264 or vpx)
- verify that these motion vectors are used: smaller output? metadata? save-to-file feature from r12144 then playing it back with a player that show motion vectors (ie: ffdshow does)
- compare with and the current search algorithms (try EXA, HEX, etc..) and find their limitations - unit test it all
- write simple motion detection for video region (exact match only and vertical only, for now) - unit test it, see Search Algorithms for Block-Matching in Motion Estimation
- pass the motion vector to the encoder (and turn off deblocking?)
- provide a DBUS api for passing those motion vectors from the app
Alternatively, maybe we could also use the motion vector internally: send the delta (using a fast stream compressor like lz4) + motion: still send the newly exposed region separately (different encoding)
We may want to force an I frame for the next video frame since the delta will be bigger.
Also worth thinking about: distinguish video from scrolling. Video has high framerate and lasts longer, we can then:
- use a different motion search algorithm: use ESA (aka exhaustive search) for video, for scrolling, we should stick with HEX, DIA or UHM (see X264 - Motion Estimation Method- Comparison)
- disable "subme" (subpixel motion) for scrolling
- disable partitions?
- only use a single reference frame for scrolling (much faster)
There are probably some limitations on motion vectors range, etc