If you have a large window but only a fraction of that window changes we waste a lot resources.
We currently create a video encoder for the whole window and waste a fair bit of time capturing, csc-ing, encoding, (sending), decoding and displaying regions of the screen that have not changed at all (and we know they haven't)
We have the "small region" code, which deals with small-ish regions by sending them using another encoding (usually rgb or png/jpeg) but it has its limits: it does not use a video encoder and is limited in size.
We should keep track of the damage areas in more detail, including their location and size (easy). Then we can detect regions of the screen that update often (easy-ish) and create a sub video encoder just for those (harder).
This could even work when the region is moving, as long as its size stays the same.
This is negated somewhat by the fact that when we send the whole window (because of other updates), it will include this region - unless we purposedly blank it out before compression and ask the client to re-assemble the two before displaying the result (hard).
An even better solution to this would be #509 or #510, as this would give us the video area precisely everytime, even before it gets converted to RGB pixels.