xpra icon
Bug tracker and wiki

Opened 19 months ago

Last modified 2 weeks ago

#1299 new task

low fps when vigorously mousing over the video

Reported by: Antoine Martin Owned by: J. Max Mena
Priority: major Milestone: 2.3
Component: server Version: trunk
Keywords: Cc:


Split from ticket:1290#comment:2: when mousing over a video region (google-chrome, youtube in this case) the framerate seems to drop from 25 down to 8.
We want to know why that is. Is it the browser, X11 or maybe our code that is struggling with the repaints?

Change History (8)

comment:1 Changed 19 months ago by Antoine Martin

Owner: changed from Antoine Martin to alas

r13689 exposes the "damage.fps", which is the number of damage events we receive from the X11 server.
So now we can see

  • damage.fps: how many screen updates we-re receiving (any update, of any size)
  • video_subregion.fps: how many of those are updates to the video region (in full - multiple damage events will be grouped into frames for this one)
  • encoder.fps: how many frames the video encoder processes


xpra info | egrep "damage.fps|encoder.fps|subregion.fps"                                            Tue Sep 13 21:35:23 2016


The youtube video I used for testing must have been pushing 29fps.
The good news is that the bottleneck is in xpra only: the framerate does drop if you move the mouse frantically over the page, we end up losing the video region and sending jpeg and png updates, which quickly use too much CPU. (this is made worse because when we have / had a video region, we increase the quality of the "other" updates, often using png... which is a CPU pig)
As soon as I stop moving the mouse, we pick up the video region again reasonably quickly (if not, add to #967) and things get back to normal.

I am tempted to close this ticket as "works well enough, don't do crazy things".
@afarr: thoughts?

comment:2 Changed 18 months ago by alas

Owner: changed from alas to Antoine Martin

I would love to encourage that... but I've actually heard people arguing that this behavior isn't "crazy".

I'm not sure if I'd go so far as to agree that it isn't, but if we could find some way of keeping the framerate above, say, 15 fps, no matter what weird behavior someone wants to define as "not crazy"... that might be a happier medium?

That said, I would suggest pushing this to the next release, and hoping for some inspiration to fall from the heavens in the meantime.

I'll hand this back to you to decide about the milestone/closure details, and assume you'll pass it back afterward (or not).

comment:3 Changed 18 months ago by Antoine Martin

Milestone: 1.03.0
Status: newassigned

I'm not sure we'll be able to reach 15fps but we can try.

comment:4 Changed 7 weeks ago by Antoine Martin

Milestone: 3.02.3
Owner: changed from Antoine Martin to alas
Status: assignednew

r18176 may help with this (disabled by default in r18178) as we can now rate limit the pointer position updates with:

XPRA_MOUSE_DELAY=20 xpra attach ...

This will limit the updates to (1000/20=50 fps), higher values may be needed if the connection can't take the bandwidth spikes caused by the sudden vigorous movement.
If that does help, we could implement this more generically in the server, and maybe even tie this with bandwidth management. (#999)

comment:5 Changed 5 weeks ago by J. Max Mena

Was finally given some time to take a look at this one since Alex is swamped at the moment.

Running a Fedora 26 trunk r18444 server and client built from source:

  • When running Google Chrome and a YouTube Video I get a steady 23FPS pushed through my network (I have to temporarily disable the bandwidth detection as it really doesn't like my network - will mention this in #999)

If I spastically mouse around it drops to <10FPS - I've seen it drop to 8 at the lowest.

When attaching with XPRA_MOUSE_DELAY=20 it'll drop down to ~10-12FPS. If I set XPRA_MOUSE_DELAY=40 then it holds much steadier around 13-17FPS.

Last edited 3 weeks ago by Antoine Martin (previous) (diff)

comment:6 Changed 2 weeks ago by Antoine Martin

Owner: changed from alas to J. Max Mena

As of r18666, we will set the mouse delay using the client's display vertical refresh rate (as seen in the "Native_GUI" info tools on win32 and macos, xpra/platform/gui.py).
This behaviour can be disabled with XPRA_MOUSE_DELAY_AUTO=0.
This changeset also adds better debug logging with "-d mouse".

This shouldn't interfere too much, I hope. If it does, I'll turn it off again.

With a standard GPU and display (60 Hz), this should give a ~16ms delay, which looks like it would correspond to ~10fps maximum with the spastic mouse action. Not much of an improvement, but setting it higher makes it possible for us to "skip" valid frames: we should be able to handle a screen update for each vertical refresh, and each of these updates could match a new pointer position.

There may be other areas we can improve so that the pointer movements don't cause the fps to drop, in particular I am looking at #1700.
Maybe, we're switching to non-video because of the other damaged areas? Or just a sub-optimal encoding. r18669 tries to do better there.

I've seen a few things that warrant further investigation (ie: the video region's screen updates are too bursty).
Please attach the "-d damage,compress" log of just a 4 or 5 seconds showing the "spastic" movement with low fps, and include the "fps" info (xpra info | egrep "fps|batch.delay"). We want to know if it is the encoder fps that is low or both the damage fps and encoder / video subregion fps?
Is it really visually slow to refresh? ("scroll" encoding can lower the fps count, whilst still updating the screen at a high fps)
Make sure to disable bandwidth detection so it won't interfere with this. And try to turn off auto-refresh to see if that helps.
It is also worth checking that the video region is still detected correctly during the spastic episode, though there probably isn't much we can do to fix it if it isn't, as the extra damage events are "legitimate".

Last edited 2 weeks ago by Antoine Martin (previous) (diff)

comment:7 Changed 2 weeks ago by J. Max Mena

I've spent about 30 minutes looking at this one and I keep reliably running into the issue I mentioned in #1781 - which I am very sure is going to heavily impact any testing I do here. I can post what I have, but I honestly feel that it isn't indicative of what's really going on.

Last edited 2 weeks ago by J. Max Mena (previous) (diff)

comment:8 Changed 2 weeks ago by Antoine Martin

See ticket:1781#comment:4, you can avoid refresh related problems by turning off the auto-refresh.

Note: See TracTickets for help on using tickets.