When using a centos 0.16.x client from the winswitch repo and a 0.17.x server on fedora 23 built from svn.
What happens is audio is scratchy and can lead to it degrading and eventually not working any more.
My test was always forcing
--speaker-codec=mp3 on the client.
I will attach logs from the client and server with -d sound.
It seems I have spoke too soon about this and I've tested it as well with a server I build from 0.16.x tags directory.
I seem to be able to reproduce the same issue here if I attach a client when there is already audio playing.
This eventually leads to the loss of sound completely. I can reconnect the client to regain sound.
I've tried the server both with gstreamer 0.10 and 1.0 using XPRA_GSTREAMER1= environment variable
My client is in virtualbox which means it is not using opengl.
I don't see the logs...
My bad. I forgot to attach the log. Here is one from the client the second one I have is 4mb in size so I will trim that down and look at some of the other things you have wrote down here.
Here is some stuff from the server with xpra info
client.speaker.actual-buffer-time=11880000 client.speaker.actual-latency-time=10000 client.speaker.buffer_count=15852 client.speaker.bytes=55918888 client.speaker.codec=wav client.speaker.codec_description=wav client.speaker.pid=5545 client.speaker.pipeline=pulsesrc device=Xpra-Speaker.monitor name=src ! queue name=queue min-threshold-time=0 max-size-buffers=0 max-size-bytes=0 max-size-time=50000000000000 leaky=2 ! volume name=volume volume=1.0 ! wav enc ! appsink name=sink emit-signals=true max-buffers=10 drop=true sync=false async=false qos=false client.speaker.queue.cur=0 client.speaker.state=active client.speaker.time=1461276776
The fact that wavpack is OK makes me think that the sound queue is getting starved and going into an underrun state. (wavpack sends very large chunks at a time). This would also explain the scratchy sound. Are you seeing underrun messages in the log output?
If so, r12454 + r12455 may help: you can adjust
XPRA_SOUND_UNDERRUN_MIN_LEVEL up from the default of 50 milliseconds, which will allow us to force the buffer to fill up more before the rest of the pipeline is allowed to pull data from it and empty it again.
Note: it is still capped by other values: the range of the measured levels (which may be part of the problem if it's empty - bumped to 150ms r12456), the maximum queue level (can't have min>max).
Another thing worth trying is changing the default sound sink on the client using
XPRA_SOUND_SINK (try autoaudiosink, pulseaudiosink and alsasink, maybe even osssink, oss4sink), it probably uses pulseaudiosink by default.
I've tried changing
XPRA_SOUND_UNDERRUN_MIN_LEVEL up in increments of 10 on my server and it almost seems like the problem gets worse but I can't tell for sure.
XPRA_SOUND_SINK i've tried all these different settings and it uses pulsesink by default.
This also doesn't seem to change anything. Sound does however seem to work fine the first time I connect and start video so I'm attaching a client log from that and also one after I disconnect the client and reconnect and sound seems to be bad at this point. Maybe this can help narrow it down?
XPRA_SOUND_UNDERRUN_MIN_LEVEL needs to be adjusted client side, and is capped by the max queue level, try larger increments too.
Started server on Fedora 23 VM with
xpra --no-daemon --bind-tcp=0.0.0.0:15000 --start-child=terminator --start-child='vlc ~/test.mp4' start :15
When connecting the client it does the same behaviour I have been describing here. Queue seems to be constantly in underrun state.
Adjusted on the client
XPRA_SOUND_UNDERRUN_MIN_LEVEL by increments of 25 from 50 -> 150 with no noticeable difference and same queue state.
Also tried large values like 1024 or 2048 just out of curiosity. Still no change.
I can attach more client logs if you like but they are very similar to the sound-bad.txt attachment.
Added a couple more logs both with underrun_min_level set to 150.
First one is with problematic mp3 audio and the second one is wavpack which while sounds fine is very unsynced with the video.
logs connecting with: XPRA_SOUND_UNDERRUN_MIN_LEVEL=30 XPRA_SOUND_SINK=pulsesink XPRA_SOUND_SOURCE_BUFFER_TIME=64 XPRA_SOUND_SOURCE_LATENCY_TIME=32 xpra attach tcp: -d sound
logs connecting with: XPRA_SOUND_UNDERRUN_MIN_LEVEL=30 XPRA_SOUND_SINK=autoaudiosink XPRA_SOUND_SOURCE_BUFFER_TIME=64 XPRA_SOUND_SOURCE_LATENCY_TIME=32 xpra attach tcp: -d sound
logs connecting with: XPRA_SOUND_UNDERRUN_MIN_LEVEL=30 XPRA_SOUND_SINK=alsasink XPRA_SOUND_SOURCE_BUFFER_TIME=64 XPRA_SOUND_SOURCE_LATENCY_TIME=32 xpra attach tcp: -d sound
XPRA_SOUND_UNDERRUN_MIN_LEVEL from 30 up to 250 (I also took the liberty of customizing the patchworks to try some borderline absurd values)... didn't seem to make any difference. Adjusting
XPRA_SOUND_SOURCE_LATENCY_TIME values didn't seem to have any real impact either (from 128/64 to 64/32 to 32/20). Smo figuered out how to disable av-sync, but that didn't help either.
When I tried alsasink though, I was able to connect/disconnect (repeat ad nauseum) without anything more than an ocasional bit of static as the sound is starting upon connection to a session with sound already playing. (I was, as usual, just using a browser and video to generate sound... connecting & disconnecting the client to induce the sound issues.)
Looking at the Session Info, though - it actually looked like the min buffer levels were adjusting to keep higher than the actual sound output levels (which, needless to say, keeps the client in a constant state of underrun).
I'll attach a screenshot of this with the
XPRA_SOUND_UNDERRUN_MIN_LEVEL=250, though I was seeing about the same with most levels (unless the scale changed, at which point it looked closer to the shot Smo attached earlier).
I did also notice (maybe useful info) that, if I paused the videos for more than a minute or so (to get up to get a beverage, for instance)... when restarting the sound, the Sound Buffer chart would usually return to a more normal looking graph, with min buffer levels no longer adjusting to stay "above" the sound levels (could something be triggering the min buffer to use the max buffer heuristics with the CentOS 6.4 using python2.6, or something like that?).
I'll attach a slideshow (3 or 4 frames, not getting carried away) to try to make that description make sense.
"Re-connect" to a session whose sound is now running, the graph
after about 30-ish seconds, the min buffer levels start to behave oddly
After a full minute(ish) the min buffer levels seem to now be "intentionally" exceeding the actual sound levels
Initial connection (with alsasink) to session with sound running - chart looks relatively as expected (also generally looks like this about 30 seconds to a minute after pausing a sound source).
Let the sound run for a vague 30 seconds and the graphs show the min buffer level continuing to be "aggressive".
Let the sound run for another long 30 seconds, and the min buffer levels seem to very efficiently exceed the sound levels, adjusting to do so.
Maxmylyn says he's done some tests with
XPRA_SOUND_MARGIN=300 and had good results.
Meanwhile, I tried with just the
XPRA_SOUND_MARGIN, set at 300, 150, & 50. At 300 sound seemed pretty good for me (without setting underrun min level)... at 150 I saw a bit of scratchiness (heard). At 50 though I saw the Sound Buffer behaving as in the bottom shot of the previous comment... and had sound go scratchy and then cut out after about 10 minutes.
Not sure that's anything consistent though.
Will do some more testing to see how consistent that is.
change pulsesink settings to try to make it work on centos6
av-sync=noto the command line?
Meanwhile, I've looked at the code and fixed some minor issues (r12526) and improved the debug output (r12527). I am attaching a quick and dirty patch which seems to improve things for me, the big problem with this patch is that it breaks #849 and win32 and osx, etc.. It seems to help here, but not in all cases, so it may need to be combined with the env vars or even something else. Warning: this is just a proof of concept at this point. Does it help?
I tried applying the patch to the 0.16.4 r12444 centOS client, but I'm sure you realize that didn't work.
Tried also with the 0.17.1 and 0.18.0 clients from your repo - but the sound wouldn't work at all with those clients against a 0.17.1 r12453 fedora 23 server. The clients connected, but the speaker was set to off (using the dock to try to re-start triggered a lot of -d sound client output, but failed to start speakers at all).
Quick log output client-side (I'll attach more complete logs of output):
2016-05-04 12:19:31,224 sound output pipeline error: GStreamer encountered a general resource error. 2016-05-04 12:19:31,225 sound output pulsesink.c(833) 2016-05-04 12:19:31,225 sound output gst_pulseringbuffer_acquire () 2016-05-04 12:19:31,225 sound output /GstPipeline:pipeline0/GstPulseSink:pulsesink0 2016-05-04 12:19:31,225 stopping speaker because of error: GStreamer encountered a general resource error.
I was able to get sound to work with the 0.17.1 r12513 client, by using the
XPRA_SOUND_SINK=alsasink though. For the most part it seemed stable, and didn't seem to cause any issues with sound for local applications (a concern I know smo had).
With the 0.17 and 0.18 clients though, especially if I included either the patch you just posted or the r12454, r12455, r12456 patches.. it seemed like av-sync was disabled (again, I suspect you realized this, but not sure if I am the only one who didn't realize how many of these options were going to disable av-sync).
Interestingly, going back to testing with the 0.16.4 r12444 client (as mentioned in comment:11), but without bothering with any of the patches, instead just using variations of
XPRA_SOUND_MARGIN, I found that the av-sync still worked, at least mostly.
XPRA_SOUND_MARGIN=50I found that the av-sync was about 4-6 frames (of 30 fps) audio late (so, ~ 130-200 ms audio late), but that was testing with a centos hardware machine without proper drivers & no opengl... and, unfortunately I still saw sound often scratchy and sometimes crashing entirely.
XPRA_SOUND_MARGIN=100I still found av-sync to be about 4-6 frames audio late, saw less and less scratchiness and less and less sound crashes... but still saw them semi-regularly (something like 1 in 4 and 1 in 5 connections before sound starts... +/- twice the rate of problems when connecting to a session with sound already running?).
XPRA_SOUND_MARGIN=150the av-sync seemed to slide to about 5-7 frames audio-late (maybe 150-210 ms?), but it wasn't lost entirely (with the patches I was seeing about 13-15 frames late... 500 ms-ish, and trying ham-fisted av-sync-delta values, up to and including 750ms, was no real help because the heuristics, as I'm sure you realize, were apparently disabled). At 150 though, I was very rarely seeing any issue with sound started after the session had been initialized, and also very rarely when disconnecting/reconnecting (maybe 1 in 12 times?... and usually only after a long period of playing after the re-connection).
Didn't get a chance to test any further along, but the fact that the sound margin seems to improve matters so much, while preserving at least the basics of av-sync, seems like it might be the way to go.
Speaking of which, unless I was missing something, it also looked like the alsasink, while it was willing to share sound with other applications, didn't seem to support av-sync?
A moment to address your questions...
XPRA_SOUND_MARGINseems to be a potentially viable solution... especially since it doesn't seem to require patches to work (I'm assuming I wasn't just getting dumb-luck results that seemed to make sense when testing without a patch), and it doesn't impact (mortally) the av-sync. I haven't tested yet with jitter, drops, etc. ... that promises to be a good time though.
XPRA_SOUND_MARGINprospects looking as promising as they are, wavpack seems like it wouldn't be worth the effort... I think.
av-sync=noon command line was sufficient, but I didn't see that as an option when searching wiki and
xpra --help, so my guesses on syntax proved, faulty. I'll check with smo and see how he found the option... might've been searching the actual code base, if so I'll go ahead and add to wiki if you'd like, or would you like me to see if someone can add to the
Ok, this is long enough now. I'm going to end the comments before i forget to attach some logs that may, possibly, be of some use.
0.18.0 CentOS client -d sound error outputs connecting to 0.17.1 server
0.17.1 CentOS client -d sound error outputs connecting to 0.17.1 server - very similar to 0.18.0, but maybe I missed something
Some more testing with my favorite 0.16.4 r12444 centos client (from winswitch beta repo), tweaking the
XPRA_SOUND_MARGIN a bit more...
XPRA_SOUND_MARGIN=175seems to produce a sound channel that's pretty reliable (maybe 1 in 20/30 connections the sound will become scratchy/crash?) even with connections/disconnections. At 200 it just superstitiously feels a little more stable. At each of these settings the av-sync (again, with opengl=off because I seem to have cobbled together a pile of hardware which insists on trying to use blacklisted swrasterizer for its opengl) is generally about 6-8 frames audio late on a 1080p display (actually reading as 1600x900 because of those driver/video card issues)... if I'm remembering the lack of opengl effects correctly (and assuming that the effects with an osx using intel vs. centos using... whatever correlate) - that would translate to about 8-10 frames audio late with a reasonable system with those settings (about 166 - 233 ms?).
Just to indulge my OCD, I double checked the 0.17.1 and 0.18.0 clients with
XPRA_GSTREAMER!=NO, but those clients still give any sound without also using
XPRA_SOUND_SINK=alsasink. Using alsasink, though, definitely doesn't seem to support av-sync.
And that's about all that I can glean. It looks to me like the
XPRA_SOUND_MARGIN=200 value makes the centos client (0.16.4 using gstreamer 0.10.29) sound stable, though it throws the av-sync a bit (further) off.
minimal patch to try to get rid of the scratchy sound
First, let's go through the previous comments to try to clarify things:
Tried also with the 0.17.1 and 0.18.0 clients from your repo - but the sound wouldn't work at all with those clients against a 0.17.1
I'm not seeing this, how do I reproduce this? (commands, etc) It works no better and no worse than previous versions for me, which makes sense considering the pipeline is unchanged. (you can view the pipeline used with "-d gstreamer", near the beginning)
Are you sure that pulseaudio is running properly at that point? (ie: run pavucontrol)
With the 0.17 and 0.18 clients though, especially if I included either the patch you just posted or the r12454, r12455, r12456 patches.. it seemed like av-sync was disabled (again, I suspect you realized this...
No. Only the patch can introduce extra latency in the sink which we will not be taking into account when calculating the video sync delay. The other 3 changesets do not have any direct impact on av-sync: they may just allow the queue levels to go higher, but the queue level is what we use for calculating the av-sync delay so this should not really matter. You should verify the av-sync delay that the server calculated to make sure that this is the case, and also verify that there is a video region on screen as non-video areas are never synchronized.
I was able to get sound to work with the 0.17.1 r12513 client, by using the XPRA_SOUND_SINK=alsasink though. For the most part it seemed stable, and didn't seem to cause any issues with sound for local applications (a concern I know smo had).
I believe that the alsasink will still actually end up using pulseaudio underneath.
That's the default value already.
With XPRA_SOUND_MARGIN=150 the av-sync seemed to slide to about 5-7 frames
Are you sure? The margin is only used to adjust the maximum level of the buffer. We still take the buffer level into account, so even if the buffer level ends up higher, we should be adjusting accordingly.
Speaking of which, unless I was missing something, it also looked like the alsasink, while it was willing to share sound with other applications, didn't seem to support av-sync?
It is possible that alsasink buffers more internally. How much was the delta? etc..
Maybe I should use rpmfusion to get mp3 plugins for the 0.17.1 & 0.18.0?
That doesn't make sense: with centos 6, either you have the gstreamer mp3 plugins or you don't. The xpra version is irrelevant.
I didn't see that as an option when searching wiki and xpra --help
These things belong in the man page - use
man xpra, which is missing an entry for this option, I will fix.
The name is
XPRA_GSTREAMER1, the possible values are
1, and this should have no effect on centos6 because centos6 does not have gstreamer 1.x
Now for some reading:
Now, everything points towards a problem in either the gstreamer sink code (especially since centos6 is based on a very old version of gstreamer 0.10...) or our configuration of the pipeline when it comes to syncing and clocks. Or both.
So I've listened to a lot of scratchy music today (butchered some great songs), whilst tweaking:
With the patch attached, we no longer try to prevent underruns at all which may be a bad thing since this code is there for a reason (remove the new "return 1" to restore it), this removes a lot of the scratchiness but not all. I'm not 100% sure which exact changes are required to get things working properly, which changes should be merged ("do-timestamp" probably) and which should be applied as workarounds for older versions of gstreamer. And as I mentioned before, these changes should be applied in the right place, cleanly, to avoid breaking all the other platforms. If that doesn't resolve it, as a last resort, it may be needed to patch the gstreamer packages to provide a more up to date pulsesink. (and hope that the problem is there rather than in pulseaudio or the kernel!)
I think clarity will help, unfortunately I'm starting to get different results from the same tests... so I'm now becoming very confused. I'll try to clarify some of my comments for you, and hope that in the process I might stumble across some explanation for what's going on.
yum install xpra-0.17.1-1.el6_7 xpra-common-0.17.1-1.el6_7. Build a 0.17.x tag server (r12534). Launch server (
dbus-launch xpra --no-daemon --bind-tcp=0.0.0.0:1203 --start-child=xterm --start-child=xterm --mdns=no start :13 --start-new-commands=yes). Launch client (
xpra attach tcp:(IP):(port)).
2016-05-06 15:05:25,118 sound output using audio codec mp3 2016-05-06 15:05:26,913 sound output pipeline error: GStreamer encountered a general resource error. 2016-05-06 15:05:26,913 sound output pulsesink.c(833) 2016-05-06 15:05:26,913 sound output gst_pulseringbuffer_acquire () 2016-05-06 15:05:26,913 sound output /GstPipeline:pipeline0/GstPulseSink:pulsesink0 2016-05-06 15:05:26,913 stopping speaker because of error: GStreamer encountered a general resource error. 2016-05-06 15:05:26,914 sound output stopping
... repeat with
yum install xpra-0.18.0-0.20160430.el6_7 xpra-common-0.18.0-0.20160430.el6_7; build trunk server (r12534); same launch commands server & client. 0.18.0 r12496 centos client; 0.18.0 r12534 fedora 23 server. Same errors.
Checked with pavucontrol, all looked as close to expected as my best guess could approximate what to expect.
-d gstreamer I noticed the list of sound devices and the indication to try
XPRA_PULSEAUDIO_DEVICE_NAME which, when I used to specify the headset I'm using, fixed the sound immediately! (for some reason, it was apparently not picking up the sound device I was using for other applications, though I may just not have set that to the satisfaction of centos).
XPRA_SOUND_MARGINchanges on the av-sync, am I sure? No. Not even a little bit. I saw the effects, but now I not only can't reproduce them, but I can't even reproduce the increased sound channel stability with the
XPRA_SOUND_MARGIN. I am at a complete loss as to what is going on with this... but that at least puts me in agreement with smo, who says he was seeing no effect from them at any point. I suspect I was doing something wrong (well, technically, something right), but I can't say what that something might be.
av-sync-delta(via control channel), however, I had no real success. At one point I'd set
av-sync=delta=700, which had dropped the delta to about 2-4 frames audio late (66-133 ms?)... only to watch it, over the course of a minute or so, stretch back out to about 6-8 frames audio late (200-266ms). At that point, obviously, the av-sync-delta starts to become more comical than useful (set at 950 ms... is it in sync or just pushed to a full second late?, or early? and, if it keeps sliding, what's the point?).
XPRA_GSTREAMER!= typo. Ooops. And yes, I was expecting that it would have no effect, and I was satisfied in that expectation... but maxmylyn had mentioned to me using the variable in conjunction with others, so I tried it to be sure. I won't bother to re-test with
on, since we don't need the extraneous confusion.
With that bit of clarification, I'm now utterly confused. Nothing that seemed to be working previously seems to be working, and my efforts to go back and find a place where I'd gone awry have been a failure.
I will read the reading and reboot the machines and ... see if I can make any sense out of it another day (best two out of three?).
I seem to have narrowed down where my mistake came from. It took a little while because it only vaguely makes any sense.
XPRA_SOUND_MARGINeffects on av-sync were an odd coincidence... or the result of something else which I haven't bothered to try to track down. Those should be ignored for now.
And here we come to the truly odd revelation.
gstlogcalls in a 0.16.4 client are not liable to be of very much help... which is why it took me so long to get my head around the fact that, somehow, that patch and the neverending tracebacks it outputs (
NameError: global name 'gstlog' is not defined) somehow makes the sound stable.
Out of curiosity, we tried deleting one, or two, or any number of permutations, of those three 'gstlog' patch bits with slightly varying results, but they all basically left the client sound to crash sooner or later. Likewise, removing that patch entirely caused the sound to sputter and crash (though, with the previous patches and with
XPRA_SOUND_MARGIN=200, or 300, the sound is somewhat stable for a little while as long as it is never running when a connection is made, but only started after initial connection (and stopped before any disconnect/reconnect).
Aside from some deranged theories of sink.py gnomes, the only thing I can think is that the code that handles the error output is somehow either triggering an alternate codepath, or is perhaps slowing things down enough so that the gstreamer 0.10.29 is able to keep up?
Very interesting. What happened was that those logging errors were cutting out all the queue self tuning code.
I think we have at least a workaround: r12544 + r12545 + r12546. This has been running for hours here on centos6 without any problems. But we may still need to increase
XPRA_SOUND_QUEUE_TIME by default so it never hits overruns.
It disables all sound queue events (or at least on centos6, we ignore them since we can't turn them off).
r12547 makes this the default on all outdated versions of gstreamer (0.10.x branch) like the one found on centos6.
XPRA_QUEUE_SILENT can be used to toggle this (-1 is "auto" which is the default, 1 disables the tuning code, 0 enables it)
r12548 catches one place where we were still adjusting min/max, causing scratchy sound to occur (just harder to hit).
The problem with this fix is that it disables the queue tuning code added for #849, so the queue may grow large as jitter accumulates and we will never do anything to bring it back down again. This can make av-sync more difficult and cost more server side RAM.
Side note: we should have tested 0.15.x and earlier versions. 0.14.x is still supported and probably immune to this problem since it doesn't have any of the #849 code.
Another interesting thing: I can now reproduce the sound issues with Fedora clients just by running with
XPRA_GSTREAMER1=0... not sure how I missed that before. (or maybe something else made things worse since the flag was added in 0.16.x - this could be useful for finding another way to fix this)
The better solution may be to trigger the queue tuning code from a timer instead of the queue events directly, and / or try to limit how many times we tune the min and max levels as this is what seems to trigger the scratchy sound. On a hunch, I had cooked up a patch that did exactly this - I will try to resurrect it and make it work. But if the fixes above work, then this is less of a priority: the better fix is just to use gstreamer 1.x wherever possible.
Scratch that (;), we have a problem: it still occasionally goes into an underrun as soon as we start the sound (just as before, this is more likely when sound is playing already before connecting) and it doesn't recover unless you pause the sound.
That's because the tuning code would forcibly raise the min level to try to prevent further underruns, but the changes above disabled it. And trying to re-instate this code, even in a milder form, just breaks things again.
Every time I introduce jitter (ie: as per ticket:999#comment:2) the sound goes scratchy, it recovers when I remove it. Which means that the sink is just draining the buffer, never allowing it to fill up again.
I've tried all sorts of things, and they all led me astray: you think it works, go down that route, only later to find out that you were just lucky when testing and that it still breaks at random..
So, the best I could come up with is the file attached (drop in replacement for the one in trunk). It deals with the underruns by pausing the pipeline, which allows the queue to fill up again. It does deal with jitter well. It does NOT deal with overruns yet and those will still cause scratchy sound, which I will try to fix. (and unfortunately, pausing the pipeline makes it more likely we'll hit overruns...)
work in progress: use pipeline pause to allow the buffer to grow
Didn't get to trying with any induced jitter, wasn't necessary to trigger scratchiness.
XPRA_SOUND_QUEUE_TIME = 450 (Smo's best guess of the default value), 550, 600, 700.
av-sync-deltavalues were no longer persistent enough to compensate and sync A & V (similarly to what I was seeing with
It looked like the Sound Buffer graphs were clear predictors of when the sound would become scratchy - once the levels begin to overrun the max buffer, the sound becomes as scratchy as it was previously becoming with the underruns (I'll attach a shot, in case you didn't already see).
Likewise, when I stop/start the speakers, the sound recovers and the graph of the sound levels returns to the expected real estate between the buffer levels... and begins climbing again. (I'll attach for comparison, again on the off chance you haven't already seen.)
Interestingly, it seems like the sound channel is considerably more stable when it is just running, than when it has to stop and then restart for whatever reason. (Though, I suppose that's just the behavior we've been seeing all along.)
SOund Buffer graph as the sound becomes scratchy, and the levels begin overrunning the max buffer
Sound buffer graph after a off/on toggling of the speakers (after an overrun state & scratchiness)
It looked like the Sound Buffer graphs were clear predictors of when the sound would become scratchy - once the levels begin to overrun the max buffer
In comment:21, I specifically stated, and emphasized with uppercase writing: It does NOT deal with overruns yet and those will still cause scratchy sound. The fix in that file was for underruns only and that part seemed to work ok when I tested it. Overruns should have been ignored.
But never mind, I believe I now have a better fix for both:
This large custom handling code was needed because the overrun / underrun handling seems to be broken with older versions of gstreamer: temporarily changing the min or max level just causes scratchy sound and does not adjust the queue level as we would expect. It would be nice if we could re-use the fadein+fadeout to smoothe out the scratchy sound with newer gstreamer versions but the volume element is placed before the queue. Maybe in a future release.
Testing: this has only been tested with mp3. It might not work well at all with the other codecs, tough. Only tested in a virtual machine. Real hardware may well behave differently, especially when it comes to clock synchronization (which is what causes the queue levels to drift up or down).
To trigger the underrun state: usually just connecting will trigger it but we now start with the sound muted for the first second or so, so this isn't audible. To trigger underruns reliably, just connect then add to the server network interface:
tc qdisc add dev eth0 root netem delay 450ms 100ms 25%
To trigger the overrun, just remove the limit that was added for triggering the underrun, after connecting:
tc qdisc del dev eth0 root netem delay 450ms 100ms 25%
If this works OK on centos6 with trunk, I'll work on a patch for older branches. Note: this form of queue tuning is crude compared to #849, so it will make av-sync harder to get right.
Err... just meant to include info about the overrun behavior in case it would be useful, guess I wasn't clear that I wasn't actually expecting the overruns to be 'handled'.
Boy have you done a wonderful job handling them now though.
Running sound is stable. Disconnecting/reconnecting is stable.
tc qdisc add dev ens3 root netem delay 400ms 100ms 25% has a moment of psychedilic sound but recovers promptly and is stable thereafter (surprisingly so). Deleting that jitter also has the moment od sound distortion, before returning to stability. I almost think it might be better with loss than the gstreamer 1.0 code - I ran it up to
tc qdisc add dev ens3 root netem loss 4% and it was still mostly stable.
This works significantly better than 'OK'.
It looked like the av-sync was impacted a bit, but surprisingly little. With a 1080p display (and working opengl) the sound seemed to be about 5-7 frames audio-late, where I would normally expect maybe 1-3 frames (or more often about 0 - 2) with opengl and a 1080p display. It seems like an impact, but certainly seems to be something that can be handled (will test with some av-sync-delta that seems to compensate for a while, see if the offset is at least consistent).
I definitely recommend it for a back port (though I suppose some testing of this sound code will be in order in addition to that needed for the gstreamer 1.0 clients when trying to find those last bits of fine tuning by display &/or whatever else seems to add or subrtact a couple of frames here and there).
patch to apply the change in full to the v0.17.x branch
Please close if this works for you.
I've tested the patch in r12567 in a vm seems to behave like the patch in trunk.
Will test the rest of them in the morning.
Works well with 16, 17, and 18. As Alex put it "A miracle has been performed".
Just found this which may allow us to tune the queue without dropping samples and therefore without causing scratchy sound?: Gstreamer 1.0 python bindings, how to implement a proper seek event:
self.pipeline.seek (self.pbRate, Gst.Format.TIME, Gst.SeekFlags.SKIP, Gst.SeekType.NONE, 0, Gst.SeekType.NONE, 0)
this ticket has been moved to: https://github.com/Xpra-org/xpra/issues/1178