as originally reported in #249, we can hit this PIL/Pillow bug
What I don't understand is why we didn't hit it before.. It may be because we now "optimize" the output in some cases (when the speed is low enough and close to the min-speed) or because we pass the image info (metadata) to Image.save when encoding.
Please clarify:
--- src/xpra/server/window_source.py (revision 3787) +++ src/xpra/server/window_source.py (working copy) @@ -957,9 +957,8 @@ if options: q = options.get("quality", q) q = int(min(99, max(1, q))) - kwargs = im.info - kwargs["quality"] = q - kwargs["optimize"] = optimize + kwargs = {"quality" : q, + "optimize": optimize} im.save(buf, "JPEG", **kwargs) client_options["quality"] = q else:
--- src/xpra/server/window_source.py (revision 3787) +++ src/xpra/server/window_source.py (working copy) @@ -959,7 +959,6 @@ q = int(min(99, max(1, q))) kwargs = im.info kwargs["quality"] = q - kwargs["optimize"] = optimize im.save(buf, "JPEG", **kwargs) client_options["quality"] = q else:
That error came when I used the quit command from the osx tray icon (which quits the client but which I saw no indication that the server had acknowledged as a client severed connection) and then tried to restart the client and connect back to the still running server session using a different encoder option on the client side.
I was using the osx beta client you built yesterday, on an osx 10.6.8 macbook.
I'll try the rest of your suggested possibilities tomorrow, but I wanted to be sure that you realized the wackiness of what I had to do to get that particular error. In particular, I wanted to be sure that the still not-finished tray/dock icon's behavior isn't just in need of that detail being kept in mind while the work is done vs. it being something that needs to be tracked down. I will also try to use the tray/dock icon to disconnect then reconnect to the same session but maintaining the same encoding, to see if that might be what was causing the issue.
list of test results with various included encodings, and error outputs
I couldn't find any PIL or Pillow on my mac. It is os x 10.6.8 running python 2.6.1 ... and the internets are full of horror stories of trying to install PIL- which is all I know on the subject.
I experimented with the min-speed and speed options with the JPEG encoding, because that seemed to be where I was seeing most of the PIL and Suspension not allowed here messages were appearing (one every .023 seconds or so). I'll attach the results and some server and client logs. Will test the patches tomorrow (I hope).
Wait, I am confused now.
The error in this ticket "Suspension not allowed here
" is a server side issue, but you are talking about your osx client.
This issue should manifest itself *server-side* only, and with any client.
What I am interested in is in which (server-side) patche(s) fix the issue.
[Jim Beam@jimbeam PIL]$ uname -a Linux jimbeam.spikes.eng 3.9.6-200.fc18.x86_64 # 1 SMP Thu Jun 13 18:56:55 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
PIL = Installed Packages Name : python-imaging Arch : x86_64 Version : 1.1.7 Release : 7.fc18
I'll try the patches as soon as I get a chance.
Raising the priority as we cannot make the new release with this bug still in it.
afarr / smo: This is a release blocker, and has been for 4 weeks!
Can't seem to reproduce this on osx 10.7.5 with r4149
forced jpeg encoding with --encoding=jpeg
and tried various
--speed
and --min-speed
settings
As per comment:3, this is a server side issue I believe which only affects JPEG (and maybe PNG) encoding (not decoding AFAIK), and only at low speed when we "optimize". We do care about those because we use them, even when x264 or vpx are set as primary encodings. Unfortunately, it sounds like this may affect only some versions of PIL/Pillow - and maybe only some picture sizes (bigger more likely), so more testing is needed on as many versions as possible. I have tested Pillow 2.0 (Fedora 19) today and I can encode 1080p without problems.
Best way to make it fire for testing is to just force it:
--- src/xpra/server/window_source.py (revision 4149) +++ src/xpra/server/window_source.py (working copy) @@ -1008,7 +1008,7 @@ raise e buf = StringIOClass() client_options = {} - optimize = options.get("optimize") + optimize = True if optimize is None and self.batch_config.delay>2*self.batch_config.START_DELAY: ces = self.get_current_speed() mes = self.get_min_speed()
Started xpra with
xpra attach tcp:1.2.3.4:1300 --opengl=off --encoding=jpeg --no-speaker
on osx 10.7.5
As you mentioned I made a build with the patch in comment:8
I cannot reproduce on fedora 19 which uses pillow 2.0.0
However under fedora 18 which uses python-imaging 1.1.7 I do get this error on the server still.
2013-08-12 11:59:31,974 error processing damage data: encoder error -2 when writing image file Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/xpra/server/source.py", line 1280, in data_to_packet encode_and_queue() File "/usr/lib/python2.7/site-packages/xpra/server/window_source.py", line 689, in make_data_packet_cb packet = self.make_data_packet(*data) File "/usr/lib/python2.7/site-packages/xpra/server/window_source.py", line 869, in make_data_packet data, client_options, outw, outh, outstride, bpp = encoder(coding, image, options) File "/usr/lib/python2.7/site-packages/xpra/server/window_source.py", line 1025, in PIL_encode im.save(buf, "JPEG", **kwargs) File "/usr/lib64/python2.7/site-packages/PIL/Image.py", line 1439, in save save_handler(self, fp, filename) File "/usr/lib64/python2.7/site-packages/PIL/JpegImagePlugin.py", line 471, in _save ImageFile._save(im, fp, [("jpeg", (0,0)+im.size, 0, rawmode)]) File "/usr/lib64/python2.7/site-packages/PIL/ImageFile.py", line 491, in _save raise IOError("encoder error %d when writing image file" % s) IOError: encoder error -2 when writing image file Suspension not allowed here 2013-08-12 11:59:31,982 error processing damage data: encoder error -2 when writing image file Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/xpra/server/source.py", line 1280, in data_to_packet encode_and_queue() File "/usr/lib/python2.7/site-packages/xpra/server/window_source.py", line 689, in make_data_packet_cb packet = self.make_data_packet(*data) File "/usr/lib/python2.7/site-packages/xpra/server/window_source.py", line 869, in make_data_packet data, client_options, outw, outh, outstride, bpp = encoder(coding, image, options) File "/usr/lib/python2.7/site-packages/xpra/server/window_source.py", line 1025, in PIL_encode im.save(buf, "JPEG", **kwargs) File "/usr/lib64/python2.7/site-packages/PIL/Image.py", line 1439, in save save_handler(self, fp, filename) File "/usr/lib64/python2.7/site-packages/PIL/JpegImagePlugin.py", line 471, in _save ImageFile._save(im, fp, [("jpeg", (0,0)+im.size, 0, rawmode)]) File "/usr/lib64/python2.7/site-packages/PIL/ImageFile.py", line 491, in _save raise IOError("encoder error %d when writing image file" % s) IOError: encoder error -2 when writing image file
Just to be sure I tried with with a linux and windows client not that I expected a different error but just as a test.
Same behavior
Right, that's what I thought - the bug is only in PIL
, it is already fixed in Pillow
.
So:
XPRA_PIL_OPTIMIZE
env var), or as a workaround should it be needed. Note: this does not force-enable it, it only allows it to be used.
CentOS
releases up to now, including 6.4
OSX and MS Windows are not affected because we build stuff ourselves from scratch and the instructions have been updated to use Pillow a while back.
I am closing this ticket because I have verified that the packages do include the patch.
this ticket has been moved to: https://github.com/Xpra-org/xpra/issues/375