Xpra: Ticket #535: server-side memory leak

0.11.4 leaks memory on server side. After using attached session for several days I've noticed that xpra process substantially grew in memory (RES > 2.5 GiB; VIRT > 3 GiB). I'm not sure how to troubleshoot this issue but I'm fairly certain that it is not related to audio which I had disabled. For now I have to run xpra upgrade periodically to release RAM.



Mon, 17 Mar 2014 03:50:22 GMT - Antoine Martin: owner, milestone changed

Could be related to #457.

Does it go away if you remove the webp codec from the system? (see #491) (remove xpra/codecs/webm)

It may also be libav related...


Mon, 17 Mar 2014 04:27:59 GMT - onlyjob:

Thank you for quick reply.

Replying to totaam:

Could be related to #457.

Does it go away if you remove the webp codec from the system? (see #491) (remove xpra/codecs/webm)

I'll try without webp but it will take days to see the difference. Besides I have impression that those bugs are about leaks on the client side while mine is on server side.

I may also be libav related...

Everything is possible. :) I'm still running libav-9.11 on client and server. It is the same libav that I was using for a while but I started to notice memory leak only recently...


Mon, 17 Mar 2014 04:54:51 GMT - Antoine Martin:

I have impression that those bugs are about leaks on the client side while mine is on server side

True, but I suspect that the webp library may well be leaking when encoding as well as decoding.

I'm still running libav-9.11 on client and server

The reason why I suspect libav by default is twofold:

So, it would be interesting to see if the leak still occurs when using an encoding we know for sure does not leak: rgb or png have never shown signs of leaking - yet. (and to be certain video does not get used in any way, the best way is to not install the codec)


Mon, 17 Mar 2014 05:17:36 GMT - onlyjob:

For what's it worth this leak is with RGB encoding (set as default in config)...


Wed, 19 Mar 2014 09:01:13 GMT - Antoine Martin:

It's still worth deleting all the other codecs to ensure that only RGB can be used. More and more the code is using the chosen encoding only as a hint and will switch to other encodings as appropriate (though in theory, RGB is probably the only one that *should* be honoured at all times...).

Can you post the output of: xpra info | grep memory then things get bad? Does minimizing the windows help? Re-connecting the client?


Fri, 28 Mar 2014 16:26:36 GMT - jamie:

I seem to be experiencing the same issue. I can easily reproduce (the memory increases with every click of the web browser that I access via xrpa).

I'm running the client and server on the same machine (so I can run my web browser as an isolated user on my machine).

Question: how do I disable the other encodings? I don't seem to have a codecs/webm folder (maybe remove codecs/vpx?). I tried moving the entire codecs folder to codecs.off but it didn't seem to have any effect.

I've attached my /etc/xpra/xrpa.conf file and the server log file.

Here's my memory after starting xpra:

0 jamie@animal:cdtemp.5reEHd$ ps -e -o rss,cmd | egrep "[X]pra" | grep iceweasel
76784 /usr/bin/Xorg-for-Xpra-:201 -dpi 96 -noreset -nolisten tcp +extension GLX +extension RANDR +extension RENDER -logfile /home/iceweasel/.xpra/Xorg.:201.log -config /etc/xpra/xorg.conf :201
0 jamie@animal:cdtemp.5reEHd$

After launching iceweasel (Firefox)

jamie@animal:cdtemp.5reEHd$ ps -e -o rss,cmd | egrep "[X]pra" | grep iceweasel
110616 /usr/bin/Xorg-for-Xpra-:201 -dpi 96 -noreset -nolisten tcp +extension GLX +extension RANDR +extension RENDER -logfile /home/iceweasel/.xpra/Xorg.:201.log -config /etc/xpra/xorg.conf :201
0 jamie@animal:cdtemp.5reEHd$

Here's my command for starting xpra (run as iceweasel user):

/usr/bin/python /usr/bin/xpra --mmap-group --socket-dir=/home/iceweasel/xpra-socket --no-mdns start :201

Here's the attach command (run as jamie user):

/usr/bin/python /usr/bin/xpra --socket-dir=/home/iceweasel/xpra-socket --mmap-group --tray-icon=/usr/share/icons/hicolor/scalable/apps/iceweasel.svg --encoding=rgb24 attach :201

Here's the output of the info command:

0 jamie@animal:bin$ sudo -u iceweasel /usr/bin/python /usr/bin/xpra --mmap-group --socket-dir=/home/iceweasel/xpra-socket --no-mdns info :201 | grep memory
memory.children.idrss=0
memory.children.inblock=0
memory.children.isrss=0
memory.children.ixrss=0
memory.children.majflt=0
memory.children.maxrss=30048
memory.children.minflt=19940
memory.children.msgrcv=0
memory.children.msgsnd=0
memory.children.nivcsw=54
memory.children.nsignals=0
memory.children.nswap=0
memory.children.nvcsw=261
memory.children.oublock=32
memory.children.stime=0
memory.children.utime=0
memory.server.idrss=0
memory.server.inblock=0
memory.server.isrss=0
memory.server.ixrss=0
memory.server.majflt=4
memory.server.maxrss=64392
memory.server.minflt=27163
memory.server.msgrcv=0
memory.server.msgsnd=0
memory.server.nivcsw=234
memory.server.nsignals=0
memory.server.nswap=0
memory.server.nvcsw=3989
memory.server.oublock=12024
memory.server.stime=0
memory.server.utime=0
0 jamie@animal:bin$

Fri, 28 Mar 2014 16:34:24 GMT - jamie: attachment set

jamie-etc-xpra-xpra.conf


Fri, 28 Mar 2014 16:35:15 GMT - jamie: attachment set

jamie-server-log


Fri, 28 Mar 2014 16:46:40 GMT - jamie:

Also, minimizing the window causes the memory usage of the server to increase and maximizing the window causes the memory to increase.

Detaching has no impact on memory usage.

Attaching causes the memory usage to increase.


Sat, 29 Mar 2014 01:39:13 GMT - Antoine Martin: owner changed

@jamie: what is your distro? What version are you using? Do you get the same results with 0.12.x? Please include the full xpra info as an attachment, not just the memory bits.

how do I disable the other encodings?


You can disable most of them at build time. At runtime, it is more difficult.

I don't seem to have a codecs/webm folder (maybe remove codecs/vpx?).


Strange, did you build without webp (aka webm) support?

I tried moving the entire codecs folder to codecs.off but it didn't seem to have any effect.


It will have an effect and will certainly break everything. If it didn't, maybe you moved the wrong one.

minimizing the window causes the memory usage of the server to increase and maximizing the window causes the memory to increase


Interesting, maximizing is understandable. But minimizing less so!

Attaching causes the memory usage to increase


This one is expected as we allocate resources for each client.


Mon, 31 Mar 2014 01:15:48 GMT - jamie: attachment set

Full output of xpra info


Mon, 31 Mar 2014 01:16:47 GMT - jamie:

Thanks for the response. I just added the full output of xpra info.

I'm running Debian Jessie and xpra 0.11.4+dfsg-1.

I will try to compile latest xpra and see if the problem goes away.


Mon, 31 Mar 2014 02:49:58 GMT - jamie:

Alas, no luck.

I tried installing using the Debian packaging with a 0.12.2 tar ball and slight modifications to the Debian-shipped patches:

diff -u xpra-debian-source/xpra-0.11.4+dfsg/debian/patches/disable_html5.patch debian/patches/disable_html5.patch
--- xpra-debian-source/xpra-0.11.4+dfsg/debian/patches/disable_html5.patch  2014-02-03 11:23:26.000000000 -0500
+++ debian/patches/disable_html5.patch  2014-03-30 22:12:09.411752548 -0400
@@ -7,7 +7,7 @@
  gtk3_ENABLED = client_ENABLED
  qt4_ENABLED = client_ENABLED
  opengl_ENABLED = client_ENABLED
--html5_ENABLED = True
+-html5_ENABLED = not WIN32 and not OSX
 +html5_ENABLED = False
  bencode_ENABLED = True
diff -u xpra-debian-source/xpra-0.11.4+dfsg/debian/patches/enable-x264-opencl.patch debian/patches/enable-x264-opencl.patch
--- xpra-debian-source/xpra-0.11.4+dfsg/debian/patches/enable-x264-opencl.patch 2014-02-05 09:54:38.000000000 -0500
+++ debian/patches/enable-x264-opencl.patch 2014-03-30 22:12:09.411752548 -0400
@@ -6,18 +6,16 @@
 --- a/xpra/codecs/enc_x264/encoder.pyx
 +++ b/xpra/codecs/enc_x264/encoder.pyx
-@@ -69,8 +69,11 @@
-         int i_bframe_pyramid    #Keep some B-frames as references: 0=off, 1=strict hierarchical, 2=normal
+@@ -117,6 +117,9 @@
          int b_open_gop
          int b_bluray_compat
 +        #not all builds support this:
 +        int b_opencl            #use OpenCL when available
 +
+         rc  rc                  #rate control
+
      ctypedef struct x264_t:
-         pass
-     ctypedef struct x264_nal_t:
-         int i_ref_idc
 @@ -250,8 +253,9 @@
          #we don't want IDR frames either:
          param.i_keyint_min = 999999
diff -u xpra-debian-source/xpra-0.11.4+dfsg/debian/patches/libav10.patch debian/patches/libav10.patch
--- xpra-debian-source/xpra-0.11.4+dfsg/debian/patches/libav10.patch  2014-03-03 07:33:40.000000000 -0500
+++ debian/patches/libav10.patch  2014-03-30 22:15:27.323962377 -0400
@@ -1,137 +1,24 @@
-Last-Update: 2014-03-03
+Description: libav10 compatibility
+Author: Antoine Martin <antoine@devloop.org.uk>
 Forwarded: not-needed
-Author: Dmitry Smirnov <onlyjob@member.fsf.org>
 Bug-Debian: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=739459
 Bug-Xpra: https://www.xpra.org/trac/ticket/524
-Description: fix FTBFS with libav10
---- a/xpra/codecs/dec_avcodec/decoder.pyx
-+++ b/xpra/codecs/dec_avcodec/decoder.pyx
-@@ -86,12 +86,12 @@
-         int thread_type
-         int flags
-         int flags2
-
--    AVPixelFormat PIX_FMT_NONE
--    AVCodecID CODEC_ID_H264
--    AVCodecID CODEC_ID_VP8
--    #AVCodecID CODEC_ID_VP9
-+    AVPixelFormat AV_PIX_FMT_NONE
-+    AVCodecID AV_CODEC_ID_H264
-+    AVCodecID AV_CODEC_ID_VP8
-+    #AVCodecID AV_CODEC_ID_VP9
-
-     #init and free:
-     void avcodec_register_all()
-     AVCodec *avcodec_find_decoder(AVCodecID id)
-@@ -164,13 +164,13 @@
-     global CODECS
-     if CODECS is None:
-         avcodec_register_all()
-         CODECS = []
--        if avcodec_find_decoder(CODEC_ID_H264)!=NULL:
-+        if avcodec_find_decoder(AV_CODEC_ID_H264)!=NULL:
-             CODECS.append("h264")
--        if avcodec_find_decoder(CODEC_ID_VP8)!=NULL:
-+        if avcodec_find_decoder(AV_CODEC_ID_VP8)!=NULL:
-             CODECS.append("vp8")
--        #if avcodec_find_decoder(CODEC_ID_VP9)!=NULL:
-+        #if avcodec_find_decoder(AV_CODEC_ID_VP9)!=NULL:
-         #    CODECS.append("vp9")
-     return CODECS
-
-
-@@ -366,24 +366,24 @@
-                 break
-         if self.colorspace==NULL:
-             error("invalid pixel format: %s", colorspace)
-             return  False
--        self.pix_fmt = FORMAT_TO_ENUM.get(colorspace, PIX_FMT_NONE)
--        if self.pix_fmt==PIX_FMT_NONE:
-+        self.pix_fmt = FORMAT_TO_ENUM.get(colorspace, AV_PIX_FMT_NONE)
-+        if self.pix_fmt==AV_PIX_FMT_NONE:
-             error("invalid pixel format: %s", colorspace)
-             return  False
-         self.actual_pix_fmt = self.pix_fmt
-
-         avcodec_register_all()
-
-         if self.encoding=="h264":
--            self.codec = avcodec_find_decoder(CODEC_ID_H264)
-+            self.codec = avcodec_find_decoder(AV_CODEC_ID_H264)
-             if self.codec==NULL:
-                 error("codec H264 not found!")
-                 return  False
-         else:
-             assert self.encoding=="vp8"
--            self.codec = avcodec_find_decoder(CODEC_ID_VP8)
-+            self.codec = avcodec_find_decoder(AV_CODEC_ID_VP8)
-             if self.codec==NULL:
-                 error("codec VP8 not found!")
-                 return  False
-
---- a/xpra/codecs/dec_avcodec2/decoder.pyx
-+++ b/xpra/codecs/dec_avcodec2/decoder.pyx
-@@ -83,12 +83,12 @@
-         int flags
-         int flags2
-         int refcounted_frames
-
--    AVPixelFormat PIX_FMT_NONE
--    AVCodecID CODEC_ID_H264
--    AVCodecID CODEC_ID_VP8
--    #AVCodecID CODEC_ID_VP9
-+    AVPixelFormat AV_PIX_FMT_NONE
-+    AVCodecID AV_CODEC_ID_H264
-+    AVCodecID AV_CODEC_ID_VP8
-+    #AVCodecID AV_CODEC_ID_VP9
-
-     #init and free:
-     void avcodec_register_all()
-     AVCodec *avcodec_find_decoder(AVCodecID id)
-@@ -155,13 +155,13 @@
-     global CODECS
-     if CODECS is None:
-         avcodec_register_all()
-         CODECS = []
--        if avcodec_find_decoder(CODEC_ID_H264)!=NULL:
-+        if avcodec_find_decoder(AV_CODEC_ID_H264)!=NULL:
-             CODECS.append("h264")
--        if avcodec_find_decoder(CODEC_ID_VP8)!=NULL:
-+        if avcodec_find_decoder(AV_CODEC_ID_VP8)!=NULL:
-             CODECS.append("vp8")
--        #if avcodec_find_decoder(CODEC_ID_VP9)!=NULL:
-+        #if avcodec_find_decoder(AV_CODEC_ID_VP9)!=NULL:
-         #    CODECS.append("vp9")
-     return CODECS
-
-
-@@ -267,24 +267,24 @@
-                 break
-         if self.colorspace==NULL:
-             error("invalid pixel format: %s", colorspace)
-             return  False
--        self.pix_fmt = FORMAT_TO_ENUM.get(colorspace, PIX_FMT_NONE)
--        if self.pix_fmt==PIX_FMT_NONE:
-+        self.pix_fmt = FORMAT_TO_ENUM.get(colorspace, AV_PIX_FMT_NONE)
-+        if self.pix_fmt==AV_PIX_FMT_NONE:
-             error("invalid pixel format: %s", colorspace)
-             return  False
-         self.actual_pix_fmt = self.pix_fmt
-
-         avcodec_register_all()
-
-         if self.encoding=="h264":
--            self.codec = avcodec_find_decoder(CODEC_ID_H264)
-+            self.codec = avcodec_find_decoder(AV_CODEC_ID_H264)
-             if self.codec==NULL:
-                 error("codec H264 not found!")
-                 return  False
-         else:
-             assert self.encoding=="vp8"
--            self.codec = avcodec_find_decoder(CODEC_ID_VP8)
-+            self.codec = avcodec_find_decoder(AV_CODEC_ID_VP8)
-             if self.codec==NULL:
-                 error("codec VP8 not found!")
-                 return  False
+Index: xpra/codecs/dec_avcodec/decoder.pyx
+===================================================================
+--- a/xpra/codecs/dec_avcodec/decoder.pyx  (revision 5906)
++++ b/xpra/codecs/dec_avcodec/decoder.pyx  (working copy)
+@@ -109,7 +109,13 @@
+     int avcodec_default_get_buffer(AVCodecContext *s, AVFrame *pic)
+     void avcodec_default_release_buffer(AVCodecContext *s, AVFrame *pic)
+
++cdef extern from "libavcodec/old_codec_ids.h":
++    AVPixelFormat PIX_FMT_NONE
++    AVCodecID CODEC_ID_H264
++    AVCodecID CODEC_ID_VP8
++    #AVCodecID CODEC_ID_VP9
+
++
+ def get_version():
+     return (LIBAVCODEC_VERSION_MAJOR, LIBAVCODEC_VERSION_MINOR, LIBAVCODEC_VERSION_MICRO)
diff -u xpra-debian-source/xpra-0.11.4+dfsg/debian/patches/libav-no0RGB.patch debian/patches/libav-no0RGB.patch
--- xpra-debian-source/xpra-0.11.4+dfsg/debian/patches/libav-no0RGB.patch 2014-02-02 22:17:00.000000000 -0500
+++ debian/patches/libav-no0RGB.patch 2014-03-30 22:12:09.411752548 -0400
@@ -36,12 +36,13 @@
  AV_PIX_FMT_GBRP
 --- a/xpra/codecs/dec_avcodec/decoder.pyx
 +++ b/xpra/codecs/dec_avcodec/decoder.pyx
-@@ -140,8 +140,6 @@
-             "YUV422P"   : "AV_PIX_FMT_YUV422P",
-             "YUV444P"   : "AV_PIX_FMT_YUV444P",
-             "RGB"       : "AV_PIX_FMT_RGB24",
--            "XRGB"      : "AV_PIX_FMT_0RGB",
--            "BGRX"      : "AV_PIX_FMT_BGR0",
-             "ARGB"      : "AV_PIX_FMT_ARGB",
-             "BGRA"      : "AV_PIX_FMT_BGRA",
-             "GBRP"      : "AV_PIX_FMT_GBRP",
+@@ -143,8 +143,6 @@
+         "YUV422P"   : "AV_PIX_FMT_YUV422P",
+         "YUV444P"   : "AV_PIX_FMT_YUV444P",
+         "RGB"       : "AV_PIX_FMT_RGB24",
+-        "XRGB"      : "AV_PIX_FMT_0RGB",
+-        "BGRX"      : "AV_PIX_FMT_BGR0",
+         "ARGB"      : "AV_PIX_FMT_ARGB",
+         "BGRA"      : "AV_PIX_FMT_BGRA",
+         "GBRP"      : "AV_PIX_FMT_GBRP",
+
diff -u xpra-debian-source/xpra-0.11.4+dfsg/debian/patches/series debian/patches/series
--- xpra-debian-source/xpra-0.11.4+dfsg/debian/patches/series 2014-03-03 16:49:16.000000000 -0500
+++ debian/patches/series 2014-03-30 22:12:09.411752548 -0400
@@ -19,4 +19,4 @@
 libav10.patch
 ## Conditional patch, must be the last one.
-old-libav.patch
+#old-libav.patch

However, same result.

Next, I did an svn checkout of the 0.12.x branch with no debian patches. I first got a compile error:

x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/python2.7 -c xpra/codecs/dec_avcodec/decoder.c -o build/temp.linux-x86_64-2.7/xpra/codecs/dec_avcodec/decoder.o -Wall -Werror=implicit-function-declaration -fPIC
xpra/codecs/dec_avcodec/decoder.c: In function ‘initdecoder’:
xpra/codecs/dec_avcodec/decoder.c:10104:45: error: ‘AV_PIX_FMT_0RGB’ undeclared (first use in this function)
   __pyx_t_1 = __Pyx_PyInt_From_unsigned_int(AV_PIX_FMT_0RGB); if (unlikely(!__pyx_t_1)) {__pyx_filename = __pyx_f[1]; __pyx_lineno = 17; __pyx_clineno = __LINE__; goto __pyx_L1_error;}
                                             ^
xpra/codecs/dec_avcodec/decoder.c:10104:45: note: each undeclared identifier is reported only once for each function it appears in
xpra/codecs/dec_avcodec/decoder.c:10116:45: error: ‘AV_PIX_FMT_BGR0’ undeclared (first use in this function)
   __pyx_t_1 = __Pyx_PyInt_From_unsigned_int(AV_PIX_FMT_BGR0); if (unlikely(!__pyx_t_1)) {__pyx_filename = __pyx_f[1]; __pyx_lineno = 18; __pyx_clineno = __LINE__; goto __pyx_L1_error;}
                                             ^
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
dh_auto_build: python setup.py build --force returned exit code 1
make: *** [build] Error 1
dpkg-buildpackage: error: debian/rules build gave error exit status 2
debuild: fatal error at line 1364:
dpkg-buildpackage -rfakeroot -D -us -uc -b failed
29 jamie@animal:src$

So, I just applied this patch:

--- a/xpra/codecs/csc_swscale/constants.txt
+++ b/xpra/codecs/csc_swscale/constants.txt
@@ -14,8 +14,8 @@
 AV_PIX_FMT_YUV422P
 AV_PIX_FMT_YUV444P
 AV_PIX_FMT_RGB24
-AV_PIX_FMT_0RGB
-AV_PIX_FMT_BGR0
+#AV_PIX_FMT_0RGB
+#AV_PIX_FMT_BGR0
 AV_PIX_FMT_ARGB
 AV_PIX_FMT_BGRA
 AV_PIX_FMT_GBRP
--- a/xpra/codecs/dec_avcodec/constants.txt
+++ b/xpra/codecs/dec_avcodec/constants.txt
@@ -14,8 +14,8 @@
 AV_PIX_FMT_YUV422P
 AV_PIX_FMT_YUV444P
 AV_PIX_FMT_RGB24
-AV_PIX_FMT_0RGB
-AV_PIX_FMT_BGR0
+#AV_PIX_FMT_0RGB
+#AV_PIX_FMT_BGR0
 AV_PIX_FMT_ARGB
 AV_PIX_FMT_BGRA
 AV_PIX_FMT_GBRP
--- a/xpra/codecs/dec_avcodec/decoder.pyx
+++ b/xpra/codecs/dec_avcodec/decoder.pyx
@@ -143,8 +143,6 @@
         "YUV422P"   : "AV_PIX_FMT_YUV422P",
         "YUV444P"   : "AV_PIX_FMT_YUV444P",
         "RGB"       : "AV_PIX_FMT_RGB24",
-        "XRGB"      : "AV_PIX_FMT_0RGB",
-        "BGRX"      : "AV_PIX_FMT_BGR0",
         "ARGB"      : "AV_PIX_FMT_ARGB",
         "BGRA"      : "AV_PIX_FMT_BGRA",
         "GBRP"      : "AV_PIX_FMT_GBRP",

And it successfully compiled.

However, I'm still seeing the server memory leak.

Let me know if there are any other tests you'd like me to perform or any other logs I can provide to help get to the bottom of this.


Thu, 03 Apr 2014 13:36:57 GMT - Antoine Martin: owner, status changed; milestone set

Confirmed here at last, took a few hours, using glxspheres made it happen quicker (lots of frames per second).

Will take care of it as a priority. Wiki link: wiki/Debugging


Thu, 03 Apr 2014 14:22:53 GMT - Antoine Martin:

Output from top and running glxspheres with XPRA_XSHM=0:

20450 antoine   20   0 65.240g 101628  32916 R  97.2  0.8   0:06.47
20450 antoine   20   0 65.240g 111532  32976 S  94.5  0.9   2:39.92
20450 antoine   20   0 65.583g 110928  32984 S  92.6  0.9   7:59.86
20450 antoine   20   0 65.587g 126728  32984 S  94.7  1.0   9:54.56
20450 antoine   20   0 65.578g 113120  32984 S  95.0  0.9  10:20.34
20450 antoine   20   0 65.587g 133464  32984 S  93.2  1.1  12:33.52
20450 antoine   20   0 65.649g 156240  32984 S  92.2  1.3  17:20.23
20450 antoine   20   0 65.645g 177276  32984 R  88.7  1.4  18:57.47

Still leaking, so it isn't xshm refcount related.


Fri, 04 Apr 2014 05:08:30 GMT - Antoine Martin:

With XPRA_DETECT_LEAKS=1 support added in r6013, I see:

2014-04-04 12:01:46,738 print_leaks():
       55 : <type 'tuple'> : 2829 matches
       27 : <type 'cell'> : 762 matches
       22 : <type 'frame'> : 274 matches
        9 : <type 'list'> : 2883 matches
        7 : <type 'builtin_function_or_method'> : []
        4 : <type 'instancemethod'> : []
        2 : <type 'instance'> : 442 matches

Then a little while later:

2014-04-04 12:03:26,848 print_leaks():
       54 : <type 'tuple'> : 3383 matches
       34 : <type 'cell'> : 1173 matches
       17 : <type 'frame'> : 469 matches
       15 : <type 'builtin_function_or_method'> : []
        9 : <type 'list'> : 2982 matches
        6 : <type 'xpra.x11.bindings.ximage.XShmImageWrapper'> : 47 matches
        3 : <type 'instancemethod'> : []
        2 : <type 'dict'> : 3783 matches
2014-04-04 13:57:21,998 print_leaks():
       52 : <type 'tuple'> : 11378 matches
       45 : <type 'cell'> : 6503 matches
       35 : <type 'instancemethod'> : []
       23 : <type 'list'> : 6089 matches
       22 : <type 'frame'> : 9958 matches
       13 : <type 'dict'> : 3923 matches
       10 : <type 'collections.deque'> : 55 matches
        7 : <type 'builtin_function_or_method'> : []
        3 : <type 'xpra.x11.bindings.ximage.XShmImageWrapper'> : 129 matches

So, it looks like we're leaking:


Fri, 04 Apr 2014 12:59:11 GMT - Antoine Martin: attachment set

disable subregion code


Fri, 04 Apr 2014 13:16:59 GMT - Antoine Martin:

After some more improvements to the leak debugging code in r6017, r6021, r6022

I have narrowed it down to the video subregions code, using the attachment/ticket/535/no-video-subregion.patch seems to have stopped the leak.

Unless I can fix this reliably in the next 24 hours, I may disable video subregions (server side) for the next stable update.

@onlyjob and @jamie: can you please confirm that the patch does prevent the leak you were seeing?


Fri, 04 Apr 2014 13:35:54 GMT - onlyjob:

I had troubles reproducing the leak lately and I'm not sure why. It could be that it was leaking only while I was using some application (libreoffice?) that I don't use too often...


Fri, 04 Apr 2014 14:37:54 GMT - Antoine Martin:

I had troubles reproducing the leak lately


Me too, and that makes me nervous... maybe the "fix" is only hiding the real cause, by making us follow a different codepath.

The debug output still points towards a very slow leak:

The other objects that cycle a lot are all fairly stable at around:

        3 : <type 'cell'>     : 459 matches
        1 : <type 'dict'>     : 3593 matches
        2 : <type 'list'>     : 2806 matches
        1 : <type 'instance'> : 427 matches
       12 : <type 'tuple'>    : 2365 matches

The two could be related. More investigation is needed.

I did find some other interesting leak issues along the way, like a fairly rare case when clients disconnect, fixed in r6023 - will need a backport.


Fri, 04 Apr 2014 14:52:00 GMT - jamie:

Unfortunately (or fortunately?) I'm still able to reproduce the memory leak, even with this patch. I tried applying the patch against the 12.x tag and against trunk itself. Here's the diff of my svn checkout that I most recently build a deb package from:

0 jamie@animal:xpra.svn.trunk$ svn diff
Index: src/xpra/codecs/csc_swscale/constants.txt
===================================================================
--- src/xpra/codecs/csc_swscale/constants.txt	(revision 6023)
+++ src/xpra/codecs/csc_swscale/constants.txt	(working copy)
@@ -14,8 +14,6 @@
 AV_PIX_FMT_YUV422P
 AV_PIX_FMT_YUV444P
 AV_PIX_FMT_RGB24
-AV_PIX_FMT_0RGB
-AV_PIX_FMT_BGR0
 AV_PIX_FMT_ARGB
 AV_PIX_FMT_BGRA
 AV_PIX_FMT_GBRP
Index: src/xpra/codecs/dec_avcodec/constants.txt
===================================================================
--- src/xpra/codecs/dec_avcodec/constants.txt	(revision 6023)
+++ src/xpra/codecs/dec_avcodec/constants.txt	(working copy)
@@ -14,8 +14,6 @@
 AV_PIX_FMT_YUV422P
 AV_PIX_FMT_YUV444P
 AV_PIX_FMT_RGB24
-AV_PIX_FMT_0RGB
-AV_PIX_FMT_BGR0
 AV_PIX_FMT_ARGB
 AV_PIX_FMT_BGRA
 AV_PIX_FMT_GBRP
Index: src/xpra/server/window_video_source.py
===================================================================
--- src/xpra/server/window_video_source.py	(revision 6023)
+++ src/xpra/server/window_video_source.py	(working copy)
@@ -245,6 +245,7 @@
     def identify_video_subregion(self):
+	return
         if self.statistics.damage_events_count < self.video_subregion_set_at:
             #stats got reset
             self.video_subregion_set_at = 0
0 jamie@animal:xpra.svn.trunk$

I've just applied one debian patch (without which it fails to compile) and the patch you attached.

However, still getting substantial memory leak every time I click a link in my web browser.

Here's a example - I ran the command once, then clicked on a link which loaded a new web page and then I ran it again:

0 jamie@animal:~$ ps -e -o rss,cmd | egrep '[X]pra' | grep ice
148928 /usr/bin/Xorg-for-Xpra-:201 -dpi 96 -noreset -nolisten tcp +extension GLX +extension RANDR +extension RENDER -logfile /home/iceweasel/.xpra/Xorg.:201.log -config /etc/xpra/xorg.conf :201
0 jamie@animal:~$ ps -e -o rss,cmd | egrep '[X]pra' | grep ice
151040 /usr/bin/Xorg-for-Xpra-:201 -dpi 96 -noreset -nolisten tcp +extension GLX +extension RANDR +extension RENDER -logfile /home/iceweasel/.xpra/Xorg.:201.log -config /etc/xpra/xorg.conf :201
0 jamie@animal:~$

Fri, 04 Apr 2014 14:58:49 GMT - jamie: attachment set

Server log with XPRA_DETECT_LEAKS=1


Fri, 04 Apr 2014 14:59:41 GMT - jamie:

I just attached a server log with XPRA_DETECT_LEAKS=1 - let me know if there is any more debugging I can do that would be helpful.


Sat, 05 Apr 2014 01:19:54 GMT - Antoine Martin:

@jamie: I have only just noticed that the process whose memory you are claiming is going up is the X11 vfb process and not xpra itself. The X11 server will allocate memory whenever the client application (firefox or other, xpra too) requests any resources. xpra's allocations will go away if you exit the xpra server process (use "xpra exit") - problem with this is that we may also unmap some of the client windows on exit IIRC, so this will free some resources we should not really be accountable for..

Even if it was the xpra process, the memory going up by just 3MB when doing something is not in itself evidence of a leak, only if the increase is persistent. Because of python's garbage collection, the memory will fluctuate, it is the long term trend that is more telling. We also collect statistics, delta buffers, video encoders, etc... and those all take time to initialize and populate fully.


Sat, 05 Apr 2014 05:50:32 GMT - Antoine Martin:

Running for over 30 minutes with the 0.12.x branch as of r6036, and I saw no real change with mmap:

Or with plain rgb:

So... I just cannot reproduce any leaks. What am I missing?


Sat, 05 Apr 2014 09:40:05 GMT - Antoine Martin: attachment set

patch I've used to verify that we are not leaking xshm image wrappers


Mon, 07 Apr 2014 05:10:03 GMT - Antoine Martin:

Another idea, please disable (or even remove) OpenCL support, either:

Does that make any difference? Some of the OpenCL ICDs (IIRC, AMD's in particular) cause problems with threads and signals when loaded.


Mon, 07 Apr 2014 13:21:03 GMT - jamie:

Replying to totaam:

@jamie: I have only just noticed that the process whose memory you are claiming is going up is the X11 vfb process and not xpra itself. The X11 server will allocate memory whenever the client application (firefox or other, xpra too) requests any resources. xpra's allocations will go away if you exit the xpra server process (use "xpra exit") - problem with this is that we may also unmap some of the client windows on exit IIRC, so this will free some resources we should not really be accountable for..

Yes! That is definitely true. Sorry - I didn't realize it was not xpra itself. Should I file this against xvfb? Are there other steps you might suggest to help keep this memory usage under control or any other ways of running xpra to avoid it?

Even if it was the xpra process, the memory going up by just 3MB when doing something is not in itself evidence of a leak, only if the increase is persistent.

I'm so sorry to waste your time on this - there is minimal increase in memory in the actual xpra server that I am noticing (the xpra client does go up, but nothing like the X11 vfb process) but nothing worth reporting and I imagine it is fluctuating like you say.

I think we should close this bug.


Mon, 07 Apr 2014 13:37:38 GMT - onlyjob:

I agree, let's close this bug for now. I can't reproduce the leak any more. First thing I tried to do is to uninstall "python-webp" on server side. When leak did not manifest for a while I installed patched version "0.2.2-3" of "python-webp" (on Debian) which was also built against "libwebp5". Leak was there with "python-webp-0.2.2-2/libwebp4". If leak was related to libwebp4 or to python-webp prior to 0.2.2-3 then perhaps it was fixed somewhere...


Mon, 07 Apr 2014 13:42:28 GMT - Antoine Martin:

Should I file this against xvfb?


Not necessarily, it could still be xpra consuming those X11 server resources.

Does stopping the browser whilst xpra is running restore all the memory? Does turning off xshm help (it will reduce the memory used, but does it affect the rate significantly?):

XPRA_XSHM=0 xpra start ...

Mon, 07 Apr 2014 14:18:07 GMT - jamie:

When closing the browser, it goes down a little (from 146064 kb to 145936) but doesn't restore all of it.

Adding XPRA_XSHM=0 doesn't seem to have any effect on the memory.


Tue, 08 Apr 2014 03:42:15 GMT - Antoine Martin:

When closing the browser, it goes down a little (from 146064 kb to 145936) but doesn't restore all of it.


I would have expected it to go down a lot more. How much does it go down by if you exit the xpra server? (using xpra exit)


Tue, 08 Apr 2014 14:15:26 GMT - jamie:

Here's memory usage after three or four clicks:

0 jamie@animal:~$ ps -e -o rss,cmd | grep Xpra | grep iceweasel
182752 /usr/bin/Xorg-for-Xpra-:201 -dpi 96 -noreset -nolisten tcp +extension GLX +extension RANDR +extension RENDER -logfile /home/iceweasel/.xpra/Xorg.:201.log -config /etc/xpra/xorg.conf :201
0 jamie@animal:~$

Here's memory usage after xpra exit:

0 jamie@animal:~$ ps -e -o rss,cmd | grep Xpra | grep iceweasel
136308 /usr/bin/Xorg-for-Xpra-:201 -dpi 96 -noreset -nolisten tcp +extension GLX +extension RANDR +extension RENDER -logfile /home/iceweasel/.xpra/Xorg.:201.log -config /etc/xpra/xorg.conf :201
0 jamie@animal:~$

Thu, 10 Jul 2014 16:28:00 GMT - Antoine Martin: owner, status changed

Sorry for taking so long to get onto this bug, should be fixed in r6874. (easily spotted with xrestop) Further improvements in r6875 and r6876. (will backport at least the first one)

@onlyjob / @jamie: can I close this ticket?


Sun, 20 Jul 2014 02:14:22 GMT - onlyjob: status changed; resolution set

Yes let's close please. I haven't seen this issue for a while.


Sat, 23 Jan 2021 04:58:38 GMT - migration script:

this ticket has been moved to: https://github.com/Xpra-org/xpra/issues/535