html5 client visual corruption

[Antoine Martin]

Seen with the win32 shadow server as tested for #2124.
Looks like some screen updates go missing. Likely to be scroll or video updates.

Links that may be related:

• ticket:1426#comment:8 - html5 visual corruption
• #2053: web client does not update regions...
• #2060: html5 h264 broadway decoder picture dimension is rounded up
• #2051: nvenc produces invalid stream - unlikely, reproducible without nvenc
• #1992: html5 scroll paint error
• #1637: Scroll paint bug in HTML5 Client
• #1432: html5 client should honour paint flush order

[Antoine Martin]

It seems that this only occurs reliably with the python3 version of the server.

• with a python2 server, the server -d compress with html5 client "paint" debug enabled looks like this:

  2019-03-10 09:03:35,967 Handshake complete; enabling connection
  2019-03-10 09:03:35,967  automatic picture encoding enabled, also available:
  2019-03-10 09:03:35,967   jpeg, png, rgb32, webp, h264
  2019-03-10 09:03:36,107 HTML5 Linux Chrome client version 2.5
  2019-03-10 09:03:36,123 shadow server: setting default keymap translation
  2019-03-10 09:03:36,123  client root window size is 2039x1415
  2019-03-10 09:03:36,170 client got hello: server version 2.5 accepted our connection
  2019-03-10 09:03:36,170 client connection_progress( Session started ,   ,  100 )
  2019-03-10 09:03:36,185 client startup complete
  2019-03-10 09:03:36,185 client connection-established
  2019-03-10 09:03:36,279 compress:  16.0ms for 1648x1060 pixels at    0,0    for wid=1     using      h264 with ratio   4.7%  ( 6823KB to   319KB), sequence     1, client_options={u'type': u'IDR', u'frame': 0, u'pts': 0L, 'csc': u'YUV420P', 'scaled_size': (824, 530)}
  2019-03-10 09:03:36,295 client draw may_paint_now() paint pending= 0 , paint queue length= 1
  2019-03-10 09:03:36,295 client draw do_paint( 326914  bytes of   h264  data  1648 x 1060  at  0 , 0 ) focused= true
  2019-03-10 09:03:36,342 client broadway decoder initialized
  2019-03-10 09:03:36,717 client draw broadway picture decoded:  1810432 bytes, size  832 x 544, paint location:  0,0 with infos= undefined
  2019-03-10 09:03:36,717 client draw request_redraw for Window(1)
  2019-03-10 09:03:36,717 client draw decode time for  h264  sequence  1 :  423 , flush= 0
  2019-03-10 09:03:36,717 client draw animation frame: 1 windows to paint
  2019-03-10 09:03:36,732 compress:   0.0ms for 1648x40   pixels at    0,1020 for wid=1     using      webp with ratio   8.5%  (  257KB to    21KB), sequence     3, client_options={u'quality': 100, u'rgb_format': 'BGRX'}
  2019-03-10 09:03:36,748 client draw may_paint_now() paint pending= 0 , paint queue length= 1
  2019-03-10 09:03:36,748 client draw do_paint( 22346  bytes of   webp  data  1648 x 40  at  0 , 1020 ) focused= true
  2019-03-10 09:03:36,763 client draw request_redraw for Window(1)
  2019-03-10 09:03:36,763 client draw decode time for  webp  sequence  2 :  4 , flush= 0
  2019-03-10 09:03:36,763 client draw may_paint_now() paint pending= 0 , paint queue length= 0
  2019-03-10 09:03:36,763 client draw animation frame: 1 windows to paint
  2019-03-10 09:03:37,170 Error: cannot connect to named pipe '\\.\pipe\Xpra\Main'
  2019-03-10 09:03:37,498 compress:  16.0ms for 1648x1060 pixels at    0,0    for wid=1     using      h264 with ratio   0.1%  ( 6823KB to     9KB), sequence     3, client_options={u'type': u'P', u'frame': 1, u'pts': 1219L, 'csc': u'YUV420P', 'scaled_size': (824, 530)}
  2019-03-10 09:03:37,513 client draw may_paint_now() paint pending= 0 , paint queue length= 1
  2019-03-10 09:03:37,513 client draw do_paint( 9384  bytes of   h264  data  1648 x 1060  at  0 , 0 ) focused= true
  2019-03-10 09:03:37,529 client draw broadway picture decoded:  1810432 bytes, size  832 x 544, paint location:  0,0 with infos= undefined
  2019-03-10 09:03:37,529 client draw request_redraw for Window(1)
  2019-03-10 09:03:37,529 client draw decode time for  h264  sequence  3 :  16 , flush= 0
  2019-03-10 09:03:37,529 client draw animation frame: 1 windows to paint
  2019-03-10 09:03:38,045 compress:   0.0ms for 1648x14   pixels at    0,989  for wid=1     using      webp with ratio   1.0%  (   90KB to     0KB), sequence     5, client_options={u'quality': 100, u'rgb_format': 'BGRX'}
  2019-03-10 09:03:38,107 client draw may_paint_now() paint pending= 0 , paint queue length= 1
  2019-03-10 09:03:38,107 client draw do_paint( 944  bytes of   webp  data  1648 x 14  at  0 , 989 ) focused= true
  2019-03-10 09:03:38,107 client draw request_redraw for Window(1)
  2019-03-10 09:03:38,107 client draw decode time for  webp  sequence  4 :  2 , flush= 0
  2019-03-10 09:03:38,107 client draw may_paint_now() paint pending= 0 , paint queue length= 0
  2019-03-10 09:03:38,123 client draw animation frame: 1 windows to paint
  2019-03-10 09:03:38,545 compress:   0.0ms for 1648x14   pixels at    0,989  for wid=1     using      webp with ratio   1.0%  (   90KB to     0KB), sequence     6, client_options={u'quality': 100, u'rgb_format': 'BGRX'}
  

Same with python3 server:

2019-03-10 09:06:59,562 Handshake complete; enabling connection
2019-03-10 09:06:59,562  automatic picture encoding enabled, also available:
2019-03-10 09:06:59,562   jpeg, png, rgb32, webp, h264
2019-03-10 09:06:59,578 HTML5 Linux Chrome client version 2.5
2019-03-10 09:06:59,593 shadow server: setting default keymap translation
2019-03-10 09:06:59,593  client root window size is 1730x1415
2019-03-10 09:06:59,641 client got hello: server version 2.5 accepted our connection
2019-03-10 09:06:59,641 client connection_progress( Session started ,   ,  100 )
2019-03-10 09:06:59,657 client startup complete
2019-03-10 09:06:59,657 client connection-established
2019-03-10 09:06:59,735 compress:  15.0ms for 1648x1060 pixels at    0,0    for wid=1     using      h264 with ratio   2.6%  ( 6823KB to   179KB), sequence     1, client_options={'frame': 0, 'pts': 0, 'type': 'IDR', 'csc': 'YUV420P', 'scaled_size': (824, 530)}
2019-03-10 09:06:59,750 client draw may_paint_now() paint pending= 0 , paint queue length= 1
2019-03-10 09:06:59,750 client draw do_paint( 183530  bytes of   h264  data  1648 x 1060  at  0 , 0 ) focused= true
2019-03-10 09:06:59,782 client broadway decoder initialized
2019-03-10 09:07:00,000 client draw request_redraw for Window(1)
2019-03-10 09:07:00,016 client draw decode time for  h264  sequence  1 :  264 , flush= 0
2019-03-10 09:07:00,016 client draw animation frame: 1 windows to paint
2019-03-10 09:07:00,047 compress:   0.0ms for 1648x14   pixels at    0,989  for wid=1     using     rgb32 with ratio   1.5%  (   90KB to     1KB), sequence     3, client_options={'rgb_format': 'BGRX', 'lz4': 1, 'flush': 1}
2019-03-10 09:07:00,047 compress:   0.0ms for 1648x40   pixels at    0,1020 for wid=1     using      webp with ratio   8.4%  (  257KB to    21KB), sequence     4, client_options={'rgb_format': 'BGRX', 'quality': 100}
2019-03-10 09:07:00,047 client draw may_paint_now() paint pending= 0 , paint queue length= 1
2019-03-10 09:07:00,047 client draw do_paint( 1415  bytes of   rgb32  data  1648 x 14  at  0 , 989 ) focused= true
2019-03-10 09:07:00,063 client draw got  92288 to paint with stride 6592
2019-03-10 09:07:00,063 client draw decode time for  rgb32  sequence  2 :  8 , flush= 1
2019-03-10 09:07:00,063 client draw may_paint_now() paint pending= 0 , paint queue length= 0
2019-03-10 09:07:00,063 client draw may_paint_now() paint pending= 0 , paint queue length= 1
2019-03-10 09:07:00,063 client draw do_paint( 22268  bytes of   webp  data  1648 x 40  at  0 , 1020 ) focused= true
2019-03-10 09:07:00,079 client draw request_redraw for Window(1)
2019-03-10 09:07:00,079 client draw decode time for  webp  sequence  3 :  5 , flush= 0
2019-03-10 09:07:00,079 client draw may_paint_now() paint pending= 0 , paint queue length= 0
2019-03-10 09:07:00,094 client draw animation frame: 1 windows to paint
2019-03-10 09:07:00,407 compress:   0.0ms for 1648x14   pixels at    0,989  for wid=1     using     rgb32 with ratio   1.5%  (   90KB to     1KB), sequence     5, client_options={'rgb_format': 'BGRX', 'lz4': 1, 'flush': 1}
2019-03-10 09:07:00,407 compress:   0.0ms for 1648x9    pixels at    0,1028 for wid=1     using     rgb32 with ratio  18.1%  (   57KB to    10KB), sequence     6, client_options={'rgb_format': 'BGRX', 'lz4': 1}
2019-03-10 09:07:00,422 client draw may_paint_now() paint pending= 0 , paint queue length= 1
2019-03-10 09:07:00,422 client draw do_paint( 1424  bytes of   rgb32  data  1648 x 14  at  0 , 989 ) focused= true
2019-03-10 09:07:00,422 client draw got  92288 to paint with stride 6592
2019-03-10 09:07:00,422 client draw decode time for  rgb32  sequence  4 :  7 , flush= 1
2019-03-10 09:07:00,422 client draw may_paint_now() paint pending= 0 , paint queue length= 0
2019-03-10 09:07:00,422 client draw may_paint_now() paint pending= 0 , paint queue length= 1
2019-03-10 09:07:00,422 client draw do_paint( 10718  bytes of   rgb32  data  1648 x 9  at  0 , 1028 ) focused= true
2019-03-10 09:07:00,422 client draw got  59328 to paint with stride 6592
2019-03-10 09:07:00,422 client draw request_redraw for Window(1)
2019-03-10 09:07:00,422 client draw decode time for  rgb32  sequence  5 :  2 , flush= 0
2019-03-10 09:07:00,422 client draw may_paint_now() paint pending= 0 , paint queue length= 0
2019-03-10 09:07:00,422 client draw animation frame: 1 windows to paint
2019-03-10 09:07:00,704 compress:   0.0ms for 1648x40   pixels at    0,1020 for wid=1     using      webp with ratio   8.9%  (  257KB to    22KB), sequence     7, client_options={'rgb_format': 'BGRX', 'quality': 100}
2019-03-10 09:07:00,719 client draw may_paint_now() paint pending= 0 , paint queue length= 1
2019-03-10 09:07:00,719 client draw do_paint( 23550  bytes of   webp  data  1648 x 40  at  0 , 1020 ) focused= true
2019-03-10 09:07:00,719 client draw request_redraw for Window(1)
2019-03-10 09:07:00,719 client draw decode time for  webp  sequence  6 :  5 , flush= 0
2019-03-10 09:07:00,719 client draw may_paint_now() paint pending= 0 , paint queue length= 0
2019-03-10 09:07:00,735 client draw animation frame: 1 windows to paint
2019-03-10 09:07:00,766 compress:   0.0ms for 1648x40   pixels at    0,1020 for wid=1     using      webp with ratio   8.9%  (  257KB to    22KB), sequence     8, client_options={'rgb_format': 'BGRX', 'quality': 100}
2019-03-10 09:07:00,782 client draw may_paint_now() paint pending= 0 , paint queue length= 1
2019-03-10 09:07:00,782 client draw do_paint( 23546  bytes of   webp  data  1648 x 40  at  0 , 1020 ) focused= true
2019-03-10 09:07:00,782 client draw request_redraw for Window(1)
2019-03-10 09:07:00,782 client draw decode time for  webp  sequence  7 :  3 , flush= 0
2019-03-10 09:07:00,782 client draw may_paint_now() paint pending= 0 , paint queue length= 0
2019-03-10 09:07:00,782 client draw animation frame: 1 windows to paint
2019-03-10 09:07:00,954 compress:   0.0ms for 1648x14   pixels at    0,989  for wid=1     using     rgb32 with ratio   1.5%  (   90KB to     1KB), sequence     9, client_options={'rgb_format': 'BGRX', 'lz4': 1}
2019-03-10 09:07:00,969 client draw may_paint_now() paint pending= 0 , paint queue length= 1
2019-03-10 09:07:00,969 client draw do_paint( 1415  bytes of   rgb32  data  1648 x 14  at  0 , 989 ) focused= true
2019-03-10 09:07:00,969 client draw got  92288 to paint with stride 6592
2019-03-10 09:07:00,969 client draw request_redraw for Window(1)
2019-03-10 09:07:00,969 client draw decode time for  rgb32  sequence  8 :  1 , flush= 0
2019-03-10 09:07:00,969 client draw may_paint_now() paint pending= 0 , paint queue length= 0
2019-03-10 09:07:00,969 client draw animation frame: 1 windows to paint

So the python3 server never triggers broadway picture decoded.

[Antoine Martin]

Using XPRA_SAVE_VIDEO_STREAMS=1 xpra shadow ..., then using ffmpeg to extract an image from the first frame:

ffmpeg -i window-1-0.1-x264.h264 -frames:v 1 -f image2 frame.png

I can see that the frames in that stream are just fine.
And ffprobe shows:

$ ffprobe window-1-0.1-x264.h264 
ffprobe version 4.0.3 Copyright (c) 2007-2018 the FFmpeg developers
  built with gcc 8 (GCC)
  libavutil      56. 14.100 / 56. 14.100
  libavcodec     58. 18.100 / 58. 18.100
  libavformat    58. 12.100 / 58. 12.100
  libavdevice    58.  3.100 / 58.  5.100
  libavfilter     7. 16.100 /  7. 16.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  1.100 /  5.  1.100
  libswresample   3.  1.100 /  3.  1.100
  libpostproc    55.  1.100 / 55.  1.100
[h264 @ 0x56024fdac4c0] decoding for stream 0 failed
Input #0, h264, from 'window-1-0.1-x264.h264':
  Duration: N/A, bitrate: N/A
    Stream #0:0: Video: h264 (High), yuv420p(progressive), 824x530, 25 tbr, 1200k tbn, 50 tbc

Then with the "same" stream from a python2 server:

$ ffprobe window-1-0.1-x264.h264 
ffprobe version 4.0.3 Copyright (c) 2007-2018 the FFmpeg developers
  built with gcc 8 (GCC)
  libavutil      56. 14.100 / 56. 14.100
  libavcodec     58. 18.100 / 58. 18.100
  libavformat    58. 12.100 / 58. 12.100
  libavdevice    58.  3.100 / 58.  5.100
  libavfilter     7. 16.100 /  7. 16.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  1.100 /  5.  1.100
  libswresample   3.  1.100 /  3.  1.100
  libpostproc    55.  1.100 / 55.  1.100
[h264 @ 0x55db912104c0] decoding for stream 0 failed
Input #0, h264, from 'window-1-0.1-x264.h264':
  Duration: N/A, bitrate: N/A
    Stream #0:0: Video: h264 (Constrained Baseline), yuv420p(progressive), 824x530, 25 tbr, 1200k tbn, 50 tbc

The big difference is in the profile:

• with python3 and -d x264:

  using default profile=high
  x264: get_tune() TUNE=None, fast_decode=1, content_type=desktop
  x264 context=0x53cf9e0, YUV420P  824x530  quality=100, speed=80, content_type=desktop
  x264 params: {'me': {'type': 'DIA', 'me-range': 16, 'mv-range': -1, 'mv-range-thread': -1,
      'subpel_refine': 2, 'weighted-pred': 0}, 'rc': {'rc-method': 'CRF', 'qp_constant': -1,
      'qp_min': 0, 'qp_max': 2147483647, 'qp_step': 4, 'bitrate': 0, 'vbv_max_bitrate': 0,
      'vbv_buffer_size': 0, 'mb-tree': False, 'lookahead': -1}, 'vfr-input': False,
      'bframe-adaptive': 'FAST', 'open-gop': True, 'bluray-compat': False, 'cabac': False,
      'deblocking-filter': False, 'intra-refresh': False, 'interlaced': False,
      'constrained_intra': False, 'threads': 'auto', 'sliced-threads': True}
  x264 encode YUV420P frame     0 as  IDR slice with 4 nals, tune=b'zerolatency', total  221909 bytes, keyframe=True , delayed=0
  

• with python2

  x264: get_tune() TUNE=None, fast_decode=1, content_type=desktop
  x264 context=0x4573df0, YUV420P  824x530  quality=100, speed=80, content_type=desktop
  x264 params: {u'vfr-input': False, u'intra-refresh': False, u'deblocking-filter': False,
     u'cabac': False, u'threads': u'auto', u'bframe-adaptive': u'FAST', u'constrained_intra' : False,
     u'open-gop': False, u'interlaced': False, u'me': {u'mv-range-thread': -1, u'weighted-pred': 0,
     u'subpel_refine': 2, u'me-range': 16, u'mv-range': -1, u'type': u'DIA'}, u'sliced-threads': True,
     u'bluray-compat': False, u'rc': {u'rc-method': u'CRF', u'vbv_buffer_size': 0, u'lookahead': -1,
     u'qp_min': 0, u'qp_step': 4, u'vbv_max_bitrate': 0, u'qp_max': 2147483647, u'mb-tree': False,
     u'bitrate': 0, u'qp_constant': -1}}
  x264 encode YUV420P frame     0 as  IDR slice with 4 nals, tune=zerolatency, total  214879 bytes, keyframe=True , delayed=0
  

That's because the broadway decoder cannot handle the other profiles.

And so it is just the painful strings vs bytes from python3 (which is also a complete waste of CPU cycles):

• we have to use a typedict function to lookup the key without caring if it's a string or bytes
• we have to convert the result to a string so that it passes validation.

With python2, we don't need to worry about any of this nonsense.

Fixed in r22037.