Xpra: Ticket #2491: macos opengl paint error

Was just running xpra keyboard-test trying to reproduce #2480.

client   1 @25.820 Error painting planar update
client   1 @25.820   File "/Users/osx/Desktop/Xpra.app/Contents/Resources/lib/python/xpra/client/gl/gl_window_backing_base.py", line 1113, in gl_paint_planar
client   1 @25.820     self.update_planar_textures(enc_width, enc_height, img, pixel_format, scaling=scaling)
client   1 @25.820   File "/Users/osx/Desktop/Xpra.app/Contents/Resources/lib/python/xpra/client/gl/gl_window_backing_base.py", line 1190, in update_planar_textures
client   1 @25.820     glTexSubImage2D(target, 0, 0, 0, w, h, GL_LUMINANCE, GL_UNSIGNED_BYTE, pixel_data)
client   1 @25.820   File "latebind.pyx", line 32, in OpenGL_accelerate.latebind.LateBind.__call__ (src/latebind.c:989)
client   1 @25.820   File "wrapper.pyx", line 318, in OpenGL_accelerate.wrapper.Wrapper.__call__ (src/wrapper.c:6561)
client   1 @25.820 GLError: GLError(
client   1 @25.820 	err = 1282,
client   1 @25.820 	description = 'invalid operation',
client   1 @25.820 	baseOperation = glTexSubImage2D,
client   1 @25.820 	pyArgs = (
client   1 @25.820 		GL_TEXTURE_RECTANGLE_ARB,
client   1 @25.820 		0,
client   1 @25.820 		0,
client   1 @25.820 		0,
client   1 @25.820 		500,
client   1 @25.820 		274,
client   1 @25.820 		GL_LUMINANCE,
client   1 @25.820 		GL_UNSIGNED_BYTE,
client   1 @25.820 		<memory at 0x11fc289b0>,
client   1 @25.820 	),
client   1 @25.820 	cArgs = (
client   1 @25.820 		GL_TEXTURE_RECTANGLE_ARB,
client   1 @25.820 		0,
client   1 @25.820 		0,
client   1 @25.820 		0,
client   1 @25.820 		500,
client   1 @25.820 		274,
client   1 @25.820 		GL_LUMINANCE,
client   1 @25.820 		GL_UNSIGNED_BYTE,
client   1 @25.820 		<memory at 0x11fc289b0>,
client   1 @25.820 	),
client   1 @25.820 	cArguments = (
client   1 @25.820 		GL_TEXTURE_RECTANGLE_ARB,
client   1 @25.820 		0,
client   1 @25.820 		0,
client   1 @25.820 		0,
client   1 @25.820 		500,
client   1 @25.820 		274,
client   1 @25.820 		GL_LUMINANCE,
client   1 @25.820 		GL_UNSIGNED_BYTE,
client   1 @25.820 		<memory at 0x11fc289b0>,
client   1 @25.820 	)
client   1 @25.820 )
Warning: client decoding error:
 OpenGL h264 paint failed: GLError( err=1282, description = 'invalid operation', baseOperation = glTexSubImage2D )
client   1 @25.828  flush=0, image=AVImageWrapper-AVFrameWrapper(0x7f98f8cdb540)(GBRP:(0, 0, 500, 274, 24):3_PLANES), coords=(20, 170, 500, 274), size=500x274
client   1 @25.938 Error painting planar update
client   1 @25.938   File "/Users/osx/Desktop/Xpra.app/Contents/Resources/lib/python/xpra/client/gl/gl_window_backing_base.py", line 1113, in gl_paint_planar
client   1 @25.938     self.update_planar_textures(enc_width, enc_height, img, pixel_format, scaling=scaling)
client   1 @25.938   File "/Users/osx/Desktop/Xpra.app/Contents/Resources/lib/python/xpra/client/gl/gl_window_backing_base.py", line 1190, in update_planar_textures
client   1 @25.938     glTexSubImage2D(target, 0, 0, 0, w, h, GL_LUMINANCE, GL_UNSIGNED_BYTE, pixel_data)
client   1 @25.938   File "latebind.pyx", line 32, in OpenGL_accelerate.latebind.LateBind.__call__ (src/latebind.c:989)
client   1 @25.938   File "wrapper.pyx", line 318, in OpenGL_accelerate.wrapper.Wrapper.__call__ (src/wrapper.c:6561)
client   1 @25.938 GLError: GLError(
client   1 @25.938 	err = 1282,
client   1 @25.938 	description = 'invalid operation',
client   1 @25.938 	baseOperation = glTexSubImage2D,
client   1 @25.938 	pyArgs = (
client   1 @25.938 		GL_TEXTURE_RECTANGLE_ARB,
client   1 @25.938 		0,
client   1 @25.938 		0,
client   1 @25.938 		0,
client   1 @25.938 		500,
client   1 @25.938 		274,
client   1 @25.938 		GL_LUMINANCE,
client   1 @25.938 		GL_UNSIGNED_BYTE,
client   1 @25.938 		<memory at 0x11fc28b90>,
client   1 @25.938 	),
client   1 @25.938 	cArgs = (
client   1 @25.938 		GL_TEXTURE_RECTANGLE_ARB,
client   1 @25.938 		0,
client   1 @25.938 		0,
client   1 @25.938 		0,
client   1 @25.938 		500,
client   1 @25.938 		274,
client   1 @25.938 		GL_LUMINANCE,
client   1 @25.938 		GL_UNSIGNED_BYTE,
client   1 @25.938 		<memory at 0x11fc28b90>,
client   1 @25.938 	),
client   1 @25.938 	cArguments = (
client   1 @25.938 		GL_TEXTURE_RECTANGLE_ARB,
client   1 @25.938 		0,
client   1 @25.938 		0,
client   1 @25.938 		0,
client   1 @25.938 		500,
client   1 @25.938 		274,
client   1 @25.938 		GL_LUMINANCE,
client   1 @25.938 		GL_UNSIGNED_BYTE,
client   1 @25.938 		<memory at 0x11fc28b90>,
client   1 @25.938 	)
client   1 @25.938 )
client   1 @25.947  flush=0, image=AVImageWrapper-AVFrameWrapper(0x7f98fc3e9000)(GBRP:(0, 0, 500, 274, 24):3_PLANES), coords=(20, 170, 500, 274), size=500x274
(..)
Warning: client decoding error:
 OpenGL RGB paint failed: GLError( err=1282, description = 'invalid operation', baseOperation = glEnd )
client   1 @26.588 Error painting planar update
client   1 @26.588   File "/Users/osx/Desktop/Xpra.app/Contents/Resources/lib/python/xpra/client/gl/gl_window_backing_base.py", line 1121, in gl_paint_planar
client   1 @26.588     self.render_planar_update(x, y, enc_width, enc_height, x_scale, y_scale, shader)
client   1 @26.588   File "/Users/osx/Desktop/Xpra.app/Contents/Resources/lib/python/xpra/client/gl/gl_window_backing_base.py", line 1218, in render_planar_update
client   1 @26.588     glEnd()
client   1 @26.588   File "latebind.pyx", line 44, in OpenGL_accelerate.latebind.Curry.__call__ (src/latebind.c:1201)
client   1 @26.588   File "/Users/osx/Desktop/Xpra.app/Contents/Resources/lib/python/site-packages.zip/OpenGL/GL/exceptional.py", line 45, in glEnd
client   1 @26.588     return baseFunction( )
client   1 @26.588   File "errorchecker.pyx", line 53, in OpenGL_accelerate.errorchecker._ErrorChecker.glCheckError (src/errorchecker.c:1218)
client   1 @26.588 GLError: GLError(
client   1 @26.588 	err = 1282,
client   1 @26.588 	description = 'invalid operation',
client   1 @26.588 	baseOperation = glEnd,
client   1 @26.588 	cArguments = ()
client   1 @26.588 )
client   1 @26.596  flush=0, image=AVImageWrapper-AVFrameWrapper(0x7f98fc3e8900)(GBRP:(0, 0, 504, 236, 24):3_PLANES), coords=(20, 151, 504, 236), size=504x236
Warning: client decoding error:
 OpenGL h264 paint failed: GLError( err=1282, description = 'invalid operation', baseOperation = glEnd )
client   1 @26.685 Error painting planar update
client   1 @26.685   File "/Users/osx/Desktop/Xpra.app/Contents/Resources/lib/python/xpra/client/gl/gl_window_backing_base.py", line 1121, in gl_paint_planar
client   1 @26.685     self.render_planar_update(x, y, enc_width, enc_height, x_scale, y_scale, shader)
client   1 @26.685   File "/Users/osx/Desktop/Xpra.app/Contents/Resources/lib/python/xpra/client/gl/gl_window_backing_base.py", line 1218, in render_planar_update
client   1 @26.685     glEnd()
client   1 @26.685   File "latebind.pyx", line 44, in OpenGL_accelerate.latebind.Curry.__call__ (src/latebind.c:1201)
client   1 @26.685   File "/Users/osx/Desktop/Xpra.app/Contents/Resources/lib/python/site-packages.zip/OpenGL/GL/exceptional.py", line 45, in glEnd
client   1 @26.685     return baseFunction( )
client   1 @26.685   File "errorchecker.pyx", line 53, in OpenGL_accelerate.errorchecker._ErrorChecker.glCheckError (src/errorchecker.c:1218)
client   1 @26.685 GLError: GLError(
client   1 @26.685 	err = 1282,
client   1 @26.685 	description = 'invalid operation',
client   1 @26.685 	baseOperation = glEnd,
client   1 @26.685 	cArguments = ()
client   1 @26.685 )
client   1 @26.694  flush=0, image=AVImageWrapper-AVFrameWrapper(0x7f98fb0f4c40)(GBRP:(0, 0, 504, 236, 24):3_PLANES), coords=(20, 151, 504, 236), size=504x236

This could well be related to the changes from #2481



Fri, 22 Nov 2019 09:37:59 GMT - Antoine Martin: priority, status, version, description changed


Fri, 29 Nov 2019 10:54:27 GMT - Antoine Martin:

Reproducible if the window is created from an xterm, but re-connecting to the same session where the window is already shown does not show any errors!?

Also not present in v3?

With -d draw,paint,opengl:

gl_paint_planar(0, 'jpeg', ImageWrapper(YUV444P:(0, 0, 540, 667, 24):PACKED), 0, 0, 540, 667, 540, 667, [<function WindowClient._do_draw.<locals>.record_decode_time at 0x11cb80f80>, <bound method ClientWindowBase.after_draw_refresh of GLClientWindow(3 : GLDrawingArea(3, (540, 667), None))>])
GLDrawingArea(3, (540, 667), None).update_planar_textures(540, 667, ImageWrapper(YUV444P:(0, 0, 540, 667, 24):PACKED), 'YUV444P')
Creating new planar textures, pixel format YUV444P
updating planar textures: 540x667 YUV444P
set_alignment(540, 540, 'Y') GL_UNPACK_ROW_LENGTH=0, GL_UNPACK_ALIGNMENT=4
texture 0: div=(1, 1), rowstride=540, 540x667, data=360180 bytes, upload=zerocopy:memoryview
set_alignment(540, 540, 'U') GL_UNPACK_ROW_LENGTH=0, GL_UNPACK_ALIGNMENT=4
texture 1: div=(1, 1), rowstride=540, 540x667, data=360180 bytes, upload=zerocopy:memoryview
set_alignment(540, 540, 'V') GL_UNPACK_ROW_LENGTH=0, GL_UNPACK_ALIGNMENT=4
texture 2: div=(1, 1), rowstride=540, 540x667, data=360180 bytes, upload=zerocopy:memoryview
GLDrawingArea(3, (540, 667), YUV444P).render_planar_update(0, 0, 540, 667, 1, 1, 2) pixel_format=YUV444P
painting planar update, format YUV444P
GLDrawingArea(3, (540, 667), YUV444P).render_planar_update(..) texture_size=(540, 667), size=(540, 667)
Error painting planar update
  File "/Users/gtk3/Desktop/Xpra.app/Contents/Resources/lib/python/xpra/client/gl/gl_window_backing_base.py", line 1101, in gl_paint_planar
    self.render_planar_update(x, y, enc_width, enc_height, x_scale, y_scale, shader)
  File "/Users/gtk3/Desktop/Xpra.app/Contents/Resources/lib/python/xpra/client/gl/gl_window_backing_base.py", line 1199, in render_planar_update
    glEnd()
  File "src/latebind.pyx", line 44, in OpenGL_accelerate.latebind.Curry.__call__
  File "/Users/gtk3/Desktop/Xpra.app/Contents/Resources/lib/python/site-packages.zip/OpenGL/GL/exceptional.py", line 45, in glEnd
    return baseFunction( )
  File "src/errorchecker.pyx", line 53, in OpenGL_accelerate.errorchecker._ErrorChecker.glCheckError
Warning: client decoding error:
GLError: GLError(
 OpenGL jpeg paint failed: GLError( err=1282, description = b'invalid operation', baseOperation = glEnd )
	err = 1282,
	description = b'invalid operation',
	baseOperation = glEnd,
	cArguments = ()
)
 flush=0, image=ImageWrapper(YUV444P:(0, 0, 540, 667, 24):PACKED), coords=(0, 0, 540, 667), size=540x667
record_decode_time(False, OpenGL jpeg paint failed: GLError( err=1282, description = b'invalid operation', baseOperation = glEnd )) decoding error on wid=3, b'jpeg': 540x667
sending ack: ('damage-sequence', 1, 3, 540, 667, -1, "OpenGL jpeg paint failed: GLError( err=1282, description = b'invalid operation', baseOperation = glEnd )")
after_draw_refresh(False, OpenGL jpeg paint failed: GLError( err=1282, description = b'invalid operation', baseOperation = glEnd )) pending_refresh=[]

Fri, 29 Nov 2019 17:22:01 GMT - Antoine Martin:

Easy to reproduce with keyboard-test and --encodings=jpeg.

Adding debug logging to the glMultiTexCoord2i and glVertex2i calls found in the glBegin / glEnd block does not show any invalid values. Could be related to the changes of how / when we call present_fbo but XPRA_OPENGL_DRAW_REFRESH=0 does not help.

Even when this bug is fixed, maybe we should re-initialize the window in case of paint errors? Maybe the opengl probe should use a JPEG YUV paint?

What is the difference between a new window and one we just connected to? It is the very first paint that fails. Keeping jpeg but switching to XPRA_JPEG_YUV=0 works OK. Delaying the first YUV paint using a counter does not help.


Sun, 01 Dec 2019 17:37:08 GMT - Antoine Martin:

Duplicate ticket: #2499.

Bisection:

...

Not so easy then as the range from r23800 to ~r24250 cannot be bisected.

Trying to find the fix for the "runtime opengl problem", to then be able to bisect before r24250:

Looks like we need r24188 + r24189 + r24191


Mon, 02 Dec 2019 17:24:30 GMT - Antoine Martin:

Restarting bisection (and adding +r24188 + r24189 + r24191):

Trying with v2.5.x: needs r23527 to be able to run the packaging scripts, r24575 automates this. v2.5.3-r24575 works OK (latest from 2.5.x branch) So the regression happened somewhere between r22133 (tag v2.5.x) and 3.0.3.


Tue, 03 Dec 2019 04:19:17 GMT - Antoine Martin: attachment set

updated patch for building trunk circa 3.0-dev


Tue, 03 Dec 2019 06:38:20 GMT - Antoine Martin:

Bisecting again, with the bundle patch attached:

r23560 is the problem, and it's not easy to split into parts or revert.


Tue, 03 Dec 2019 12:33:02 GMT - Antoine Martin:

Splitting the patch and applying piece by piece. No problems with these parts of r23560 (some for obvious reasons since unused, ie: gtk2, gtkgl):

The last few are harder to untangle. gtk3/gtk3_client_window.py started as a copy of gtk3/client_window.py. Applying this part of the change triggers the bug: browser/xpra/trunk/src/xpra/client/gl/gtk3/gl_client_window.py. But unsurprisingly, keeping gtk3/gtk3_client_window.py as a verbatim copy of gtk3/client_window.py works OK. So the problem comes from the seemingly innocuous changes here: changeset/23560/xpra

Applying piece by piece, it is the init_drawing_area vs init_widget_events change that breaks things. Specifically, overriding init_widget_events and registering the "draw" event callback.

This ensures we use our custom code for painting the drawing area widget, and this calls backing.cairo_draw which calls present_fbo with the opengl backend. Skipping cairo_draw also works. The updates end up on screen thanks to queue_draw_area which calls gl_expose_rect. Removing the queue_draw_area overrides from both gl_client_window and gtk3_client_window and letting the normal GTK code call down to our drawing_area_draw also triggers the error.. Again, only reliably with a new window, not when connecting to an existing window!?!


Tue, 03 Dec 2019 16:27:06 GMT - Antoine Martin:

Interestingly, XPRA_OPENGL_DRAW_REFRESH=0 triggers the bug more.

Using the big hammer, looking for log differences with -d opengl,metadata,geometry,state,window:

Disabling the cursor code makes no difference. The two AGLWindowContext instances look suspicious: the GdkQuartzWindow they're for is not the same!? One is preceded by a clip_to_backing which is not the case when things work OK.

Turns out that this is due to the "context-reinit" workaround added in r23441 for macos: #2372, so smooth resizing #478 and window gravity #2217 will need to be re-tested. We can't change the value of the flag because then window contents look weird after resizing. "context-reinit" needs to be fixed or fbo resizing made to work on macos.. no easy fix!


Wed, 04 Dec 2019 11:21:00 GMT - Antoine Martin:

Fixing "fbo resizing" would also fix #2373.


Wed, 04 Dec 2019 14:30:32 GMT - Antoine Martin:

Found the solution here: NSOpenGLContext.update(). So r24590 fixes "fbo resizing" on macos, and r24591 disables "context reinit" mode. One more thing needed: we need to repaint the outer edge of the window which is not part of the drawing area.


Thu, 05 Dec 2019 12:47:01 GMT - Antoine Martin: status changed; resolution set

Not going to worry about the outer edge as it's pretty difficult to trigger it and there's nothing to see there: it only happens when there's a geometry mismatch and the client geometry is bigger than it should be because GTK3 didn't honour our geometry constraints...


Sat, 23 Jan 2021 05:52:44 GMT - migration script:

this ticket has been moved to: https://github.com/Xpra-org/xpra/issues/2491