Xpra: Ticket #314: VirtualBox win32 clients can crash hard just doing the initial OpenGL probing

When we start the application without specifying "--opengl=no" (or via the config file equivallent: opengl=no), the xpra.gl.gl_check class does some driver probing to see if the system's OpenGL driver is up to the task, checking:

I was hoping that blacklisting "Humper" (done as part of r3006) would be enough to prevent the hard VM crashes that I had been seeing, unfortunately that was not the case.
So in r3073, we try to detect if we are running in VirtualBox, and if so we set the default for the opengl option to False, this prevents the driver probing and avoids the crashes.
The (unavoidable?) problem with this approach is that if and when VirtualBox fix the driver issue, we will still end up ignoring it and OpenGL will remain disabled.. Unless we can somehow load the driver version from the DLL and detect the versions that do work?
Interestingly, running the OpenGL_test.exe command, which is installed as part of xpra and runs the exact same code, does not seem to be able to crash the VM - though I did get this error instead (you have to run it many many times to hit errors):

Looks like a bad pointer somewhere. Note: I reproduced the exact same values above twice..

The VBox Additions source can be found here, this assertion is in packspu_VBoxDetachThread:

for (i=0; i<CR_MAX_CONTEXTS; ++i)
                ContextInfo *ctx = &pack_spu.context[i];
                if (ctx->currentThread == thread)
                    ctx->currentThread = NULL;

Looks like it expects fAutoFlush to be true because the thread matches. Maybe when we cleanup we end up closing the window from the wrong thread.
AFAICT, this happens with all versions of VirtualBox, up to and including 4.2.12

This happens with both an nvidia host driver and radeon, here are the log contents:

00:00:16.643984 Display::handleDisplayResize(): uScreenId = 0, pvVRAM=00007fbf1c000000 w=1378 h=1050 bpp=24 cbLine=0x1028, flags=0
00:00:16.687557 2D video acceleration is enabled.
00:00:21.642813 PCNet#0: The link is back up again after the restore.
03:59:02.713599 Switching Gl mode on
03:59:02.886585 OpenGL Info: Render SPU: GL_VENDOR:   ATI Technologies Inc.
03:59:02.886692 OpenGL Info: Render SPU: GL_RENDERER: ATI Radeon HD 5450
03:59:02.886716 OpenGL Info: Render SPU: GL_VERSION:  4.2.12002 Compatibility Profile Context 9.002
03:59:02.886770 OpenGL Info: Render SPU: GL_EXTENSIONS: GL_AMDX_debug_output (...)
03:59:22.734485 Switching Gl mode off
04:00:14.899835 Switching Gl mode on

Absolutely nothing when it crashes, and nothing on the host's "dmesg" either. Now running with gdb attached to see if I can get a backtrace.

Probably unrelated, but I am also seeing these errors after closing a window:

2013-04-16 10:29:30,894 OpenGL paint error: GLError(
        err = 1282,
        description = 'invalid operation',
        baseOperation = glEnd,
        cArguments = ()
Traceback (most recent call last):
  File "xpra\gl\gl_window_backing.pyc", line 210, in do_gl_paint
  File "xpra\gl\gl_window_backing.pyc", line 309, in render_image
  File "latebind.pyx", line 44, in OpenGL_accelerate.latebind.Curry.__call__ (src\latebind.c:832)
  File "C:\Program Files\Xpra\OpenGL\GL\exceptional.py", line 57, in glEnd
  File "errorchecker.pyx", line 50, in OpenGL_accelerate.errorchecker._ErrorChecker.glCheckError (src\errorchecker.c:854

Finally a link to virtual box "opengl crash" bugs - and there are quite a few, seemingly related:

Also it seems that some bugs require the VM to be suspended then resumed to trigger. Indeed, vbox failed to suspend my VM if an opengl window was active when I suspended it..

Tue, 16 Apr 2013 07:10:32 GMT - Antoine Martin: owner, status, description changed

Tue, 16 Apr 2013 08:53:42 GMT - Antoine Martin: description changed

Tue, 16 Apr 2013 08:59:56 GMT - Antoine Martin: description changed

Tue, 16 Apr 2013 09:34:45 GMT - Antoine Martin: description changed

Tue, 16 Apr 2013 09:48:10 GMT - Antoine Martin: description changed

Tue, 19 Aug 2014 04:01:02 GMT - Antoine Martin: status changed; resolution set

Newer versions of virtualbox have improved a lot, they no longer crash when probing and no longer get blacklisted.

I think we're doing enough to workaround problems that aren't really ours to solve. Closing.

Tue, 03 Mar 2015 08:18:17 GMT - Antoine Martin:

FWIW: I've been running XP in the latest virtualbox with opengl enabled, works very well... I wished there was a way to detect if it is safe to use.

Sat, 24 Oct 2015 05:04:06 GMT - Antoine Martin:

Note: in 0.16.0, we allow opengl to be enabled with virtualbox guests, it just starts disabled by default (except with Ubuntu 12.04 where we just don't allow it at all - as it is just too buggy)

Mon, 20 Feb 2017 12:17:28 GMT - Antoine Martin: version set

See also #1367

Thu, 05 Mar 2020 12:02:54 GMT - Antoine Martin: milestone changed

Sat, 23 Jan 2021 04:51:26 GMT - migration script:

this ticket has been moved to: https://github.com/Xpra-org/xpra/issues/314