Xpra: Ticket #736: Getting error in window_video_source after out of memory error

Seeing the following two stacktraces while testing #733:

2014-11-19 12:59:45,328 setup_pipeline failed for (66, (1, 1), None, 0, 0, None, 'BGRX', (1, 1), 600, 400, codec_spec(nvenc:nvenc4))
Traceback (most recent call last):
  File "/usr/lib64/python2.7/site-packages/xpra/server/window_video_source.py", line 1095, in setup_pipeline
    if self.setup_pipeline_option(width, height, src_format, *option):
  File "/usr/lib64/python2.7/site-packages/xpra/server/window_video_source.py", line 1156, in setup_pipeline_option
    ve.init_context(enc_width, enc_height, enc_in_format, dst_formats, encoder_spec.encoding, quality, speed, encoder_scaling, self.encoding_options)
  File "encoder.pyx", line 1254, in xpra.codecs.nvenc4.encoder.Encoder.init_context (xpra/codecs/nvenc4/encoder.c:8739)
NVENCException: opening session - returned 10: This indicates that the API call failed because it was unable to allocate enough memory to perform the requested operation.
2014-11-19 12:59:45,853 error on window 88
Traceback (most recent call last):
  File "/usr/lib64/python2.7/site-packages/xpra/server/source.py", line 362, in recalculate_delays
  File "/usr/lib64/python2.7/site-packages/xpra/server/window_source.py", line 746, in reconfigure
  File "/usr/lib64/python2.7/site-packages/xpra/server/window_video_source.py", line 630, in update_encoding_options
  File "/usr/lib64/python2.7/site-packages/xpra/server/window_video_source.py", line 670, in check_pipeline_score
    if ve is None or csce is None or ve.is_closed() or csce.is_closed():
AttributeError: 'xpra.codecs.csc_swscale.colorspace_converter.Color' object has no attribute 'is_closed'

Wed, 19 Nov 2014 23:51:37 GMT - Antoine Martin: owner changed

I believe this was caused by r8108: although the calls to is_closed were there before, we never actually hit that codepath because we never kept the same csc module whilst changing the encoder. That's why this only occurs now with nvenc enabled as per #733.

I fixed a similar bug in r8121 for the csc cython module, csc opencl already had it, r8125 fixes this for csc swscale. Please close if this bug is gone.

Wed, 03 Dec 2014 20:28:57 GMT - Antoine Martin: status changed; resolution set

Backports for v0.14.x were in r8128, I assume this is fixed so closing. Feel free to re-open if I've missed anything.

Sat, 23 Jan 2021 05:04:28 GMT - migration script:

this ticket has been moved to: https://github.com/Xpra-org/xpra/issues/736