xpra icon
Bug tracker and wiki

Opened 12 days ago

Closed 12 days ago

#2415 closed defect (fixed)

CUDA initialization is slow - skip devices

Reported by: Antoine Martin Owned by: Antoine Martin
Priority: major Milestone: 3.0
Component: encodings Version: 3.0.x
Keywords: Cc:

Description

See also #2414, #2048.

23:50:58,712 CUDA initialization (this may take a few seconds)
23:50:59,387 CUDA 10.0.0 / PyCUDA 2018.1.1, found 8 devices:
23:50:59,387   + Tesla V100-SXM2-32GB @ 0000:1A:00.0 (memory: 98% free, compute: 7.0)
23:50:59,624   + Tesla V100-SXM2-32GB @ 0000:1B:00.0 (memory: 98% free, compute: 7.0)
23:50:59,851   + Tesla V100-SXM2-32GB @ 0000:3D:00.0 (memory: 98% free, compute: 7.0)
23:51:00,075   + Tesla V100-SXM2-32GB @ 0000:3E:00.0 (memory: 98% free, compute: 7.0)
23:51:00,303   + Tesla V100-SXM2-32GB @ 0000:88:00.0 (memory: 98% free, compute: 7.0)
23:51:00,540   + Tesla V100-SXM2-32GB @ 0000:89:00.0 (memory: 98% free, compute: 7.0)
23:51:00,783   + Tesla V100-SXM2-32GB @ 0000:B2:00.0 (memory: 98% free, compute: 7.0)
23:51:01,003   + Tesla V100-SXM2-32GB @ 0000:B3:00.0 (memory: 98% free, compute: 7.0)
23:51:01,072 NVidia driver version 418.56
23:51:01,072 Warning: NVidia driver version 418.56 is untested with NVENC
23:51:01,073  (this encoder has been tested with versions up to 396.x only)
23:51:01,236 device matches preferred device id 0: Tesla V100-SXM2-32GB @ 0000:1A:00.0
23:51:01,692 device matches preferred device id 0: Tesla V100-SXM2-32GB @ 0000:1A:00.0
23:51:02,519 device matches preferred device id 1: Tesla V100-SXM2-32GB @ 0000:1B:00.0
23:51:03,284 device matches preferred device id 1: Tesla V100-SXM2-32GB @ 0000:1B:00.0
23:51:04,074 device matches preferred device id 2: Tesla V100-SXM2-32GB @ 0000:3D:00.0
23:51:04,756 device matches preferred device id 2: Tesla V100-SXM2-32GB @ 0000:3D:00.0
23:51:05,647 device matches preferred device id 3: Tesla V100-SXM2-32GB @ 0000:3E:00.0
23:51:06,526 device matches preferred device id 3: Tesla V100-SXM2-32GB @ 0000:3E:00.0
23:51:07,865 device matches preferred device id 4: Tesla V100-SXM2-32GB @ 0000:88:00.0
23:51:09,170 device matches preferred device id 4: Tesla V100-SXM2-32GB @ 0000:88:00.0
23:51:10,691 device matches preferred device id 5: Tesla V100-SXM2-32GB @ 0000:89:00.0
23:51:12,112 device matches preferred device id 5: Tesla V100-SXM2-32GB @ 0000:89:00.0
23:51:14,018 device matches preferred device id 6: Tesla V100-SXM2-32GB @ 0000:B2:00.0
23:51:15,879 device matches preferred device id 6: Tesla V100-SXM2-32GB @ 0000:B2:00.0
23:51:18,009 device matches preferred device id 7: Tesla V100-SXM2-32GB @ 0000:B3:00.0
23:51:20,134 device matches preferred device id 7: Tesla V100-SXM2-32GB @ 0000:B3:00.0
23:51:20,601 NVENC successfully initialized

That's roughly 2 seconds for init_all_devices which is fine, but then a lot more time spent in select_device as we iterate over all the devices to find the one we're trying to use for testing each codec.

Change History (1)

comment:1 Changed 12 days ago by Antoine Martin

Resolution: fixed
Status: newclosed

Should be fixed in:

  • r23782 : don't enumerate all the devices to find the one we want
  • r23783 + r23784: make it possible to specify which devices will be initialized and later used by NVENC

Unlike the existing blacklist and device-name options found in /etc/xpra/cuda.conf which can use device names, the new enabled-devices and disabled-devices options use device numbers for now (will improve this in #2416).

Note: See TracTickets for help on using tickets.