Xpra: Ticket #1113: improve webcam support

Follow up from #1030. See wiki/Webcam.

there is no systray menu for it
framerate is low
black and white only, low resolution (hardcoded csc dependencies)
no support for multiple sessions per server..
detecting when devices are added or removed? (dbus?)
it uses too much bandwidth (mostly wasted too)
Fedora dkms packaging for v4l2loopback?
investigate win32 and osx support (shadow server / remote webcam)

Thu, 04 Feb 2016 19:24:55 GMT - Antoine Martin: status, description changed

status changed from new to assigned
description modified (diff)

We should also add support for native H264: Using the Logitech C920 webcam with Gstreamer

Tue, 14 Jun 2016 17:12:03 GMT - Antoine Martin:

Updates:

finally found the fix for the offset hack: r12822 (will backport)
update webcam tray menu when devices are added or removed (linux only via python-inotify): r12819, packaging updates in r12821, fixup in r12820
r12823 should now support multiple users: load the kernel module with modprobe v4l2loopback devices=4 and the xpra server will try all devices until it finds one it can use (not fully tested)
r12825 makes sure we filter out video devices that cannot do capture

The webcam tool now shows a lot more device information.

Maybe too late for us, but I've just found a python ctypes interface for v4l2: https://pypi.python.org/pypi/v4l2

As for v4l2loopback, packaging it for dkms is too hard (dkms always seem to cause more problems that it solves for me too). Why isn't this merged upstream? Can we help push it? The code is stable and does not change much.

Remaining issues:

framerate and hardcoded csc dependency
win32 and osx support:
- enumerate devices on win32: Win32_USBHub
- enumerate devices on osx: How to get a list of video capture devices (web cameras) on Mac OS? (C++)
- shadow servers (low priority)

This is what the systray now looks like:

Wed, 15 Jun 2016 16:15:54 GMT - Antoine Martin:

r12827 detects video capture devices on win32 using ISampleGrabber)
r12828 detects device change events using WM_DEVICECHANGE (not specific to video devices... but seems to work nonetheless)

Wed, 15 Jun 2016 16:19:52 GMT - Antoine Martin: attachment set

attachment set to new-webcam-menu.png

new systray menu shows my webcam as option

Thu, 16 Jun 2016 11:22:02 GMT - Antoine Martin: attachment set

attachment set to osx-device-notifications.patch

try to get notifications when USB devices are added or removed

Thu, 16 Jun 2016 12:27:30 GMT - Antoine Martin:

The patch above attempts to use IOKit.py to get device notifications for OSX, but it has a number of problems:

can't re-use the same main loop as GTK, so it uses its own thread and main loop - which may be the cause of other problems..
not very stable
had to modify IOKit.py
can't get the device path (that part crashes if you try to use it)
webcams don't work on OSX anyway since we cannot build opencv..

Similar / related code:

Webcam API links:

Still and Video Media Capture
Image Capture Applications Programming Guide
capturing image from webcam
python Barcode Scanner (and update code for new pyobjc..)
ffmpeg webcam - probably easiest to implement, 10.7 onwards...

Thu, 16 Jun 2016 16:09:24 GMT - Antoine Martin: owner, status changed

owner changed from Antoine Martin to alas
status changed from assigned to new

r12834 solves the framerate issue over slower links by calculating how many frames-in-flight we should have and try to keep to that number. It adds XPRA_WEBCAM_FPS means we try to send that many frames per second (ie: the default value of 20 will send a frame every 50 milliseconds), unless the ack packets start coming in late. Debug output:

may_send_webcam_frame() latency=307, not acked=6, target=6

Not dealing with shadow servers, because I don't see how this would work.

This will do for this release, will follow up in #1231.

Tue, 12 Jul 2016 16:52:22 GMT - Antoine Martin: milestone changed

milestone changed from 0.18 to 1.0

Milestone renamed

Fri, 23 Sep 2016 07:12:17 GMT - Antoine Martin:

r13832: improved webcam timers management, etc (see commit message)
see also #1268

Mon, 26 Dec 2016 09:22:23 GMT - Antoine Martin: status changed; resolution set

status changed from new to closed
resolution set to fixed

Not heard back, closing.

Mon, 21 Oct 2019 02:37:30 GMT - Antoine Martin: description changed

description modified (diff)

For win32 and macos see #1231

Sat, 23 Jan 2021 05:15:20 GMT - migration script:

this ticket has been moved to: https://github.com/Xpra-org/xpra/issues/1113