#1113 closed enhancement (fixed)
improve webcam support
Reported by: | Antoine Martin | Owned by: | alas |
---|---|---|---|
Priority: | major | Milestone: | 1.0 |
Component: | encodings | Version: | trunk |
Keywords: | Cc: |
Description (last modified by )
Follow up from #1030. See wiki/Webcam.
- there is no systray menu for it
- framerate is low
- black and white only, low resolution (hardcoded csc dependencies)
- no support for multiple sessions per server..
- detecting when devices are added or removed? (dbus?)
- it uses too much bandwidth (mostly wasted too)
- Fedora dkms packaging for v4l2loopback?
- investigate win32 and osx support (shadow server / remote webcam)
Attachments (2)
Change History (12)
comment:1 Changed 5 years ago by
Description: | modified (diff) |
---|---|
Status: | new → assigned |
comment:2 Changed 5 years ago by
Updates:
- finally found the fix for the offset hack: r12822 (will backport)
- update webcam tray menu when devices are added or removed (linux only via python-inotify): r12819, packaging updates in r12821, fixup in r12820
- r12823 should now support multiple users: load the kernel module with
modprobe v4l2loopback devices=4
and the xpra server will try all devices until it finds one it can use (not fully tested) - r12825 makes sure we filter out video devices that cannot do capture
The webcam tool now shows a lot more device information.
Maybe too late for us, but I've just found a python ctypes interface for v4l2: https://pypi.python.org/pypi/v4l2
As for v4l2loopback, packaging it for dkms is too hard (dkms always seem to cause more problems that it solves for me too). Why isn't this merged upstream? Can we help push it? The code is stable and does not change much.
Remaining issues:
- framerate and hardcoded csc dependency
- win32 and osx support:
- enumerate devices on win32: How to get a list of video capture devices (web cameras) on windows? (C++), win32: Selecting a Capture Device, DirectShow, Microsoft Media Foundation, Audio/Video Capture, ctypes enumeration code, using WMI? Win32_USBHub
- enumerate devices on osx: How to get a list of video capture devices (web cameras) on Mac OS? (C++)
- shadow servers (low priority)
comment:3 Changed 5 years ago by
- r12827 detects video capture devices on win32 using comtypes. (maybe we could capture to a PIL image more or less directly, and drop the dependency on opencv: http://pydoc.net/Python/jaraco.video/1.2/jaraco.video.capture/ using ISampleGrabber)
- r12828 detects device change events using WM_DEVICECHANGE (not specific to video devices... but seems to work nonetheless)
More links on win32 device events API:
OSX:
- Introduction to USB Device Interface Guide
- IOKit device adding/removal notifications - only fire once?
- IOKit.py
As for framerate, without going as far as refactoring the window-source code so we can use it for sending webcam frames, maybe we can send frames based on the line latency. (ie: 200ms round trip -> allow 4 frames in flight for a 20fps target)
Changed 5 years ago by
Attachment: | new-webcam-menu.png added |
---|
new systray menu shows my webcam as option
Changed 5 years ago by
Attachment: | osx-device-notifications.patch added |
---|
try to get notifications when USB devices are added or removed
comment:4 Changed 5 years ago by
The patch above attempts to use IOKit.py to get device notifications for OSX, but it has a number of problems:
- can't re-use the same main loop as GTK, so it uses its own thread and main loop - which may be the cause of other problems..
- not very stable
- had to modify IOKit.py
- can't get the device path (that part crashes if you try to use it)
- webcams don't work on OSX anyway since we cannot build opencv..
Similar / related code:
- USBNotifier.m
- How can I find Device File Name of an USB?
- Can't find bsd name of usb device - bug in 10.11.x?
- iodisplayregistry.c traverses and dumps all services, also: GPUModelVRAMInfo.c
- IOKitLib header reference
Webcam API links:
- Still and Video Media Capture
- Image Capture Applications Programming Guide
- capturing image from webcam
- python Barcode Scanner (and update code for new pyobjc..)
- ffmpeg webcam - probably easiest to implement, 10.7 onwards...
comment:5 Changed 5 years ago by
Owner: | changed from Antoine Martin to alas |
---|---|
Status: | assigned → new |
r12834 solves the framerate issue over slower links by calculating how many frames-in-flight we should have and try to keep to that number.
It adds XPRA_WEBCAM_FPS
means we try to send that many frames per second (ie: the default value of 20 will send a frame every 50 milliseconds), unless the ack packets start coming in late.
Debug output:
may_send_webcam_frame() latency=307, not acked=6, target=6
Not dealing with shadow servers, because I don't see how this would work.
This will do for this release, will follow up in #1231.
comment:7 Changed 5 years ago by
comment:10 Changed 3 months ago by
this ticket has been moved to: https://github.com/Xpra-org/xpra/issues/1113
We should also add support for native H264: Using the Logitech C920 webcam with Gstreamer