Xpra: Ticket #1030: webcam and scanner forwarding

So that applications that want to access the webcam can be made to work (ie: skype, etc). Scanners use the same framework as webcams, so maybe we can handle those too? (means potentially more than one device...)

We will need a simplified "window video source" class that the client can use to send pixel data with the same adaptability as the server (but maybe simplified a bit when it comes to batch delay, speed and quality tuning). It would be good if we could know when frames are being consumed from the server device so that we don't bother updating it when no-one is watching. It will only deal with full frames, and only using a video encoder (if one is available) and no mmap. It should have the same controls as sound: on, off or disabled.

The server should only need to pass the frames to a virtual video device for this to work:

The initial version should not worry about:

etc..



Fri, 13 Nov 2015 06:44:15 GMT - Antoine Martin: status, description, summary changed


Mon, 01 Feb 2016 06:30:43 GMT - Antoine Martin: attachment set

almost working patch using opencv, png encoding only for now


Mon, 01 Feb 2016 20:31:21 GMT - Antoine Martin:

Proof of concept code merged in r11808 with some severe limitations:


Build updates.

These things will need to be added to wiki/Dependencies. We don't actually use or need some of those, but installing them anyway is easier than excising them out of matplotlib / opencv.

Almost gets you there, except it fails on directx.. (why on earth is it even trying to build directx on osx??)


Still TODO:


Thu, 04 Feb 2016 19:27:34 GMT - Antoine Martin: owner, status changed

Added a wiki page with details: wiki/Webcam. Will follow up in #1113. This is good enough as a proof-of-concept only, it will need more work - but not for this release.

@afarr: ready for an initial round of testing. Please re-assign to me for bugs or to smo so that he can deal with the required win32 / osx / Fedora build updates.


Fri, 05 Feb 2016 21:41:24 GMT - J. Max Mena:

Attempting --webcam=yes outputs the following error:

2016-02-05 13:40:09,266 Warning: virtual video directory /sys/devices/virtual/video4linux not found
2016-02-05 13:40:09,266  webcam forwarding disabled

And, obviously, the webcam forwarding does not work. Am I missing something?

I'll keep looking at it to see if I need to do anything else, but I'm pretty sure all the bits are there and ready.


Fri, 05 Feb 2016 22:19:33 GMT - J. Max Mena:

Update:

No more error.

However whatever I try, I cannot get it to detect a webcam on the client side and successfully forward it (even using a Win8.1 r11850 client).


Fri, 05 Feb 2016 22:20:55 GMT - J. Max Mena:

Oh, also some logs:

Client:

C:\Program Files (x86)\Xpra>Xpra_cmd.exe attach tcp:10.0.32.189:2200 --webcam=ye
s -d webcam
2016-02-05 14:19:57,086 Xpra gtk2 client version 0.17.0-r11850
2016-02-05 14:19:57,086  running on Microsoft Windows 8.1
2016-02-05 14:19:57,086 webcam forwarding: yes
2016-02-05 14:19:57,275 GStreamer version 1.4 for Python 3.4
2016-02-05 14:19:57,634 OpenGL_accelerate module loaded
2016-02-05 14:19:57,711  detected keyboard: layout=us
2016-02-05 14:19:57,727  desktop size is 2560x1440 with 1 screen:
2016-02-05 14:19:57,727   Default (677x381 mm - DPI: 96x96) workarea: 2560x1400
2016-02-05 14:19:57,727     DISPLAY1 (597x336 mm - DPI: 108x108)
2016-02-05 14:19:57,727  upscaled by 125%, virtual screen size: 2048x1152
2016-02-05 14:19:57,727   Default (677x381 mm - DPI: 76x76) workarea: 2048x1120
2016-02-05 14:19:57,727     DISPLAY1 (597x336 mm - DPI: 87x87)
2016-02-05 14:19:57,775 Xpra X11 server version 0.17.0-r11382
2016-02-05 14:19:57,775  running on Linux Fedora 23 Twenty Three
2016-02-05 14:19:57,775 webcam server support: True (1 devices)
2016-02-05 14:19:57,789 Attached to tcp:10.0.32.189:2200 (press Control-C to det
ach)
2016-02-05 14:19:57,789 webcam_state() webcam forwarding=True, server virtual vi
deo devices=1
2016-02-05 14:20:00,556 sound output using audio codec: Vorbis
2016-02-05 14:20:14,983 setting scaling to 100%:
2016-02-05 14:20:14,983 sending updated screen size to server: 2560x1440 with 1
screens
2016-02-05 14:20:14,983   Default (677x381 mm - DPI: 96x96) workarea: 2560x1400
2016-02-05 14:20:14,983     DISPLAY1 (597x336 mm - DPI: 108x108)

Server:

2016-02-05 13:57:41,569 init_virtual_video_devices
2016-02-05 13:57:41,570 found 1 virtual video device

(the only webcam related print)


Sat, 06 Feb 2016 05:56:19 GMT - Antoine Martin: owner changed

one must first load the device modprobe v4l2loopback


Yes, it is on the wiki page wiki/Webcam. r11862 tries to make it clearer in the server warning message.


C:\Program Files (x86)\Xpra>Xpra_cmd.exe attach tcp:10.0.32.189:2200 --webcam=yes -d webcam


As per the wiki page.. use the value "on" if you want it to start when you connect. Unless you disable webcam completely (values "no", "false"), the webcam support will be available but needs to be started from the systray.


r11863 and r11864 improve logging, command line parsing and error handling. New beta builds with those changes are available.


Tue, 09 Feb 2016 17:17:50 GMT - J. Max Mena:

Upped client to r11892 Win8.1 and Server to trunk r11892:

(Clicked it a few times hoping to maybe get that menu)

Will put server side prints in a second (have to restart session to get logs)


Tue, 09 Feb 2016 17:21:04 GMT - J. Max Mena:

Server side -d webcam print:

2016-02-09 09:20:50,363 v4l2 using device /dev/video0
2016-02-09 09:20:50,363 Error setting up webcam forwarding:
2016-02-09 09:20:50,363  [Errno 13] Permission denied: '/dev/video0'
2016-02-09 09:20:50,386 stopping webcam device 0:
2016-02-09 09:20:50,386 Warning: cannot stop webcam device 0: no such context!

Looks like it can't access the camera? Is there a group in which I should be a member?


Tue, 09 Feb 2016 18:06:49 GMT - J. Max Mena:

Update:


Tue, 09 Feb 2016 19:22:31 GMT - J. Max Mena: owner changed

Another update:

Turns out it just doesn't like cheese or Chrome. If you install camorama and launch it with camorama -d /dev/$cameradeviceid (usually is video0), then it works. (greyscale and chopped oddly, but 30fps)

Passing back to you, antoine.


Tue, 09 Feb 2016 20:58:55 GMT - Antoine Martin: status changed

Specifying the device with camorama should not be necessary as long as the device is the first one (video0).

The fps will be lower, much lower, when running on a slower connection as we wait for the server ack before sending the next frame. ie: on a 100ms link, that's going to limit things to 10fps maximum.

As for compatibility with other applications like cheese and chrome, I will try to take a look but this will not be considered a blocker for the 0.17 release as this is mostly a feature preview at this point. This can be dealt with in the next release: #1113.


Fri, 26 Feb 2016 05:37:54 GMT - Antoine Martin:

Note we will need a newer gcc to build webcam support on OSX because of this compiler crash: Internal compiler error in directx.cpp on Mac OS X 10.6. Getting around this by hacking the source files only leads to more gcc pain down the road. (tbb and gcd don't build, etc)


Mon, 28 Mar 2016 05:51:36 GMT - Antoine Martin: status changed; resolution set

I think this will do for this release.


Mon, 25 Apr 2016 10:29:16 GMT - Antoine Martin: milestone changed


Mon, 21 Oct 2019 02:37:36 GMT - Antoine Martin:

Follow up: #1231


Sat, 23 Jan 2021 05:12:59 GMT - migration script:

this ticket has been moved to: https://github.com/Xpra-org/xpra/issues/1030