html5 client

[Antoine Martin]

Similar to ​noVNC, and probably re-using some of its code (websockets, etc)

Made much easier by #474

I can start a websockets proxy (on port 8080) that points back to xpra (on port 10000):

./websockify.py --web ./Xpra/trunk/html5/ 8080 localhost:10000

And if I use the new tcp-proxy code:

xpra start :10 --bind-tcp=0.0.0.0:10000 --tcp-proxy=127.0.0.1:8080

When we make websocket request to the xpra port (10000), the http traffic goes through xpra to the proxy then back to xpra as tcp!
Eventually, the websockets proxy code should be moved into xpra, but this is good enough for experimenting and testing.

[Antoine Martin]

experimental html5 client is now capable of showing an xterm window!

[Antoine Martin]

modified bencoder that can talk to an xpra server and handle byte buffers efficiently (original code is MIT licensed)

[Antoine Martin]

What I've used:

• ​websockify (many licenses for all the different parts: LGPL version 3, MPL 2, BSD... see: ​LICENSE.txt)
• bencode.js (modified version - MIT licensed)
• The canvas code is based on ​shapesdemo, itself based on these tutorials: ​A Gentle Introduction to Making HTML5 Canvas Interactive, ​Selectable Shapes Part 2: Resizable, Movable Shapes on HTML5 Canvas.

The license reads: Free to use and distribute at will, so long as you are nice to people, etc. May need clarification... as free licenses are unrestricted, and asking for people to "be nice" may be incompatible with other free licenses!

• For zlib, I've used ​imaya's zlib.js (​english link), which is MIT licensed.

I will attach some code once it is cleaned up.

What still needs to be done:

• move the network code to a ​worker (see: ​Using webworkers)
• use ​FileReaderSync so packets get delivered in the same order as they are received!
• use ​Transferable objects from the worker thread to the ui thread
• instead of invalidating the whole canvas, repaint just the area we have updated using ​CanvasRenderingContext2D.drawImage or ​CanvasRenderingContext2D.putImageData
• handle png, jpeg and webp picture formats (ie: ​receiving image through websocket and ​Display image from blob using javascript and websockets)
• fix compatibility with other browsers (only works on Firefox for now!)
• handle browser window resizing (fit to page)
• handle window decorations using stylesheets
• tell server about the size of our decorations so it can start sending pixel data immediately without having to wait for the client map-event packet which is unnecessary using HTML5 (we can place things precisely wherever we like)
• better UI with connection dialog, options, etc (and add no decorations mode option for full-window content)
• handle focus, clicks and keyboard

And maybe further down the line:

• handle notifications (floating divs?), ​custom cursors, bell events (blink? bell icon?), ​clipboard, etc..
• use ​MediaSource API to pass video data to a video element to support h264/vp8/etc
• use webgl for rendering and csc
• use int protocol aliases to make packets shorter
• move websockify calls inside the xpra process to avoid all the back and forth, extra sockets, etc
• ​lz4 support (after removing the need for node.js from the module)
• implement ​rencode in js

[Antoine Martin]

Initial code merged in r5028: sort of works in chrome and firefox (partial window updates don't work, partial keyboard support, no mouse support, ..)

[Antoine Martin]

Many fixes (see changesets) in: r5029 + r5030 + r5031 + r5032 + r5033

Actually makes this almost usable!