xpra icon
Bug tracker and wiki

Opened 3 years ago

Last modified 2 months ago

#639 assigned task

UDP transport

Reported by: Antoine Martin Owned by: Antoine Martin
Priority: major Milestone: 2.1
Component: core Version: trunk
Keywords: Cc:

Description

Rather than using mosh #219, we should be able to do our own UDP transport: we know which packets must arrive and may need to be re-sent (window metadata, etc), and which ones we can just skip when they go missing: we just send a newer update instead (window pixels, cursors, etc..).

Then we can also tune the video encoders and insert key frames when needed (and maybe also lower the default gap between key frames when UDP is used), etc..

This may also helps us achieve other things: we could more easily use parallel processes for encoding since each process can then send its data without needing to synchronize or share the socket. Also useful when using hardware encoders which have their own network ports.

We should not need to worry about UDP hole punching for now, if ever.
I don't think we want to be using a library like UDT either: we would lose some flexibility, and the python bindings are old and not very portable...

Change History (5)

comment:1 Changed 2 years ago by Antoine Martin

Status: newassigned

Some preliminary notes:

  • we can't do encryption to begin with (see #198, #584) because we use AES CBC: In CBC mode, each block of plaintext is XORed with the previous ciphertext block before being encrypted. This way, each ciphertext block depends on all plaintext blocks processed up to that point. (this breaks as soon as we get packets out of order, or if we drop any..)
  • if we don't get an ACK from the client after a delay, we must probe it (we could augment the paint packet with a new capability)
  • pass flag to video encoders so they can enable support from dropped frames?
  • maybe use a new command line argument: --bind= which would bind to both TCP and UDP ports?
  • how do we take into account the maximum UDP packet size (and detect or change it..)
  • the protocol class would need new threads for UDP: read and write (assuming each end does both - not necessarily the case..)
  • client will need to wait for the paint packets to arrive in the right order, and timeout if one goes MIA, deal with dupes - we could also decode them (at least some of them... like non video) whilst waiting
  • client can ask server for a refresh when things go wrong, we already reset the video pipeline when the client reports decoding errors

very distant future: handle UDP broadcasts for multiple clients

comment:2 Changed 2 years ago by Antoine Martin

Milestone: 0.15future

Not as useful as first thought, too many implementation issues, other more pressing issues (ie: #835) would clash with this. So re-scheduling as future.

comment:3 Changed 15 months ago by Antoine Martin

For the encryption part, we can use CBC with a new IV based on the packet sequence number, or a counter mode, or just DTLS.

comment:4 Changed 15 months ago by Antoine Martin

See also #1124

The packet loss rate may have an effect on our encoding selection: maybe we should use encodings that tolerate packet loss better. ie: not png or jpeg for big areas.
We already keep track of what hasn't been acked yet, and we can re-send if necessary.

comment:5 Changed 2 months ago by Antoine Martin

Milestone: future2.1
Note: See TracTickets for help on using tickets.