xpra icon
Bug tracker and wiki

Opened 5 months ago

Closed 5 months ago

Last modified 5 months ago

#2104 closed defect (fixed)

support websockify > 0.8

Reported by: Antoine Martin Owned by: Antoine Martin
Priority: major Milestone: 2.5
Component: network Version: 2.4.x
Keywords: Cc:

Description (last modified by Antoine Martin)

They've moved things around..

Attachments (2)

websockify-new-import.patch (3.6 KB) - added by Antoine Martin 5 months ago.
work in progress, needs guess_type()
websockify-latest.patch (2.2 KB) - added by Antoine Martin 5 months ago.
some changes required to run against the latest websockify

Download all attachments as: .zip

Change History (8)

Changed 5 months ago by Antoine Martin

Attachment: websockify-new-import.patch added

work in progress, needs guess_type()

comment:1 Changed 5 months ago by Antoine Martin

Status: newassigned

Preliminary work in r21365.

They've changed something I struggle to figure out.
Hitting the TCP port using wget:

  • with websockify 0.8.0:
    2019-01-14 21:13:06,407 peer: (0, -1, -1)
    2019-01-14 21:13:06,408 new_connection((1, <socket._socketobject object at 0x7fa88fbf8d00>)) sock=<socket._socketobject object at 0x7fa88ccc2980>, socket_info=('0.0.0.0', 10000), timeout=0.1, address=('127.0.0.1', 47214), peername=('127.0.0.1', 47214). timeout=0.1
    2019-01-14 21:13:06,408 SocketConnection(<socket._socketobject object at 0x7fa88ccc2980>, ('127.0.0.1', 10000), ('127.0.0.1', 47214), ('127.0.0.1', 47214), 'tcp', {})
    2019-01-14 21:13:06,408 Connection(('127.0.0.1', 47214), 'tcp', {})
    2019-01-14 21:13:06,408 handle_new_connection(tcp socket: 127.0.0.1:10000 <- 127.0.0.1:47214, <socket._socketobject object at 0x7fa88ccc2980>, ('127.0.0.1', 47214), 'tcp', ('127.0.0.1', 47214), ('0.0.0.0', 10000)) sockname=('127.0.0.1', 10000), target=('127.0.0.1', 47214)
    2019-01-14 21:13:06,659 socket tcp socket: 127.0.0.1:10000 <- 127.0.0.1:47214 peek: got 152 bytes
    2019-01-14 21:13:06,659 socket peek='GET /index.html HTTP/1.1\r\nUser-Agent: Wget/1.20.1 (linux-gnu)\r\nAccept: */*\r\nAccept-Encoding: identity\r\nHost: localhost:10000\r\nConnection: Keep-Alive\r\n\r\n'
    2019-01-14 21:13:06,659 socket peek hex=474554202f696e6465782e68746d6c20485454502f312e310d0a557365722d4167656e743a20576765742f312e32302e3120286c696e75782d676e75290d0a4163636570743a202a2f2a0d0a4163636570742d456e636f64696e673a206964656e746974790d0a486f73743a206c6f63616c686f73743a31303030300d0a436f
    2019-01-14 21:13:06,659 socket peek line1='GET /index.html HTTP/1.1'
    2019-01-14 21:13:06,659 may_wrap_socket(..) peek_data=474554202f696e6465782e68746d6c20485454502f312e310d0a557365722d4167656e743a20576765742f312e32302e3120286c696e75782d676e75290d0a4163636570743a202a2f2a0d0a4163636570742d456e636f64696e673a206964656e746974790d0a486f73743a206c6f63616c686f73743a31303030300d0a436f6e6e656374696f6e3a204b6565702d416c6976650d0a0d0a from 127.0.0.1:47214
    2019-01-14 21:13:06,659 start_http_socket(tcp, tcp socket: 127.0.0.1:10000 <- 127.0.0.1:47214, False, ..) http proto=http, line1='GET /index.html HTTP/1.1'
    2019-01-14 21:13:06,659 New http GET request received from 127.0.0.1:47214 for '/index.html'
    2019-01-14 21:13:06,659 start_websockify(tcp, tcp socket: 127.0.0.1:10000 <- 127.0.0.1:47214, False, http GET, ('127.0.0.1', 47214)) www dir=/usr/share/xpra/www, headers dir=/usr/share/xpra/http-headers
    2019-01-14 21:13:06,659 may_wrap_socket(..)=(False, tcp socket: 127.0.0.1:10000 <- 127.0.0.1:47214, None)
    2019-01-14 21:13:06,660 send_head() script(/index.html)=None
    2019-01-14 21:13:06,660 translate_path(/index.html)=/usr/share/xpra/www/index.html
    2019-01-14 21:13:06,660 guess_type(/usr/share/xpra/www/index.html)=('text/html', None)
    2019-01-14 21:13:06,660 accept-encoding=['identity']
    2019-01-14 21:13:06,660 "GET /index.html HTTP/1.1" 200 -
    2019-01-14 21:13:06,661 may_reload_headers() http headers time={}, mtime=1540297043.29
    2019-01-14 21:13:06,661 may_reload_headers() loading from '/usr/share/xpra/http-headers/00_nocache.txt'
    2019-01-14 21:13:06,661 may_reload_headers() loading from '/usr/share/xpra/http-headers/10_content_security_policy.txt'
    2019-01-14 21:13:06,661 may_reload_headers() headers={'Expires': '0', 'Pragma': 'no-cache', 'Cache-Control': 'no-cache, no-store, must-revalidate'}, mtime=1540297043.29
    2019-01-14 21:13:06,661 handle_one_request()=None
    
  • with websockify trunk:
    2019-01-14 22:35:38,578 peer: (0, -1, -1)
    2019-01-14 22:35:38,578 new_connection((1, <socket._socketobject object at 0x7f2b006c0d00>)) sock=<socket._socketobject object at 0x7f2afd7d3130>, socket_info=('0.0.0.0', 10000), timeout=0.1, address=('127.0.0.1', 37130), peername=('127.0.0.1', 37130). timeout=0.1
    2019-01-14 22:35:38,578 SocketConnection(<socket._socketobject object at 0x7f2afd7d3130>, ('127.0.0.1', 10000), ('127.0.0.1', 37130), ('127.0.0.1', 37130), 'tcp', {})
    2019-01-14 22:35:38,578 Connection(('127.0.0.1', 37130), 'tcp', {})
    2019-01-14 22:35:38,579 handle_new_connection(tcp socket: 127.0.0.1:10000 <- 127.0.0.1:37130, <socket._socketobject object at 0x7f2afd7d3130>, ('127.0.0.1', 37130), 'tcp', ('127.0.0.1', 37130), ('0.0.0.0', 10000)) sockname=('127.0.0.1', 10000), target=('127.0.0.1', 37130)
    2019-01-14 22:35:38,829 socket tcp socket: 127.0.0.1:10000 <- 127.0.0.1:37130 peek: got 152 bytes
    2019-01-14 22:35:38,829 socket peek='GET /index.html HTTP/1.1\r\nUser-Agent: Wget/1.20.1 (linux-gnu)\r\nAccept: */*\r\nAccept-Encoding: identity\r\nHost: localhost:10000\r\nConnection: Keep-Alive\r\n\r\n'
    2019-01-14 22:35:38,830 socket peek hex=474554202f696e6465782e68746d6c20485454502f312e310d0a557365722d4167656e743a20576765742f312e32302e3120286c696e75782d676e75290d0a4163636570743a202a2f2a0d0a4163636570742d456e636f64696e673a206964656e746974790d0a486f73743a206c6f63616c686f73743a31303030300d0a436f
    2019-01-14 22:35:38,830 socket peek line1='GET /index.html HTTP/1.1'
    2019-01-14 22:35:38,830 may_wrap_socket(..) peek_data=474554202f696e6465782e68746d6c20485454502f312e310d0a557365722d4167656e743a20576765742f312e32302e3120286c696e75782d676e75290d0a4163636570743a202a2f2a0d0a4163636570742d456e636f64696e673a206964656e746974790d0a486f73743a206c6f63616c686f73743a31303030300d0a436f6e6e656374696f6e3a204b6565702d416c6976650d0a0d0a from 127.0.0.1:37130
    2019-01-14 22:35:38,830 start_http_socket(tcp, tcp socket: 127.0.0.1:10000 <- 127.0.0.1:37130, False, ..) http proto=http, line1='GET /index.html HTTP/1.1'
    2019-01-14 22:35:38,830 New http GET request received from 127.0.0.1:37130 for '/index.html'
    2019-01-14 22:35:38,830 start_websockify(tcp, tcp socket: 127.0.0.1:10000 <- 127.0.0.1:37130, False, http GET, ('127.0.0.1', 37130)) www dir=/usr/share/xpra/www, headers dir=/usr/share/xpra/http-headers
    2019-01-14 22:35:38,830 may_wrap_socket(..)=(False, tcp socket: 127.0.0.1:10000 <- 127.0.0.1:37130, None)
    2019-01-14 22:35:38,830 127.0.0.1: Plain non-SSL (ws://) WebSocket connection
    2019-01-14 22:35:38,830 127.0.0.1: Path: '/index.html'
    2019-01-14 22:35:38,830 new_websocket_client(<xpra.server.websocket.WSRequestHandler instance at 0x7f2afd911368>) socket=<socket._socketobject object at 0x7f2afd7d3130>
    2019-01-14 22:35:38,830 SocketConnection(<socket._socketobject object at 0x7f2afd7d3130>, ('127.0.0.1', 10000), ('127.0.0.1', 37130), ('127.0.0.1', 37130), 'ws', {})
    2019-01-14 22:35:38,830 Connection(('127.0.0.1', 37130), 'ws', {})
    2019-01-14 22:35:38,831 make_protocol(ws, ws websocket: 127.0.0.1:10000 <- 127.0.0.1:37130)
    2019-01-14 22:35:38,831 send_head() script(/index.html)=None
    2019-01-14 22:35:38,831 translate_path(/index.html)=/usr/share/xpra/www/index.html
    2019-01-14 22:35:38,831 io_thread_loop(read, <bound method Protocol._read of Protocol(ws websocket: 127.0.0.1:10000 <- 127.0.0.1:37130)>) loop starting
    2019-01-14 22:35:38,831 Error: read on ws websocket: 127.0.0.1:10000 <- 127.0.0.1:37130 failed: <type 'exceptions.AttributeError'>
    Traceback (most recent call last):
      File "/usr/lib64/python2.7/site-packages/xpra/net/protocol.py", line 591, in _io_thread_loop
        while not self._closed and callback():
      File "/usr/lib64/python2.7/site-packages/xpra/net/protocol.py", line 664, in _read
        buf = self._conn.read(READ_BUFFER_SIZE)
      File "/usr/lib64/python2.7/site-packages/xpra/server/websocket.py", line 352, in read
        bufs, closed_string = self.ws_handler.recv_frames()
      File "/usr/lib/python2.7/site-packages/websockify-0.8.0-py2.7.egg/websockify/websockifyserver.py", line 183, in recv_frames
        buf = self.request.recvmsg()
    AttributeError: '_socketobject' object has no attribute 'recvmsg'
    2019-01-14 22:35:38,831 guess_type(/usr/share/xpra/www/index.html)=('text/html', None)
    2019-01-14 22:35:38,832 Protocol.close() closed=False, connection=ws websocket: 127.0.0.1:10000 <- 127.0.0.1:37130
    2019-01-14 22:35:38,832 accept-encoding=['identity']
    2019-01-14 22:35:38,832 Protocol.close() calling <bound method WebSocketConnection.close of ws websocket: 127.0.0.1:10000 <- 127.0.0.1:37130>
    2019-01-14 22:35:38,832 process default packet connection-lost
    2019-01-14 22:35:38,834 netifaces loaded sucessfully
    2019-01-14 22:35:38,834 process_connection_lost(Protocol(None), ['connection-lost'])
    2019-01-14 22:35:38,834 "GET /index.html HTTP/1.1" 200 -
    2019-01-14 22:35:38,835 ws websocket: 127.0.0.1:10000 <- 127.0.0.1:37130.close() for socket={'fileno': 43, 'family': 2, 'proto': 0, 'timeout': 5000, 'type': 1, 'options': {'IP': {'IP_TTL': 64, 'IP_TOS': 0, 'IP_OPTIONS': 0}, 'SOCKET': {'SO_TYPE': 1, 'SO_SNDBUF': 2626560, 'SO_ERROR': 0, 'SO_RCVLOWAT': 1, 'SO_KEEPALIVE': 0, 'SO_RCVTIMEO': 0, 'SO_BROADCAST': 0, 'SO_OOBINLINE': 0, 'SO_REUSEPORT': 0, 'SO_DONTROUTE': 0, 'SO_RCVBUF': 1062000, 'SO_LINGER': 0, 'SO_SNDTIMEO': 0, 'SO_REUSEADDR': 1}, 'TCP': {'TCP_NODELAY': 1, 'TCP_MAXSEG': 21888}}}
    2019-01-14 22:35:38,835 ws websocket: 127.0.0.1:10000 <- 127.0.0.1:37130.close() done
    2019-01-14 22:35:38,835 terminate_queue_threads()
    2019-01-14 22:35:38,835 cleanup_protocol(Protocol(None))
    2019-01-14 22:35:38,836 may_reload_headers() http headers time={}, mtime=1540297043.29
    2019-01-14 22:35:38,836 Protocol.close() done
    2019-01-14 22:35:38,836 may_reload_headers() loading from '/usr/share/xpra/http-headers/00_nocache.txt'
    2019-01-14 22:35:38,836 may_reload_headers() loading from '/usr/share/xpra/http-headers/10_content_security_policy.txt'
    2019-01-14 22:35:38,836 may_reload_headers() headers={'Expires': '0', 'Pragma': 'no-cache', 'Cache-Control': 'no-cache, no-store, must-revalidate'}, mtime=1540297043.29
    
Last edited 5 months ago by Antoine Martin (previous) (diff)

Changed 5 months ago by Antoine Martin

Attachment: websockify-latest.patch added

some changes required to run against the latest websockify

comment:2 Changed 5 months ago by Antoine Martin

Couldn't figure out what was going wrong, so bisected websockify instead:

Next commit is:

comment:3 Changed 5 months ago by Antoine Martin

So the first breakage comes from the changes to websocket.py diff-653cb1e48180f6007df323a009690197.
This commit is huge, and it's not clear at all why calling socket.recv on the underlying tcp socket returns nothing... (even in blocking mode)

Meanwhile, this may have uncovered a bug in python-websocket-client:

Traceback (most recent call last):
  File "/usr/lib64/python2.7/site-packages/xpra/scripts/main.py", line 967, in connect_or_fail
    return connect_to(display_desc, opts)
  File "/usr/lib64/python2.7/site-packages/xpra/scripts/main.py", line 1098, in connect_to
    return websocket_client_connection(host, port, conn, dtype)
  File "/usr/lib64/python2.7/site-packages/xpra/net/websocket_connection.py", line 79, in websocket_client_connection
    ws = websocket.create_connection(url, SOCKET_TIMEOUT, subprotocols=subprotocols, socket=sock)
  File "/usr/lib/python2.7/site-packages/websocket/_core.py", line 502, in create_connection
    websock.connect(url, **options)
  File "/usr/lib/python2.7/site-packages/websocket/_core.py", line 222, in connect
    self.handshake_response = handshake(self.sock, *addrs, **options)
  File "/usr/lib/python2.7/site-packages/websocket/_handshake.py", line 70, in handshake
    success, subproto = _validate(resp, key, options.get("subprotocols"))
  File "/usr/lib/python2.7/site-packages/websocket/_handshake.py", line 160, in _validate
    subproto = headers.get("sec-websocket-protocol", None).lower()
AttributeError: 'NoneType' object has no attribute 'lower'
Version 0, edited 5 months ago by Antoine Martin (next)

comment:4 Changed 5 months ago by Antoine Martin

Description: modified (diff)

comment:5 Changed 5 months ago by Antoine Martin

Description: modified (diff)

Found that there were typos in that changeset: self._sendmg should be self._sendmsg.

During testing, I also hit this one:

Traceback (most recent call last):
  File "/usr/lib64/python2.7/site-packages/xpra/net/protocol.py", line 1042, in close
    c.close()
  File "/usr/lib64/python2.7/site-packages/xpra/server/websocket.py", line 342, in close
    SocketConnection.close(self)
  File "/usr/lib64/python2.7/site-packages/xpra/net/bytestreams.py", line 334, in close
    s.close()
  File "/usr/lib/python2.7/site-packages/websockify-0.8.0-py2.7.egg/websockify/websocket.py", line 489, in close
    self._close()
  File "/usr/lib/python2.7/site-packages/websockify-0.8.0-py2.7.egg/websockify/websocket.py", line 689, in _close
    self.socket.close()
AttributeError: 'NoneType' object has no attribute 'close'
Last edited 5 months ago by Antoine Martin (previous) (diff)

comment:6 Changed 5 months ago by Antoine Martin

Resolution: fixed
Status: assignedclosed

Fixed in r21463.
Support added for websockify>0.8 in the performance test in r21465. (#1926)

Maybe we should propagate the EWOULDBLOCK write event. (and not just from websockets IO)

Will follow up in:

  • #2120 websocket-client vs websockify>0.8 error: Text frames are not supported
  • #2121 speedup websockify send / hybi
Last edited 5 months ago by Antoine Martin (previous) (diff)
Note: See TracTickets for help on using tickets.