Xpra: Ticket #2560: Pressing AltGr+< on a Finnish keyboard layout produces some kind of letter i instead of the pipe character

    keycode  31 = i I i I bar brokenbar bar

Wed, 22 Jan 2020 16:33:25 GMT - Antoine Martin: status, description, milestone changed

• status changed from new to assigned
• description modified (diff)
• milestone changed from 3.1 to 4.0

• we should detect 'fi' keyboard layout automatically - not sure why that doesn't happen
• look for brokenbar key events

Wed, 22 Jan 2020 16:35:32 GMT - Antoine Martin: description changed

• description modified (diff)

Wed, 22 Jan 2020 16:39:26 GMT - Antoine Martin: description changed

• description modified (diff)

Wed, 22 Jan 2020 17:24:48 GMT - Antoine Martin: attachment set

• attachment set to Finnish_Multilingual.png

Finnish Keyboard Layout

Wed, 22 Jan 2020 17:25:28 GMT - Antoine Martin:

Keyboard layout confirmed as (without the blue keys):

Wed, 22 Jan 2020 17:44:42 GMT - totaamwin32:

Testing in my win10 VM connecting to a Fedora 31 server, after adding a new keyboard layout and all the keys do show up, though the client shows some warnings parsing the keysyms we get from GTK:

Warning: failed to parse string for key
 keyname=notsign, keycode=223
 'utf-8' codec can't decode byte 0xac in position 0: invalid start byte
Warning: failed to parse string for key
 keyname=Odiaeresis, keycode=192
 'utf-8' codec can't decode byte 0xd6 in position 0: unexpected end of data
Warning: failed to parse string for key
 keyname=Adiaeresis, keycode=222
 'utf-8' codec can't decode byte 0xc4 in position 0: unexpected end of data
Warning: failed to parse string for key
 keyname=odiaeresis, keycode=192
 'utf-8' codec can't decode byte 0xf6 in position 0: invalid start byte
Warning: failed to parse string for key
 keyname=adiaeresis, keycode=222
 'utf-8' codec can't decode byte 0xe4 in position 0: unexpected end of data
Warning: failed to parse string for key
 keyname=Aring, keycode=221
 'utf-8' codec can't decode byte 0xc5 in position 0: unexpected end of data
Warning: failed to parse string for key
 keyname=aring, keycode=221
 'utf-8' codec can't decode byte 0xe5 in position 0: unexpected end of data

Let's fix that too.

As for the missing characters with your setup, there must be something different. Not sure what, but if I can't reproduce the problem, it's going to be hard to fix.

Thu, 23 Jan 2020 02:11:29 GMT - akikoo:

/ should be on the left side of right shift, but it's not.

That comment was in relation to the xpra client defaults. If I don't enter --keyboard-layout=fi, setxkbmap -query shows 'us' but the layout is not working as '/' is not on the left side of right shift.

Thu, 23 Jan 2020 05:13:49 GMT - Antoine Martin:

For correct layout detection, it seems that we need to use GetKeyboardLayoutName, definition added in r25046. Mappings added in r25047, more to come. This was useful: List all valid kbd layouts, variants and toggle options (to use with setxkbmap)

Thu, 23 Jan 2020 08:41:29 GMT - totaamwin32: owner, status changed

• owner changed from Antoine Martin to akikoo
• status changed from assigned to new

Fixed in r25048 + r25052. @akikoo: does the latest win32 beta build work for you: https://xpra.org/beta/windows/. (try r25052 or later builds from there)

Could be improved further to handle variants: #2561.

Tested OK with 'French', 'German', 'UK', and 'Finnish' layouts and detection worked in all cases. As for the key events, they look correct to me too.

Thu, 23 Jan 2020 11:49:26 GMT - totaamwin32:

For the record, here's what I get on windows 10 when I connect with a Finnish layout:

C:\Program Files\Xpra>Xpra_cmd.exe attach tcp://192.168.0.10:10000
Xpra GTK3 client version 4.0-r25052 64-bit
 running on Microsoft Windows 10
Warning: failed to import opencv:
 No module named 'cv2'
webcam forwarding is disabled
GStreamer version 1.16.2 for Python 3.8.0 64-bit
keyboard layout code 0x40b
identified as 'Finnish' : fi
 keyboard settings: layout=fi
 desktop size is 1919x1440 with 1 screen:
  Default (507x381 mm - DPI: 96x96) workarea: 1919x1400
    (Standard monitor types) Generic Non-PnP Monitor (508x381 mm - DPI: 95x96)
enabled remote logging
Xpra GTK2 X11 server version 3.0.6-r24992 64-bit
 running on Linux Fedora 31 ThirtyOne
Attached to tcp://192.168.0.10:10000/
 (press Control-C to detach)

Thu, 23 Jan 2020 13:23:13 GMT - Antoine Martin: owner, status changed

• owner changed from akikoo to Antoine Martin
• status changed from new to assigned

OK, so the first part is confirmed as fixed and we detect the 'Finnish' keyboard correctly on MS Windows 10. (not going to worry about win7)

Next, the pipe | key, debug output with -d keyboard at both ends:

• client sends AltGr key press:

  send_key_action(1, <GTKKeyEvent object, contents: \
    {'modifiers': ['mod5'], 'keyname': '', 'keyval': -1, 'keycode': -1, 'group': -1, 'string': '', 'pressed': True}>)
  

• server finds the correct keycode to use:

  change_mask(add) ('mod5',) modifier 'mod5' using keycode 92
  

But it also triggers a spurious Alt_L event:

get_keycode(-1, , ('mod5',))=204 (level=2)
fake_key(204, True)

(not sure why we got keycode 204 there... will investigate later)

• the client sends bar:

  send_key_action(1, <GTKKeyEvent object, contents: \
    {'modifiers': ['mod5', 'mod2'], 'keyname': 'bar', 'keyval': 124, 'keycode': 226, 'group': 1, 'string': '|', 'pressed': True}>)
  

• the server does this:

  set_keyboard_layout_group(1) config=KeyboardConfig(fi /  / None), current keyboard group=0
  setting XKB layout group 1
  get_keycode(226, bar, ('mod5', 'mod2'))=31 (level=2)
  ...
  fake_key(31, False)
  

And we choose 31 instead of 94.

keycode  31 = i I i I idotless bar idotless
keycode  94 = less greater less greater bar brokenbar bar

$ xpra info | grep ^keyboard.keysyms.31
keyboard.keysyms.31=((0, 'i'), (1, 'I'), (2, 'i'), (3, 'I'), (4, 'idotless'), (5, 'bar'), (6, 'idotless'))
$ xpra info | grep ^keyboard.keysyms.94
keyboard.keysyms.94=((0, 'less'), (1, 'greater'), (2, 'less'), (3, 'greater'), (4, 'bar'), (5, 'brokenbar'), (6, 'bar'))

This is the important part:

get_keycode(226, bar, ('mod5', 'mod2'))=31 (level=2)

The level is correct: 2 for AltGr but we're ignoring the group! This works for Linux clients because for those we use a native keymap, and X11Keyboard.set_layout_group!

Fixed in r25057. This is a server side fix.

Still TODO:

• key repeat packets need to provide the group (meh: #1724)
• spurious Alt_L event server-side
• stacktrace on exit with 'Finnish' layout during clear_keys_pressed

Thu, 23 Jan 2020 17:13:07 GMT - Antoine Martin:

Here's what the newer servers should print with -d keyboard:

client   1 @16.915 send_key_action(1, <GTKKeyEvent object, contents: \
  {'modifiers': ['mod5', 'mod2'], 'keyname': 'bar', 'keyval': 124, 'keycode': 226, 'group': 1, 'string': '|', 'pressed': True}>)
set_keyboard_layout_group(1) config=KeyboardConfig(fi /  / None), current keyboard group=0
setting XKB layout group 1
get_keycode(226, bar, ('mod5', 'mod2'), 1)=94 (level=6)

Other fixes:

• the warnings from comment:5 are unavoidable: they come from GTK3, this GTK function has had problems in the past: Crash when accessing the "string" attribute of GdkEventKey on 64bit Windows - at least now it no longer crashes!
• the stacktrace was due to some invalid keycodes that we then try to unpress during cleanup, r25059 fixes all that
• the spurious key event bug: don't try to find a keycode for no data: r25060
• also don't set an invalid layout group value: r25061
• key repeat packet is EOL / legacy and unlikely to cause problems, so not going to bother with that

Fri, 24 Jan 2020 15:33:40 GMT - akikoo:

Here's the -d keyboard log when I press altgr+<

2020-01-24 17:16:39,530 client   1 @27.269 parse_key_event(<Gdk.EventKey object at 0x00000000007b3c70 (void at 0x00000000068c4a10)>, True)=<GTKKeyEvent object, contents: {'modifiers': ['mod5'], 'keyname': 'Control_L', 'keyval': 65507, 'keycode': 17, 'group': 0, 'string': '', 'pressed': True}>
2020-01-24 17:16:39,531 client   1 @27.269 handle_key_action(ClientWindow(1), <GTKKeyEvent object, contents: {'modifiers': ['mod5'], 'keyname': 'Control_L', 'keyval': 65507, 'keycode': 17, 'group': 0, 'string': '', 'pressed': True}>) wid=1
2020-01-24 17:16:39,533 client   1 @27.270 parse_key_event(<Gdk.EventKey object at 0x00000000007b3a40 (void at 0x00000000068c4830)>, True)=<GTKKeyEvent object, contents: {'modifiers': ['mod5'], 'keyname': 'Alt_R', 'keyval': 65514, 'keycode': 165, 'group': 0, 'string': '', 'pressed': True}>
2020-01-24 17:16:39,534 client   1 @27.270 handle_key_action(ClientWindow(1), <GTKKeyEvent object, contents: {'modifiers': ['mod5'], 'keyname': 'Alt_R', 'keyval': 65514, 'keycode': 165, 'group': 0, 'string': '', 'pressed': True}>) wid=1
2020-01-24 17:16:39,539 client   1 @27.271 send_key_action(1, <GTKKeyEvent object, contents: {'modifiers': ['mod5'], 'keyname': '', 'keyval': -1, 'keycode': -1, 'group': -1, 'string': '', 'pressed': True}>)
2020-01-24 17:16:39,539 set_keyboard_layout_group(-1) config=KeyboardConfig(fi /  / None), current keyboard group=0
2020-01-24 17:16:39,540 get_keycode(-1, , ('mod5',), -1)=204 (level=2)
2020-01-24 17:16:39,628 client   1 @27.364 parse_key_event(<Gdk.EventKey object at 0x00000000007b3a40 (void at 0x00000000068c48d0)>, True)=<GTKKeyEvent object, contents: {'modifiers': ['mod5'], 'keyname': 'bar', 'keyval': 124, 'keycode': 226, 'group': 0, 'string': '|', 'pressed': True}>
2020-01-24 17:16:39,629 client   1 @27.364 handle_key_action(ClientWindow(1), <GTKKeyEvent object, contents: {'modifiers': ['mod5'], 'keyname': 'bar', 'keyval': 124, 'keycode': 226, 'group': 0, 'string': '|', 'pressed': True}>) wid=1
2020-01-24 17:16:39,630 client   1 @27.365 send_key_action(1, <GTKKeyEvent object, contents: {'modifiers': ['mod5'], 'keyname': 'bar', 'keyval': 124, 'keycode': 226, 'group': 0, 'string': '|', 'pressed': True}>)
2020-01-24 17:16:39,631 set_keyboard_layout_group(0) config=KeyboardConfig(fi /  / None), current keyboard group=0
2020-01-24 17:16:39,631 get_keycode(226, bar)=31 (keyname translation)
2020-01-24 17:16:39,706 client   1 @27.445 parse_key_event(<Gdk.EventKey object at 0x00000000007b3a40 (void at 0x00000000068c4970)>, False)=<GTKKeyEvent object, contents: {'modifiers': ['mod1'], 'keyname': 'Control_L', 'keyval': 65507, 'keycode': 17, 'group': 0, 'string': '', 'pressed': False}>
2020-01-24 17:16:39,712 client   1 @27.445 handle_key_action(ClientWindow(1), <GTKKeyEvent object, contents: {'modifiers': ['mod1'], 'keyname': 'Control_L', 'keyval': 65507, 'keycode': 17, 'group': 0, 'string': '', 'pressed': False}>) wid=1
2020-01-24 17:16:39,714 client   1 @27.446 send_key_action(1, <GTKKeyEvent object, contents: {'modifiers': ['mod1'], 'keyname': 'Control_L', 'keyval': 65507, 'keycode': 17, 'group': 0, 'string': '', 'pressed': False}>)
2020-01-24 17:16:39,715 client   1 @27.446 parse_key_event(<Gdk.EventKey object at 0x00000000007b3cc0 (void at 0x00000000068c4a10)>, False)=<GTKKeyEvent object, contents: {'modifiers': [], 'keyname': 'Alt_R', 'keyval': 65514, 'keycode': 165, 'group': 0, 'string': '', 'pressed': False}>
2020-01-24 17:16:39,716 client   1 @27.446 handle_key_action(ClientWindow(1), <GTKKeyEvent object, contents: {'modifiers': [], 'keyname': 'Alt_R', 'keyval': 65514, 'keycode': 165, 'group': 0, 'string': '', 'pressed': False}>) wid=1
2020-01-24 17:16:39,720 client   1 @27.447 send_key_action(1, <GTKKeyEvent object, contents: {'modifiers': ['mod5'], 'keyname': '', 'keyval': -1, 'keycode': -1, 'group': -1, 'string': '', 'pressed': False}>)
2020-01-24 17:16:39,720 set_keyboard_layout_group(0) config=KeyboardConfig(fi /  / None), current keyboard group=0
2020-01-24 17:16:39,733 set_keyboard_layout_group(-1) config=KeyboardConfig(fi /  / None), current keyboard group=0
2020-01-24 17:16:39,746 client   1 @27.484 parse_key_event(<Gdk.EventKey object at 0x00000000007b3db0 (void at 0x00000000068c4830)>, False)=<GTKKeyEvent object, contents: {'modifiers': [], 'keyname': 'less', 'keyval': 60, 'keycode': 226, 'group': 0, 'string': '<', 'pressed': False}>
2020-01-24 17:16:39,746 client   1 @27.484 handle_key_action(ClientWindow(1), <GTKKeyEvent object, contents: {'modifiers': [], 'keyname': 'less', 'keyval': 60, 'keycode': 226, 'group': 0, 'string': '<', 'pressed': False}>) wid=1
2020-01-24 17:16:39,747 client   1 @27.485 send_key_action(1, <GTKKeyEvent object, contents: {'modifiers': [], 'keyname': 'less', 'keyval': 60, 'keycode': 226, 'group': 0, 'string': '<', 'pressed': False}>)
2020-01-24 17:16:39,748 set_keyboard_layout_group(0) config=KeyboardConfig(fi /  / None), current keyboard group=0

Fri, 24 Jan 2020 16:35:32 GMT - Antoine Martin: owner, status changed

• owner changed from Antoine Martin to akikoo
• status changed from assigned to new

Thank you for your patience!

get_keycode(226, bar)=31 (keyname translation)

That's the fall-through code.

We end up there because your key event doesn't have the group set:

send_key_action(1, <GTKKeyEvent object, contents: \
    {'modifiers': ['mod5'], 'keyname': 'bar', 'keyval': 124, 'keycode': 226, 'group': 0, 'string': '|', 'pressed': True}>)

Whereas mine did (group=1).

So r25073 will try harder to find a matching keycode, matching just the "shift" state, if that's the best we can find, before finally using the fall-through case. This should ensure that we match with keycode 94 and not 31. I don't like this sort of kludgery / loose matching, but I can't think of a better solution and it's not exactly the first time we do this either..

There are updated Fedora builds in the beta area: https://xpra.org/beta. Please close the ticket if that works for you.

Sat, 25 Jan 2020 06:07:50 GMT - Antoine Martin:

More keyboard layout related fixes:

• r25075 handle capslock better (though this actually differs from the docs: x11 keyboard encoding
• r25076 ignore spurious keyname sent by GTK3 clients on mswindows
• r25077 avoid using random keycodes when we have to use keycode translation

Sat, 25 Jan 2020 06:42:23 GMT - akikoo: status changed; resolution set

• status changed from new to closed
• resolution set to fixed

The letters åäö and the pipe character work now without the need to specify --keyboard-layout=fi.

Thu, 06 Feb 2020 03:21:25 GMT - Antoine Martin:

This may help with #2301, see also #2574.

Sat, 23 Jan 2021 05:54:37 GMT - migration script:

this ticket has been moved to: https://github.com/Xpra-org/xpra/issues/2560