Pressing AltGr+< on a Finnish keyboard layout produces some kind of letter i instead of the pipe character

[akikoo]

Summoning a bug from 8 years ago: #7

I can get the pipe character by pressing AltGr+shift+< but that's not normal in the Finnish keyboard layout.

I can fix the situation with:

    keycode  31 = i I i I bar brokenbar bar

I have to use the option --keyboard-layout=fi, otherwise the layout is totally weird.

/ should be on the left side of right shift, but it's not.

Also missing: åäö

XPRA server version 3.0.3 Fedora 30
XPRA client version 3.0.5
client machine: Windows 10

[Antoine Martin]

• we should detect 'fi' keyboard layout automatically - not sure why that doesn't happen
• look for brokenbar key events

[Antoine Martin]

Finnish Keyboard Layout

[Antoine Martin]

Keyboard layout confirmed as (without the blue keys):

[totaamwin32]

Testing in my win10 VM connecting to a Fedora 31 server, after adding a new keyboard layout and all the keys do show up, though the client shows some warnings parsing the keysyms we get from GTK:

Warning: failed to parse string for key
 keyname=notsign, keycode=223
 'utf-8' codec can't decode byte 0xac in position 0: invalid start byte
Warning: failed to parse string for key
 keyname=Odiaeresis, keycode=192
 'utf-8' codec can't decode byte 0xd6 in position 0: unexpected end of data
Warning: failed to parse string for key
 keyname=Adiaeresis, keycode=222
 'utf-8' codec can't decode byte 0xc4 in position 0: unexpected end of data
Warning: failed to parse string for key
 keyname=odiaeresis, keycode=192
 'utf-8' codec can't decode byte 0xf6 in position 0: invalid start byte
Warning: failed to parse string for key
 keyname=adiaeresis, keycode=222
 'utf-8' codec can't decode byte 0xe4 in position 0: unexpected end of data
Warning: failed to parse string for key
 keyname=Aring, keycode=221
 'utf-8' codec can't decode byte 0xc5 in position 0: unexpected end of data
Warning: failed to parse string for key
 keyname=aring, keycode=221
 'utf-8' codec can't decode byte 0xe5 in position 0: unexpected end of data

Let's fix that too.

As for the missing characters with your setup, there must be something different.
Not sure what, but if I can't reproduce the problem, it's going to be hard to fix.

[akikoo]

/ should be on the left side of right shift, but it's not.

That comment was in relation to the xpra client defaults. If I don't enter --keyboard-layout=fi, setxkbmap -query shows 'us' but the layout is not working as '/' is not on the left side of right shift.

[Antoine Martin]

For correct layout detection, it seems that we need to use ​GetKeyboardLayoutName, definition added in r25046.
Mappings added in r25047, more to come.
This was useful: ​List all valid kbd layouts, variants and toggle options (to use with setxkbmap)

[totaamwin32]

Fixed in r25048 + r25052.
@akikoo: does the latest win32 beta build work for you: https://xpra.org/beta/windows/. (try r25052 or later builds from there)

Could be improved further to handle variants: #2561.

Tested OK with 'French', 'German', 'UK', and 'Finnish' layouts and detection worked in all cases.
As for the key events, they look correct to me too.

[totaamwin32]

For the record, here's what I get on windows 10 when I connect with a Finnish layout:

C:\Program Files\Xpra>Xpra_cmd.exe attach tcp://192.168.0.10:10000
Xpra GTK3 client version 4.0-r25052 64-bit
 running on Microsoft Windows 10
Warning: failed to import opencv:
 No module named 'cv2'
webcam forwarding is disabled
GStreamer version 1.16.2 for Python 3.8.0 64-bit
keyboard layout code 0x40b
identified as 'Finnish' : fi
 keyboard settings: layout=fi
 desktop size is 1919x1440 with 1 screen:
  Default (507x381 mm - DPI: 96x96) workarea: 1919x1400
    (Standard monitor types) Generic Non-PnP Monitor (508x381 mm - DPI: 95x96)
enabled remote logging
Xpra GTK2 X11 server version 3.0.6-r24992 64-bit
 running on Linux Fedora 31 ThirtyOne
Attached to tcp://192.168.0.10:10000/
 (press Control-C to detach)

[Antoine Martin]

OK, so the first part is confirmed as fixed and we detect the 'Finnish' keyboard correctly on MS Windows 10. (not going to worry about win7)

Next, the pipe | key, debug output with -d keyboard at both ends:

• client sends AltGr key press:

  send_key_action(1, <GTKKeyEvent object, contents: \
    {'modifiers': ['mod5'], 'keyname': '', 'keyval': -1, 'keycode': -1, 'group': -1, 'string': '', 'pressed': True}>)
  

• server finds the correct keycode to use:

  change_mask(add) ('mod5',) modifier 'mod5' using keycode 92
  

But it also triggers a spurious Alt_L event:

get_keycode(-1, , ('mod5',))=204 (level=2)
fake_key(204, True)

(not sure why we got keycode 204 there... will investigate later)

• the client sends bar:

  send_key_action(1, <GTKKeyEvent object, contents: \
    {'modifiers': ['mod5', 'mod2'], 'keyname': 'bar', 'keyval': 124, 'keycode': 226, 'group': 1, 'string': '|', 'pressed': True}>)
  

• the server does this:

  set_keyboard_layout_group(1) config=KeyboardConfig(fi /  / None), current keyboard group=0
  setting XKB layout group 1
  get_keycode(226, bar, ('mod5', 'mod2'))=31 (level=2)
  ...
  fake_key(31, False)
  

And we choose 31 instead of 94.

keycode  31 = i I i I idotless bar idotless
keycode  94 = less greater less greater bar brokenbar bar

$ xpra info | grep ^keyboard.keysyms.31
keyboard.keysyms.31=((0, 'i'), (1, 'I'), (2, 'i'), (3, 'I'), (4, 'idotless'), (5, 'bar'), (6, 'idotless'))
$ xpra info | grep ^keyboard.keysyms.94
keyboard.keysyms.94=((0, 'less'), (1, 'greater'), (2, 'less'), (3, 'greater'), (4, 'bar'), (5, 'brokenbar'), (6, 'bar'))

This is the important part:

get_keycode(226, bar, ('mod5', 'mod2'))=31 (level=2)

The level is correct: 2 for AltGr but we're ignoring the group!
This works for Linux clients because for those we use a native keymap, and X11Keyboard.set_layout_group!

Fixed in r25057. This is a server side fix.

Still TODO:

• key repeat packets need to provide the group!
• spurious Alt_L event server-side
• stacktrace on exit with 'Finnish' layout during clear_keys_pressed

[Antoine Martin]

Here's what the newer servers should print with -d keyboard:

client   1 @16.915 send_key_action(1, <GTKKeyEvent object, contents: \
  {'modifiers': ['mod5', 'mod2'], 'keyname': 'bar', 'keyval': 124, 'keycode': 226, 'group': 1, 'string': '|', 'pressed': True}>)
set_keyboard_layout_group(1) config=KeyboardConfig(fi /  / None), current keyboard group=0
setting XKB layout group 1
get_keycode(226, bar, ('mod5', 'mod2'), 1)=94 (level=6)

Other fixes:

• the warnings from comment:5 are unavoidable: they come from GTK3, this GTK function has had problems in the past: ​Crash when accessing the "string" attribute of GdkEventKey on 64bit Windows - at least now it no longer crashes!
• the stacktrace was due to some invalid keycodes that we then try to unpress during cleanup, r25059 fixes all that
• the spurious key event bug: don't try to find a keycode for no data: r25060
• also don't set an invalid layout group value: r25061
• key repeat packet is EOL / legacy and unlikely to cause problems, so not going to bother with that

[akikoo]

Here's the -d keyboard log when I press altgr+<

2020-01-24 17:16:39,530 client   1 @27.269 parse_key_event(<Gdk.EventKey object at 0x00000000007b3c70 (void at 0x00000000068c4a10)>, True)=<GTKKeyEvent object, contents: {'modifiers': ['mod5'], 'keyname': 'Control_L', 'keyval': 65507, 'keycode': 17, 'group': 0, 'string': '', 'pressed': True}>
2020-01-24 17:16:39,531 client   1 @27.269 handle_key_action(ClientWindow(1), <GTKKeyEvent object, contents: {'modifiers': ['mod5'], 'keyname': 'Control_L', 'keyval': 65507, 'keycode': 17, 'group': 0, 'string': '', 'pressed': True}>) wid=1
2020-01-24 17:16:39,533 client   1 @27.270 parse_key_event(<Gdk.EventKey object at 0x00000000007b3a40 (void at 0x00000000068c4830)>, True)=<GTKKeyEvent object, contents: {'modifiers': ['mod5'], 'keyname': 'Alt_R', 'keyval': 65514, 'keycode': 165, 'group': 0, 'string': '', 'pressed': True}>
2020-01-24 17:16:39,534 client   1 @27.270 handle_key_action(ClientWindow(1), <GTKKeyEvent object, contents: {'modifiers': ['mod5'], 'keyname': 'Alt_R', 'keyval': 65514, 'keycode': 165, 'group': 0, 'string': '', 'pressed': True}>) wid=1
2020-01-24 17:16:39,539 client   1 @27.271 send_key_action(1, <GTKKeyEvent object, contents: {'modifiers': ['mod5'], 'keyname': '', 'keyval': -1, 'keycode': -1, 'group': -1, 'string': '', 'pressed': True}>)
2020-01-24 17:16:39,539 set_keyboard_layout_group(-1) config=KeyboardConfig(fi /  / None), current keyboard group=0
2020-01-24 17:16:39,540 get_keycode(-1, , ('mod5',), -1)=204 (level=2)
2020-01-24 17:16:39,628 client   1 @27.364 parse_key_event(<Gdk.EventKey object at 0x00000000007b3a40 (void at 0x00000000068c48d0)>, True)=<GTKKeyEvent object, contents: {'modifiers': ['mod5'], 'keyname': 'bar', 'keyval': 124, 'keycode': 226, 'group': 0, 'string': '|', 'pressed': True}>
2020-01-24 17:16:39,629 client   1 @27.364 handle_key_action(ClientWindow(1), <GTKKeyEvent object, contents: {'modifiers': ['mod5'], 'keyname': 'bar', 'keyval': 124, 'keycode': 226, 'group': 0, 'string': '|', 'pressed': True}>) wid=1
2020-01-24 17:16:39,630 client   1 @27.365 send_key_action(1, <GTKKeyEvent object, contents: {'modifiers': ['mod5'], 'keyname': 'bar', 'keyval': 124, 'keycode': 226, 'group': 0, 'string': '|', 'pressed': True}>)
2020-01-24 17:16:39,631 set_keyboard_layout_group(0) config=KeyboardConfig(fi /  / None), current keyboard group=0
2020-01-24 17:16:39,631 get_keycode(226, bar)=31 (keyname translation)
2020-01-24 17:16:39,706 client   1 @27.445 parse_key_event(<Gdk.EventKey object at 0x00000000007b3a40 (void at 0x00000000068c4970)>, False)=<GTKKeyEvent object, contents: {'modifiers': ['mod1'], 'keyname': 'Control_L', 'keyval': 65507, 'keycode': 17, 'group': 0, 'string': '', 'pressed': False}>
2020-01-24 17:16:39,712 client   1 @27.445 handle_key_action(ClientWindow(1), <GTKKeyEvent object, contents: {'modifiers': ['mod1'], 'keyname': 'Control_L', 'keyval': 65507, 'keycode': 17, 'group': 0, 'string': '', 'pressed': False}>) wid=1
2020-01-24 17:16:39,714 client   1 @27.446 send_key_action(1, <GTKKeyEvent object, contents: {'modifiers': ['mod1'], 'keyname': 'Control_L', 'keyval': 65507, 'keycode': 17, 'group': 0, 'string': '', 'pressed': False}>)
2020-01-24 17:16:39,715 client   1 @27.446 parse_key_event(<Gdk.EventKey object at 0x00000000007b3cc0 (void at 0x00000000068c4a10)>, False)=<GTKKeyEvent object, contents: {'modifiers': [], 'keyname': 'Alt_R', 'keyval': 65514, 'keycode': 165, 'group': 0, 'string': '', 'pressed': False}>
2020-01-24 17:16:39,716 client   1 @27.446 handle_key_action(ClientWindow(1), <GTKKeyEvent object, contents: {'modifiers': [], 'keyname': 'Alt_R', 'keyval': 65514, 'keycode': 165, 'group': 0, 'string': '', 'pressed': False}>) wid=1
2020-01-24 17:16:39,720 client   1 @27.447 send_key_action(1, <GTKKeyEvent object, contents: {'modifiers': ['mod5'], 'keyname': '', 'keyval': -1, 'keycode': -1, 'group': -1, 'string': '', 'pressed': False}>)
2020-01-24 17:16:39,720 set_keyboard_layout_group(0) config=KeyboardConfig(fi /  / None), current keyboard group=0
2020-01-24 17:16:39,733 set_keyboard_layout_group(-1) config=KeyboardConfig(fi /  / None), current keyboard group=0
2020-01-24 17:16:39,746 client   1 @27.484 parse_key_event(<Gdk.EventKey object at 0x00000000007b3db0 (void at 0x00000000068c4830)>, False)=<GTKKeyEvent object, contents: {'modifiers': [], 'keyname': 'less', 'keyval': 60, 'keycode': 226, 'group': 0, 'string': '<', 'pressed': False}>
2020-01-24 17:16:39,746 client   1 @27.484 handle_key_action(ClientWindow(1), <GTKKeyEvent object, contents: {'modifiers': [], 'keyname': 'less', 'keyval': 60, 'keycode': 226, 'group': 0, 'string': '<', 'pressed': False}>) wid=1
2020-01-24 17:16:39,747 client   1 @27.485 send_key_action(1, <GTKKeyEvent object, contents: {'modifiers': [], 'keyname': 'less', 'keyval': 60, 'keycode': 226, 'group': 0, 'string': '<', 'pressed': False}>)
2020-01-24 17:16:39,748 set_keyboard_layout_group(0) config=KeyboardConfig(fi /  / None), current keyboard group=0

[Antoine Martin]

Thank you for your patience!

get_keycode(226, bar)=31 (keyname translation)

That's the fall-through code.

We end up there because your key event doesn't have the group set:

send_key_action(1, <GTKKeyEvent object, contents: \
    {'modifiers': ['mod5'], 'keyname': 'bar', 'keyval': 124, 'keycode': 226, 'group': 0, 'string': '|', 'pressed': True}>)

Whereas mine did (group=1).

So r25073 will try harder to find a matching keycode, matching just the "shift" state, if that's the best we can find, before finally using the fall-through case.
This should ensure that we match with keycode 94 and not 31.
I don't like this sort of kludgery / loose matching, but I can't think of a better solution and it's not exactly the first time we do this either..

There are updated Fedora builds in the beta area: https://xpra.org/beta.
Please close the ticket if that works for you.

[Antoine Martin]

More keyboard layout related fixes:

• r25075 handle capslock better (though this actually differs from the docs: ​x11 keyboard encoding
• r25076 ignore spurious keyname sent by GTK3 clients on mswindows
• r25077 avoid using random keycodes when we have to use keycode translation

[akikoo]

The letters åäö and the pipe character work now without the need to specify --keyboard-layout=fi.

[Antoine Martin]

This may help with #2301, see also #2574.

[migration script]

this ticket has been moved to: ​https://github.com/Xpra-org/xpra/issues/2560