Xpra: Ticket #1665: Keyboard mapping: German keyboard special characters wrong (client: Win7, server:CentOS7)

Unfortunately the keyboard mapping doesn't work well for a german keyboard (CentOS 7 <-> CentOS 7 works well). I've tried and can confirm his observation using different 2.x.x Clients on Win 7. Switching the keyboard (in xpra-context-menu) from "auto" to "de" makes no difference.

Example: The pipe symbol | is AltGR + < on a german keyboard. In an xpra session you only get <. Same for @ (AltGr + q) and others.

Server:

Client:

xev output for local input on server:

    KeyPress event, serial 30, synthetic NO, window 0x4400001,
        root 0x28e, subw 0x0, time 535304170, (389,-605), root:(889,416),
        state 0x10, keycode 108 (keysym 0xfe03, ISO_Level3_Shift), same_screen YES,
        XKeysymToKeycode returns keycode: 92
        XLookupString gives 0 bytes:
        XmbLookupString gives 0 bytes:
        XFilterEvent returns: False
    KeyPress event, serial 33, synthetic NO, window 0x4400001,
        root 0x28e, subw 0x0, time 535304218, (389,-605), root:(889,416),
        state 0x90, keycode 52 (keysym 0xbb, guillemotright), same_screen YES,
        XLookupString gives 2 bytes: (c2 bb) "»"
        XmbLookupString gives 2 bytes: (c2 bb) "»"
        XFilterEvent returns: False
    KeyRelease event, serial 33, synthetic NO, window 0x4400001,
        root 0x28e, subw 0x0, time 535304307, (389,-605), root:(889,416),
        state 0x90, keycode 52 (keysym 0xbb, guillemotright), same_screen YES,
        XLookupString gives 2 bytes: (c2 bb) "»"
        XFilterEvent returns: False
    KeyRelease event, serial 33, synthetic NO, window 0x4400001,
        root 0x28e, subw 0x0, time 535304530, (389,-605), root:(889,416),
        state 0x90, keycode 108 (keysym 0xfe03, ISO_Level3_Shift), same_screen YES,
        XKeysymToKeycode returns keycode: 92
        XLookupString gives 0 bytes:
        XFilterEvent returns: False

xev output in xpra session:

    KeyPress event, serial 33, synthetic NO, window 0xa00001,
        root 0x25d, subw 0x0, time 536295354, (568,425), root:(697,581),
        state 0x10, keycode 37 (keysym 0xffe3, Control_L), same_screen YES,
        XLookupString gives 0 bytes:
        XmbLookupString gives 0 bytes:
        XFilterEvent returns: False
    KeyRelease event, serial 36, synthetic NO, window 0xa00001,
        root 0x25d, subw 0x0, time 536295384, (568,425), root:(697,581),
        state 0x6014, keycode 37 (keysym 0xffe3, Control_L), same_screen YES,
        XLookupString gives 0 bytes:
        XFilterEvent returns: False
    KeyPress event, serial 36, synthetic NO, window 0xa00001,
        root 0x25d, subw 0x0, time 536295384, (568,425), root:(697,581),
        state 0x6010, keycode 77 (keysym 0xff7f, Num_Lock), same_screen YES,
        XLookupString gives 0 bytes:
        XmbLookupString gives 0 bytes:
        XFilterEvent returns: False
    KeyRelease event, serial 36, synthetic NO, window 0xa00001,
        root 0x25d, subw 0x0, time 536295384, (568,425), root:(697,581),
        state 0x6010, keycode 77 (keysym 0xff7f, Num_Lock), same_screen YES,
        XLookupString gives 0 bytes:
        XFilterEvent returns: False
    KeyPress event, serial 36, synthetic NO, window 0xa00001,
        root 0x25d, subw 0x0, time 536295384, (568,425), root:(697,581),
        state 0x6000, keycode 113 (keysym 0xfe03, ISO_Level3_Shift), same_screen YES,
        XLookupString gives 0 bytes:
        XmbLookupString gives 0 bytes:
        XFilterEvent returns: False
    KeyPress event, serial 36, synthetic NO, window 0xa00001,
        root 0x25d, subw 0x0, time 536295547, (568,425), root:(697,581),
        state 0x80, keycode 77 (keysym 0xff7f, Num_Lock), same_screen YES,
        XLookupString gives 0 bytes:
        XmbLookupString gives 0 bytes:
        XFilterEvent returns: False
    KeyRelease event, serial 36, synthetic NO, window 0xa00001,
        root 0x25d, subw 0x0, time 536295547, (568,425), root:(697,581),
        state 0x90, keycode 77 (keysym 0xff7f, Num_Lock), same_screen YES,
        XLookupString gives 0 bytes:
        XFilterEvent returns: False
    KeyPress event, serial 36, synthetic NO, window 0xa00001,
        root 0x25d, subw 0x0, time 536295547, (568,425), root:(697,581),
        state 0x90, keycode 183 (keysym 0x3c, less), same_screen YES,
        XKeysymToKeycode returns keycode: 94
        XLookupString gives 1 bytes: (3c) "<"
        XmbLookupString gives 1 bytes: (3c) "<"
        XFilterEvent returns: False
    KeyRelease event, serial 36, synthetic NO, window 0xa00001,
        root 0x25d, subw 0x0, time 536295629, (568,425), root:(697,581),
        state 0x90, keycode 183 (keysym 0x3c, less), same_screen YES,
        XKeysymToKeycode returns keycode: 94
        XLookupString gives 1 bytes: (3c) "<"
        XFilterEvent returns: False
    KeyPress event, serial 36, synthetic NO, window 0xa00001,
        root 0x25d, subw 0x0, time 536295901, (568,425), root:(697,581),
        state 0x6090, keycode 77 (keysym 0xff7f, Num_Lock), same_screen YES,
        XLookupString gives 0 bytes:
        XmbLookupString gives 0 bytes:
        XFilterEvent returns: False
    KeyRelease event, serial 36, synthetic NO, window 0xa00001,
        root 0x25d, subw 0x0, time 536295901, (568,425), root:(697,581),
        state 0x6090, keycode 77 (keysym 0xff7f, Num_Lock), same_screen YES,
        XLookupString gives 0 bytes:
        XFilterEvent returns: False
    KeyRelease event, serial 36, synthetic NO, window 0xa00001,
        root 0x25d, subw 0x0, time 536297311, (568,425), root:(697,581),
        state 0x6080, keycode 113 (keysym 0xfe03, ISO_Level3_Shift), same_screen YES,
        XLookupString gives 0 bytes:
        XFilterEvent returns: False
    KeyPress event, serial 36, synthetic NO, window 0xa00001,
        root 0x25d, subw 0x0, time 536297312, (568,425), root:(697,581),
        state 0x6000, keycode 77 (keysym 0xff7f, Num_Lock), same_screen YES,
        XLookupString gives 0 bytes:
        XmbLookupString gives 0 bytes:
        XFilterEvent returns: False
    KeyRelease event, serial 36, synthetic NO, window 0xa00001,
        root 0x25d, subw 0x0, time 536297312, (568,425), root:(697,581),
        state 0x6010, keycode 77 (keysym 0xff7f, Num_Lock), same_screen YES,
        XLookupString gives 0 bytes:
        XFilterEvent returns: False

Local xkbmap on server is:

    xkb_keymap {
            xkb_keycodes  { include "evdev+aliases(qwertz)" };
            xkb_types     { include "complete"      };
            xkb_compat    { include "complete"      };
            xkb_symbols   { include "pc+de+inet(evdev)"     };
            xkb_geometry  { include "pc(pc105)"     };
    };
    rules:      evdev
    model:      pc105
    layout:     de

Xpra session xkbmap is:

    xkb_keymap {
    };
    rules:      base
    layout:    de

Logs will be attached.



Tue, 17 Oct 2017 16:14:44 GMT - Alex: attachment set

keyboard info on client


Tue, 17 Oct 2017 16:15:16 GMT - Alex: attachment set

keymap info on client


Tue, 17 Oct 2017 16:19:23 GMT - Alex: attachment set

system info client


Tue, 17 Oct 2017 16:25:15 GMT - Alex: attachment set

system info server


Tue, 17 Oct 2017 16:28:46 GMT - Antoine Martin: status changed

Thanks for the details.

I assume that the keyboard looks like the one in the picture here: https://en.wikipedia.org/wiki/German_keyboard_layout.

We have had problems with AltGr in the past: #62, #602. Hopefully this is not related to this bug: #1578, or even this one: #1607 as this would require a more fundamental change: #1049.

I'll take a look when I get a chance. This may not make the cut for 2.2


Tue, 17 Oct 2017 16:33:07 GMT - Alex: attachment set

"xpra_cmd.exe [...] --debug=keyboard" stderr on client


Tue, 17 Oct 2017 16:45:41 GMT - Alex:

Replying to Antoine Martin:

I assume that the keyboard looks like the one in the picture here: https://en.wikipedia.org/wiki/German_keyboard_layout.

It looks pretty much like that. (At least the pipe and @-sign are in place.)

We have had problems with AltGr in the past: #62, #602.

Now i get where the Level3_shift is coming from ))

Hopefully this is not related to this bug: #1578, or even this one: #1607 as this would require a more fundamental change: #1049.

My understanding of X11-keymap-handling is poor, so please just tell me if you need more info or so.

I'll take a look when I get a chance. This may not make the cut for 2.2

Thanks!)


Mon, 04 Dec 2017 09:07:58 GMT - Antoine Martin:

Some related changes in r17553, r17554, r17555 (debug logging, etc)

Actual fixes in r17556 (see #1380), r17557. Tested with win7 client, Fedora 27 server.

We still have two problems to fix here:


Tue, 05 Dec 2017 17:45:39 GMT - Antoine Martin: attachment set

standard? german keyboard layout


Tue, 05 Dec 2017 18:21:23 GMT - Antoine Martin: priority, description changed

r17569 fixes the issue when overriding keyboard options on non-X11 clients: the options are sent separately, so the server no longer switches to "native" mode when it shouldn't.


The other issue turns out to be much more tricky: the reason why this sort of works is because for non-X11 clients we use a flat keymap, without using level or group. We map the keys we get from the client using the formulae:

index = group*2+level

This works reasonably well in most cases - that is for keyboards without too many levels and groups. But with the german layout, the default X11 keymap we get with setxkbmap de looks like this (using our keymap tool to dump it):

keyval          name                        keycode group   level
51              3                           12      0       0
167             section                     12      0       1
179             threesuperior               12      0       2
163             sterling                    12      0       3

All very straightforward and I believe that this matches the standard layout: standard? german keyboard layout

But the one we get from MS Windows using the same API, looks like this (ouch):

keyval          name                        keycode group   level
51              3                           51      0       0
163             sterling                    51      0       1
51              3                           51      1       0
167             section                     51      1       1
179             threesuperior               51      1       2
34              quotedbl                    51      2       0
51              3                           51      2       1
35              numbersign                  51      2       2

That's 3 groups, with up to 3 levels in groups 1 and 2. No idea how users access the extra groups and levels. And we end up mapping it like this, which isn't quite right:

51              3                           12      0       0
163             sterling                    12      0       1
179             threesuperior               12      0       2
51              3                           12      0       3

And I don't see a way to generate the correct index values from the MS Windows keymap.

This is where it gets more complicated. Here's what we get when we press the keys client side on win32 (ignoring the workarounds we need for properly detecting AltGr..):

This last one differs from the X11 keymap, which gives "sterling" instead.

I believe that what we should be doing is to leave the server-side "de" keymap as it is after we call setxkbmap de (I am oversimplifying here), then:


Wed, 06 Dec 2017 12:11:33 GMT - Antoine Martin: owner, status changed

More fixes - this ticket keeps on giving!

With these changes, we keep the keymap intact and the client can trigger the right keys. These changes should be backwards compatible, and some may be backported (r17574 at least)

This should do, may follow up in #1716.

@Alex: does that work for you?


Sun, 10 Dec 2017 08:04:03 GMT - Antoine Martin:

r17591 reverts the win32 group changes (this only affected the python2 builds, which explains why this went unnoticed): non zero groups are only defined for non-us layouts, causing this regression with us layouts:

client failed to provide any keycodes

This may reduce improvements to the keymap compatibility code when connecting to older server versions.

See also wiki/Keyboard and #1380.


Sat, 16 Dec 2017 08:50:49 GMT - Antoine Martin: owner, status changed

r17573 has caused a regression with my UK layout.


Sat, 16 Dec 2017 11:13:45 GMT - Antoine Martin: status changed; resolution set

Fixed in r17666, backport in r17667.


Sat, 23 Jan 2021 05:30:31 GMT - migration script:

this ticket has been moved to: https://github.com/Xpra-org/xpra/issues/1665