xpra icon
Bug tracker and wiki

Opened 4 years ago

Last modified 2 years ago

#856 assigned enhancement

deal with unicode keysyms

Reported by: Antoine Martin Owned by: Antoine Martin
Priority: trivial Milestone: 3.0
Component: platforms Version: 0.15.x
Keywords: keyboard Cc:

Description (last modified by Antoine Martin)

At the moment I get this warning with osx clients:

the following keysyms are invalid and have not been mapped: U+2030, U+0005, U+0010, U+220F, U+2039, U+2211, U+2248, U+FB01, U+F8FF, U+203A, U+FB02, U+02C6, U+02DA, U+02DC, U+2206, U+2044, U+25CA

These unicode keysym translate to:

Unicode ValueNameSymbol (upper / lower)
2030permille
0005enquiry
0010data link escape
220FZn-ary-product
2239SINGLE LEFT-POINTING ANGLE QUOTATION MARK
2211N-ARY SUMMATION
2248ALMOST EQUAL TO
FB01LATIN SMALL LIGATURE FIFI / fi
F8FFPrivate Use, Last
203ASINGLE RIGHT-POINTING ANGLE QUOTATION MARK
FB01LATIN SMALL LIGATURE FLFI / fl
02C6MODIFIER LETTER CIRCUMFLEX ACCENT
02DARING ABOVE˚
02DCSMALL TILDE˜
2206INCREMENT
2044FRACTION SLASH
25CALOZENGE

At first glance, some of these keysyms already exist:

for x in U+2030, U+0005, U+0010, U+220F, U+2039, U+2211, U+2248, U+FB01, U+F8FF, U+203A, U+FB02, U+02C6, U+02DA, U+02DC, U+2206, U+2044, U+25CA; do
  y=`echo $x|sed s+U.++g | sed s+,++g`;
  echo $y; grep -ri $y  /usr/include/X11/keysymdef.h; done

Finds:

  • #define XK_permille 0x0ad5 /* U+2030 PER MILLE SIGN */
  • #define XK_Ccircumflex 0x02c6 /* U+0108 LATIN CAPITAL LETTER C WITH CIRCUMFLEX */

And for the others: http://www.cl.cam.ac.uk/~mgk25/ucs/keysyms.txt

Change History (11)

comment:1 Changed 4 years ago by Antoine Martin

Description: modified (diff)
Status: newassigned

comment:2 Changed 4 years ago by Antoine Martin

Owner: changed from Antoine Martin to alas
Priority: minortrivial
Status: assignednew

Trivial: done in r9515. Result:

  • there should be no more server warnings about invalid keysyms
  • you should be able to use those keys out of the box

This should be fine to backport, especially since #837 now ensures we should have enough free keycodes to use in the keymap, but since no-one complained about those missing keysyms yet on OSX, I will wait for confirmation.
@afarr: please re-assign to me, I hope you can figure out how to type those keys!

comment:3 Changed 4 years ago by alas

Owner: changed from alas to Antoine Martin

Hmmm... I can't seem to succeed at typing/inputting any of those keysyms... though I'm also no seeing any errors about keycodes (not that I really recall seeing any before).

I was testing with 0.16.0 r10655 win32 client (win8.1) against a fedora 21 0.16.0 r10624 server.

I tried the input methods listed at http://www.fileformat.info/tip/microsoft/enter_unicode.htm, as well as the usual ALt - 0233 which, on my local windows applications, will output as an é.

Running the gtk_view_keyboard.py tool server side, the keys seem to be inputting as expected (as per that page's instructions for input and the keysyms I enter as I usually use them.

I even went so far as to set a new value in my registry (method 1) - which allowed me to enter a new set of keysyms in my local text files, but I was still neither able to enter in a gedit window or a browser window. (Note - the ALT key, in combo with numpad keys, seems to do some wacky things, shifting between tabs, maximizing windows and zooming in in dysfunctional ways... all sorts of good things.)

In a last desperate attempt, I tried the instructions from http://askubuntu.com/questions/31258/how-can-i-type-a-unicode-character-for-example-em-dash ... but again nothing, except a little extra stretching of my hands while trying to hold the ctrl+shift+u keys while entering 2014...

Re-assigning to you, hoping for some inspiration.

  • Ohh, one last detail, amongst all the wacky things that happened, some combination in the vicinity of alt shift + (non-numpad attempt) 023 3.. and whatever additional keys I might've bumped in the process, also started creating some interesting errors client & server side (which may not be relevant, since it was almost literally physically impossible to hit the key combination).

Client side:

2015-09-17 17:10:06,365 do_paint_rgb24 error
Traceback (most recent call last):
  File "xpra\client\window_backing_base.pyc", line 277, in do_paint_rgb24
  File "xpra\client\gl\gl_window_backing_base.pyc", line 622, in _do_paint_rgb24
  File "xpra\client\gl\gl_window_backing_base.pyc", line 635, in _do_paint_rgb
  File "xpra\client\gl\gl_window_backing_base.pyc", line 363, in gl_init
  File "errorchecker.pyx", line 53, in OpenGL_accelerate.errorchecker._ErrorChecker.glCheckError (c:\Users\mcfletch\OpenGL-dev\OpenGL-ctypes\OpenGL_accelerate\src\er
rorchecker.c:1218)
GLError: GLError(
        err = 1281,
        description = 'invalid value',
        baseOperation = glOrtho,
        cArguments = (0.0, 0, 0, 0.0, -1.0, 1.0)
)

Server-side:

2015-09-17 17:10:07,617 client_decode_error: gl_paint_planar error: GLError( err=1281, description = 'invalid value', baseOperation = glOrtho ) (-1)
2015-09-17 17:10:07,638 client_decode_error: do_paint_rgb24 error: GLError(
        err = 1281,
        description = 'invalid value',
        baseOperation = glOrtho,
        cArguments = (0.0, 0, 0, 0.0, -1.0, 1.0)
) (-1)

comment:4 Changed 4 years ago by Antoine Martin

Owner: changed from Antoine Martin to alas

Hmmm... I can't seem to succeed at typing/inputting any of those keysyms..


This only affects OSX clients.

Since the OS claims to have those keysyms, it should be possible to trigger them.
The Helpers/Keymap_info tool may help here:

$ ./Desktop/Xpra.app/Contents/Helpers/Keymap_info | egrep "U+|group"
keyval            name              keycode           group             level
16785992          U+2248            7                 1                 0
16786890          U+25CA            9                 1                 1
16785937          U+2211            13                1                 0
16785456          U+2030            14                1                 1
16785476          U+2044            18                1                 1
16785465          U+2039            20                1                 1
16785466          U+203A            21                1                 1
16841474          U+FB02            22                1                 1
16841473          U+FB01            23                1                 1
16785935          U+220F            35                1                 1
16785926          U+2206            38                1                 0
16777946          U+02DA            40                1                 0
16840959          U+F8FF            40                1                 1
16777926          U+02C6            45                1                 1
16777948          U+02DC            46                1                 1
16777232          U+0010            102               0                 0
16777232          U+0010            104               0                 0
16777232          U+0010            108               0                 0
16777232          U+0010            110               0                 0
16777232          U+0010            112               0                 0
16777221          U+0005            114               0                 0

You can just run Keymap_info without the grep and the order seems to match the layout of the keyboard: 'a' (and its group + level variants), followed by 's' (and its variants), etc...

With the full keymap output, I see some of those unicode characters on the keys 'x', 'v', 'w', 'e', exclam, sterling, '4', '5', '6', 'P', 'j', 'k', 'm' and some function keys. (I am using a UK layout for this virtual machine, which may be slightly different from your US layout).

You just have to figure out how to hit "group=1" (Alt or Altgr - or whatever it is called on osx), which is where most of those unicode keys are found with my layout: only the function keys use "group=0". Some use "level=1" which normally means using the shift key.

amongst all the wacky things that happened, some combination in the vicinity of alt shift + ...


That's the new shortcut used for scaling: see #976, in particular ticket:976#comment:5

You should have seen the window flash and resize briefly too.
Please move the bug report there. r10660 should trap this problem a little earlier (just before it paints), but I would probably need -d scaling,opengl to debug this properly - it should not happen.

Server-side: ... client_decode_error...


That's the same error, but less detailed, being reported back to the server (new feature in 0.16 - this is different from remote logging, only applies to paint errors).

comment:5 Changed 3 years ago by alas

Owner: changed from alas to Antoine Martin

Ok... that took some random keyboard mashing to stumble across the answer.

Testing out of the box with 0.16.0 r11077 osx client against 0.16.0 r11031 fedora 21 server...

The first on the initial list of the ticket, I have a feeling that the server is not figuring these out (perhaps that's expected and you just need server interpretations to remap?)... so I will endeavour to make this as organized as possible, by just taking that same chart and adding the keystrokes to trigger client side, and the keys that the server thinks it is seeing.


Unicode ValueNameSymbol (upper / lower)client keystroke comboserver keyboard tool interpretation
2030permille shift-option-rShift-Meta-R
0005enquiry ??(U0005 = 114; F15 = 113 & Home = 115???
0010data link escape ??(U0010 = 112; F15 = 113, F14 = 107???
220Fn-ary-productShift-Option-PShift-Meta-P
2239SINGLE LEFT-POINTING ANGLE QUOTATION MARKShift-Option-3Shift-Meta-numbersign
2211N-ARY SUMMATIONOption-wAlt-w
2248ALMOST EQUAL TOOption-xAlt-x
FB01LATIN SMALL LIGATURE FIFI / fiShift-Option-5Shift-Meta-percent
F8FFPrivate Use, Last Shift-Option-KShift-Meta-K
203ASINGLE RIGHT-POINTING ANGLE QUOTATION MARKShift-Option-4Shift-Meta-dollar
FB02LATIN SMALL LIGATURE FLFI / flShift-Option-6Shift-Meta-percent (Seems to display the same as Shift-Option-5/FB01)
02C6MODIFIER LETTER CIRCUMFLEX ACCENTˆShift-Option-iShift-Meta-I
02DARING ABOVE˚Option-kAlt-k
02DCSMALL TILDE˜Shift-Option-NShift-Meta-N
2206INCREMENTOption-jAlt-j
2044FRACTION SLASHShift-Option-1Shift-Meta-exclam
25CALOZENGEShift-Option-VShift-Meta-V


That seems to cover everything in the initial summary and the list in comment:4. Hopefully that's all of them (I'd really rather not think about how long it took to find the key combos that corresponded to the "group" and "level" flags...). I couldn't find keys that mapped to the '114' or '112' values either with a laptop keyboard or with an attached 'proper' OSX keyboard though - so those values aren't hiding in the OSX numpad either, for whatever that's worth.

Passing this back to you to shoehorn into the mappings for OSX clients when next you visit the keyboard code.

comment:6 Changed 3 years ago by Antoine Martin

Milestone: 0.161.0
Status: newassigned

If the text area doesn't show the requested character then maybe the location where we mapped those unicode "U+" characters is not correct.

$ DISPLAY=:10 xmodmap -pke | egrep -i "0x25ca|0x0005|0x0010|0x220f|0x2239|0x2211|0x2248|0xfb01|0xf8ff|0x203a|0xfb02|0x02c6|0x02da|0x0dc|0x2206|0x25ca"
keycode  13 = 4 dollar cent 0x203a
keycode  14 = 5 percent infinity 0xfb01
keycode  15 = 6 asciicircum section 0xfb02
keycode  25 = w W 0x2211 doublelowquotemark
keycode  33 = p P Greek_pi 0x220f
keycode  44 = j J 0x2206 Ocircumflex
keycode  45 = k K 0x02da 0xf8ff
keycode  53 = x X 0x2248 Ugrave
keycode  55 = v V radical 0x25ca
keycode  93 = 0x0010 NoSymbol 0x0010 NoSymbol 0x0010
keycode 101 = 0x0010 NoSymbol 0x0010 NoSymbol 0x0010
keycode 120 = 0x0010 NoSymbol 0x0010 NoSymbol 0x0010
keycode 121 = 0x0010 NoSymbol 0x0010 NoSymbol 0x0010
keycode 122 = 0x0010 NoSymbol 0x0010 NoSymbol 0x0010
keycode 129 = 0x0005 NoSymbol 0x0005 NoSymbol 0x0005

This looks mostly right though... so maybe "meta" does not trigger the group switch?
Maybe the Control-Command key swap is interfering? Maybe the modifiers aren't mapped properly?

No time for this though, re-scheduling.

comment:7 Changed 3 years ago by alas

It looks like the Option key is the... key... on OSX.

Just for future reference, when I click the Option key:

  • On client - reads as
    down  Alt_L            65513    58    1 1 []
    up    Alt_L            65513    58    1 0 ['1']
    
  • On server - reads as
    down    Alt_L        65513    64    1 0 ['2']
    up      Alt_L        65513    64    1 0 ['2', '1']
    

comment:8 Changed 3 years ago by Antoine Martin

Just for future reference, when I click the Option key:
...


The only difference between the client and server is mod2, which usually maps to:

$ DISPLAY=:10 xmodmap -pm | grep mod2
mod2        Num_Lock (0x4d)

So numlock may not synced properly, no big deal.

comment:9 Changed 3 years ago by Antoine Martin

Milestone: 1.01.1

Milestone renamed

comment:10 Changed 3 years ago by Antoine Martin

Milestone: 1.12.0

Milestone renamed

comment:11 Changed 2 years ago by Antoine Martin

Milestone: 2.03.0

Using {0x25CA aka LOZENGE as an example, I can see that we're sending the right values and the server picks up the correct keycode (skipping modifiers setup steps):

client 1: mask_to_names(<flags GDK_SHIFT_MASK | GDK_MOD1_MASK of type GdkModifierType>)=['mod1', 'shift', 'mod2']
client 1: parse_key_event(<gtk.gdk.Event at 0x123f158f0: GDK_KEY_PRESS keyval=U+25CA>, True)= \
    <GTKKeyEvent object, contents: {'modifiers': ['mod1', 'shift', 'mod2'], 'group': 1, 'string': '\xe2\x97\x8a', \
     'keyname': 'U+25CA', 'pressed': True, 'keyval': 16786890, 'keycode': 9}>
client 1: handle_key_action(ClientWindow(3), <GTKKeyEvent object, contents: \
    {'modifiers': ['mod1', 'shift', 'mod2'], 'group': 1, 'string': '\xe2\x97\x8a', 'keyname': 'U+25CA', \
     'pressed': True, 'keyval': 16786890, 'keycode': 9}>) wid=3
client 1: send_key_action(3, <GTKKeyEvent object, contents: \
    {'modifiers': ['mod1', 'shift', 'mod2'], 'group': 1, 'string': '\xe2\x97\x8a', 'keyname': 'U+25CA', \
     'pressed': True, 'keyval': 16786890, 'keycode': 9}>)
get_keycode(9, U+25CA, ('mod1', 'shift', 'mod2')) is_native_keymap=False, found using translation: 55
handle_key(3,True,U+25CA,16786890,55,('mod1', 'shift', 'mod2')) keyboard_sync=False
handle keycode pressing 55: key U+25CA
fake_key(55, True)

And xev sees the keysym as "0x25ca (no name)", but the XLookupString doesn't see the keysym string.

The server mapping also looks correct:

$ DISPLAY=:3 xmodmap -pke | grep " 55"
keycode  55 = v V radical 0x25ca

Do we need to find a free keysym and rebind it to the string we want using XRebindKeysym.

Note: See TracTickets for help on using tickets.