Xpra: Ticket #559: handle anti-aliasing and subpixel rendering, honour client component order

Somewhat related to #163: Subpixel rendering (wikipedia)

"RGB is the most common" (you can test your monitor some are!)

We already synchronize the Xresources from Posix clients, so the options Xft.hinting, Xft.rgba, etc should be updated on the server automagically already. We should be able to get the values from X11 if the Xresources are not specified. Conversely we also want to update the X11 screen data (XRender / XRandR) and this won't work for non-Posix clients (osx and win32). We probably even want subpixel rendering by default for non-Posix clients already (as this gives us clearer text).

A serious potential problem is the overwhelming number of files and configuration options which can potentially override what we set... dconf, gsettings, fonts.conf (many potential places!), X11, ... Which means that we may be better off turning it off completely if we can't be sure it will do what we want. See:

Links:

X11 side:



Wed, 16 Apr 2014 07:58:40 GMT - Antoine Martin: owner, status, description changed


Mon, 15 Sep 2014 07:23:13 GMT - Antoine Martin: description changed


Mon, 22 Sep 2014 10:08:58 GMT - Antoine Martin:

r7721 added fontoptions and r7746 added settings information to gtk_util, and r7718 added this display info to the report tool data.

With a dual monitor setup, using a different pixel order on each, I still get the same information out of gtk (at least identical with gtk2 and gtk3):

$ python3 ./xpra/gtk_common/gtk_util.py  | grep -i fontoptions
* screen[0].fontoptions.antialias : subpixel
* screen[0].fontoptions.hint_metrics : on
* screen[0].fontoptions.hint_style : slight
* screen[0].fontoptions.subpixel_order : RGB

Which seems to match xrdb:

$ xrdb  -all -query | grep Xft
Xft.antialias:	1
Xft.dpi:	96
Xft.hinting:	1
Xft.hintstyle:	hintslight
Xft.rgba:	rgb

And gsettings.. more or less:

$ gsettings get org.gnome.settings-daemon.plugins.xsettings antialiasing
'rgba'
$ gsettings get org.gnome.settings-daemon.plugins.xsettings rgba-order
'rgb'
$ gsettings get org.gnome.settings-daemon.plugins.xsettings hinting
'medium'

Since a "screen" is a grouping of two monitors, I was hoping that it would at least disable sub-pixel rendering when things don't match rather than using something which is right for only one monitor..

Monitor info:


Mon, 22 Sep 2014 11:21:20 GMT - Antoine Martin:

As for windows and OSX:

Interesting read: http://bjango.com/articles/subpixeltext/:

So.. not only is it next to impossible to figure out what the current setting is, on some platforms I see no way to access the per-monitor value.

I think the safest thing to do is to disable it.


Mon, 29 Sep 2014 09:56:23 GMT - Antoine Martin: owner, priority, status, milestone changed

I stumbled upon the API as I was dealing with #697 (which is related to this ticket), so r7830 adds an exe wrapper script called NativeGUI_info.exe which will dump the antialias information it finds using the SystemParametersInfo API, amongst other things. It correctly detects my BGR display running windows7, and my virtual RGB display using XP. As of r7834 we can also get similar information for posix systems (Linux).

@afarr: can you please run this on a few win32 systems, preferably those that have 4k screens or screens that have BGR pixel order (those are pretty rare), or vertical / rotated screens - that is, if you can find any... Note: on some systems, you may need to tweak the settings if the machine is not configured properly to begin with! (I was hoping that the OS would read the EDID and configure things automatically, sadly this is not the case and some 4k displays are misconfigured out of the box..)


Tue, 30 Sep 2014 10:16:37 GMT - Antoine Martin:

Implemented in r7845 (+fix for win32 in r7846). Notes:

If that turns out to be a big problem, we can add a command line switch (ie: --antialias=disabled or --antialias=enabled,rgba=bgr,contrast=2000 ..) so we can preset some of those settings before the client connects.

@afarr: please see comment:5. I was hoping to find a straightforward way of verifying that the subpixel hinting is being honoured by taking screenshots of an application rendering some large fonts (I had used fontmatrix) under different conditions:

I was expecting to see more blue on the pixels on the left edges of things (for RGB, red for BGR). It turns out that it's not so clear cut.. For a start, the application still renders slightly differently in each case. Then it uses subtle colour shades..

So, things to check - easy ones first:


Tue, 07 Oct 2014 01:21:21 GMT - alas:

Testing with 0.15.0 r7897 win32 client (windows 8.1, with 4K monitor) vs. same server (Fedora 20).



python xpra/platform/gui.py vs. NativeGUI_info.exe...

The rest were either the same (antialias.enabled, antialias.hinting, double_click.distance, double_click.time, dpi, dpi.x, dpi.y, icon_size, native_tray_menu_helpers, system_bell, workarea), or only displayed by the gui.py with no correlating values on the Win32 side.


xrdb -all -query | grep Xft ...


Visual inspection: Uhh, looks about the same... a little sharper than some of the local applications...



Tue, 07 Oct 2014 02:49:49 GMT - Antoine Martin: status changed; resolution set

NativeGUI_info.exe ... shows RGB (and correct DPI), though rotating screens doesn't seem to introduce VRGB.


Then I guess that mswindows does not detect the change. So our rendering will be wrong, just like all the other applications.. Not much we can do about this.


Wed, 05 Nov 2014 03:18:24 GMT - Antoine Martin:

Note: we may want to update the dummy driver crtc structure with the same pixel order value, see 728#comment:1.


Fri, 01 May 2015 23:37:22 GMT - alas:

Re-tested with win32 client 0.15.0 r9202 against fedora 20 0.15.0 r9137.

DPI is still listing (again) as 96 x 96. I presume trying to get the 144 x 144 will wait for another release.


Sat, 02 May 2015 04:46:27 GMT - Antoine Martin:

@afarr: XP, win7, win8, win8.1? They all behave differently with regards to DPI. Is the patched xorg driver installed? Client output? Server output? etc..


Fri, 06 Nov 2015 01:12:44 GMT - alas:

Double checking now that we're using scaling much more (#976).

Firstly, as mentioned in #697, this windows 8.1 client, when interrogated with NativeGUI_info.exe, always says its DPI is 96 - no matter how I change screen resolution or text size rendering through the control panel. (Increasingly, I'm convinced that SetDpiAwareness is hardwired to an awareness of 96.) Server/client output changes dpi as scaling is shifted.

Running ./xpra/platform/gui.py:


Sun, 08 Nov 2015 13:58:18 GMT - Antoine Martin:

I see the monitor's vertical refresh is now reading at 25 (it was apparently reading at 10 in comment above, while reading at 29 client side). (Oddly, with --encodings=webp, the vertical refresh is back to 10, but with --encodings=jpg the vertical refresh reads as 50???... despite the display having a 29 locally?)


The vertical refresh rate reported by the server should not vary with the encodings used - only with the current screen resolution. Are you certain you are seeing this? The refresh rate reported is pretty much meaningless, it comes from the pre-defined modelines in the xorg.conf we ship. We cannot make it match the client's. (it should always match the current resolution as found with xrandr | grep "*")


Looks like inside session is reading dpi.x & dpi.y at 21 .. and adjusted to 32 when scaling changed the client output likewise)


I don't understand: is there a bug or not? The dpi reported using both guy.py and xrdb -query should always be the same as the client's (taking into account scaling).


Eyeballing with --encodings=webp, it looks like the fuzziness begins to set in at a DPI of 32x32.


This sounds like a bug unrelated to this ticket: no matter which encoding is used, after a brief delay, the auto-refresh should give you 100% the same quality as plain rgb. (and if it is sent using webp lossless, then it doesn't need a refresh at all)


Tried with --antialias=disabled, despite the lack of any indication that it had actually been implemented


As you found out, that's not implemented. Added to my TODO list as this would be quite useful to have:


Fri, 13 Nov 2015 22:43:00 GMT - alas:

The vertical refresh rate reported by the server should not vary with the encodings used - only with the current screen resolution. Are you certain you are seeing this?

Ok, I'm sure I was seeing it, but it wasn't nearly as random as I thought. @64 & @32 dpi (scaling to) - I'm seeing the vertical refresh at 10. @48 dpi vertical refresh is at 50. @24 dpi vertical refresh is 25. And that explains all the numbers I was seeing above, must've been checking at different scalings.


In the meantime, I will attach a number of screenshots of varying encodings at varying dpi, both text from gedit (including some non-English characters) & from a chromium-browser on a relatively random site (abcnews) - grabbing the same bit of text.

Quick summary of my impressions:


Fri, 13 Nov 2015 23:09:49 GMT - alas:

Ohh... one last detail... since the DPI numbers feel so counter-intuitive (at least to me) I'll also include server side display sizes corresponding to the dpi settings.

24 DPI:

2015-11-13 12:37:37,277 client display size is 1280x540 with 1 screen:
2015-11-13 12:37:37,277   2\WinSta-Default (1354x571 mm - DPI: 24x24) workarea: 1280x530
2015-11-13 12:37:37,277     DISPLAY1 960x540 at 320x0 (621x341 mm - DPI: 39x40) workarea: 960x530
2015-11-13 12:37:37,277     DISPLAY2 320x180 (597x336 mm - DPI: 13x13) workarea: 320x160
2015-11-13 12:37:37,308 DPI set to 24 x 24
2015-11-13 12:37:37,310 sent updated screen size to 1 client: 1280x682 (max 8192x4096)

32 DPI

2015-11-13 12:36:26,575 client display size is 1707x720 with 1 screen:
2015-11-13 12:36:26,579   2\WinSta-Default (1354x571 mm - DPI: 32x32) workarea: 1707x707
2015-11-13 12:36:26,579     DISPLAY1 1280x720 at 427x0 (621x341 mm - DPI: 52x53) workarea: 1280x707
2015-11-13 12:36:26,579     DISPLAY2 427x240 (597x336 mm - DPI: 18x18) workarea: 427x213
2015-11-13 12:36:26,612 DPI set to 32 x 32
2015-11-13 12:36:26,613 sent updated screen size to 1 client: 1920x1080 (max 8192x4096)

48 DPI:

2015-11-13 12:36:10,824 client display size is 2560x1080 with 1 screen:
2015-11-13 12:36:10,825   2\WinSta-Default (1354x571 mm - DPI: 48x48) workarea: 2560x1060
2015-11-13 12:36:10,826     DISPLAY1 1920x1080 at 640x0 (621x341 mm - DPI: 78x80) workarea: 1920x1060
2015-11-13 12:36:10,826     DISPLAY2 640x360 (597x336 mm - DPI: 27x27) workarea: 640x319
2015-11-13 12:36:10,877 DPI set to 48 x 48
2015-11-13 12:36:10,881 sent updated screen size to 1 client: 3200x1080 (max 8192x4096)
2015-11-13 12:36:26,574 server virtual display now set to 1920x1080 (best match for 1707x720)

And... 64 DPI:

2015-11-13 12:34:14,581  client root window size is 3413x1440 with 1 display:
2015-11-13 12:34:14,582   2\WinSta-Default (1354x571 mm - DPI: 64x64) workarea: 3413x1413
2015-11-13 12:34:14,582     DISPLAY1 2560x1440 at 853x0 (621x341 mm - DPI: 104x107) workarea: 2560x1413
2015-11-13 12:34:14,582     DISPLAY2 853x480 (597x336 mm - DPI: 36x36) workarea: 853x425
2015-11-13 12:34:14,724 server virtual display now set to 3904x1440 (best match for 3413x1440)

Hopefully, that will improve context.


Sat, 14 Nov 2015 07:05:55 GMT - Antoine Martin: status, summary changed; resolution deleted

Edit: removed lots of screenshots showing nothing useful.

I'm not posting the other screenshots, as I mentioned before, they should all be identical. If they're not, then either the auto-refresh is failing to kick in (a different bug), or a supposedly "lossless" encoding maybe isn't lossless after all (also a different bug).

Note: png/L and png/P are NOT lossless so they are not useful here for comparing anything, only plain png and rgb are lossless. png/L and png/P are palette based encodings and they may drop some of the subtle shades which are used for anti-aliasing.

I am re-opening this ticket because there seems to be misunderstandings about what this feature is meant to do and how to observe it. If the wikipedia article is not clear enough, maybe these pictures will help:

This ticket is not directly about DPI or scaling:

What we really want to compare is the sub-pixel rendering between a locally rendered application (one that honours the sub-pixel hints) and an application started on the server after we have set the correct anti-alias settings (and hopefully honouring our forwarded settings). To do that, the client must have sub-pixel rendering enabled, and it must be detected correctly by our code.

To see what we actually render on the server, you can use "xpra screenshot" or "sync-xvfb" with a screenshot utility like "scrot". An added benefit of doing it this way is that it can be scripted so as to place the window in the exact same location each time, and the capture can then take the exact same region of the screen, which could then be fed to a tool to look for any rendering differences (ie: http://www.imagemagick.org/script/compare.php) Without scaling, this should be 100% identical to what is rendered on the client, both with opengl on or off. (except for cases where the OS decides to upscale / downscale our window - which should not happen now that we claim to be DPI aware)

Scaling will introduce its own anti-aliasing, done by the graphics card as it stretches the pixel buffer (which is almost impossible to control from the application: graphics drivers often offer the ability to overrule the OS and application settings).


Fri, 20 Nov 2015 23:37:45 GMT - alas:

Unfortunately - after trying to carefully compare a variety of screenshots, I'm coming to the conclusion that my eye doesn't care enough to note the fine gradations of detail involved here.

I went through and took a number of screenshots with 0.16.0 r11185 server and client though, and then did the same with 0.16.0 r11304. To my eye the r11304 looks a bit better - both with a browser and with gedit. The local browser isn't much different from the r11304, as far as I can tell... but text in a .rtf file seems significantly crisper (though pasted in Devanagari characters don't display as reliably, for whatever that's worth).

I tried to take some server-side screen shots as well, but they end up so tiny that I have to scale the desktop by 300%, crop, and then scale up another 25% before the text is anywhere near a comparable size to what the 150% scaling on the client side is displaying. I'll include one of those monstrosities with rgb encoding both the r11185 and the r11304... on the off chance you can make any use of those (the shot with the r11304 I even thought to set the server, rather than the client, though with all the scaling and editing, it doesn't seem to make much difference).


Tue, 08 Dec 2015 23:09:01 GMT - alas: attachment set

shot of rgb encoding on rotated monitor (vrgb?) of text rendered in a chrome browser with no scaling


Tue, 08 Dec 2015 23:10:46 GMT - alas: attachment set

shot of rgb encoding on rotated monitor (vrgb?) of text rendered in a chrome browser with 150% scaling


Tue, 08 Dec 2015 23:14:04 GMT - alas: attachment set

shot of rgb encoding on rotated monitor (vrgb?) of text rendered in a chrome browser with 150% scaling and XPRA_SET_ANTIALIAS=1


Tue, 08 Dec 2015 23:52:51 GMT - alas: owner, status changed

Went ahead and rotated a display to get some more screenshots for comparison (added to chart in comment:18).

0.16.0 r11359 client (win32), 0.16.0 r11357 server (fedora 21).

Running /usr/lib64/python2.7/site-packages/xpra/platform/gui.py after connecting with a single display set to portrait orientation, the antialias.orientation is still listing as RGB. Checking the antialias-test page linked above though, I'm at least slightly more than mildly convinced that VRGB antialiasing was active on the display.

Actually used this ticket's comment buffer in a google-chrome window to paste text for screenshots, with and without XPRA_SCALED_FONT_ANTIALIAS=1. Oddly enough, these shots seem clearer to my eye than previous ones... though I'm torn about whether the simple/grey anti-aliasing looks better for these letters when scaled or not.

I did install compare (http://www.imagemagick.org/script/compare.php) on my server though, and scrotted a shot or rgb encoding and webp encoding then tried to create a diff file... it turned into a red mess that I won't bother you with though (and even just using a 2560x1440 display alone, the scrot/xpra screenshots are such a low resolution/large area that any of my attempts to zoom into any image to examine it seems to be useless). I'll read up a bit more on scrot & compare to see if I can't manage to get some finer control over what I'm doing.

In the meantime, I'm going to assign this back to you to see if you're comfortable with the anti-aliasing that you're seeing (one of the links at the top of the ticket mentioned, though, that browsers don't support anti-aliasing, but it mentioned it as of 2005... I'll see if I can find an application that I can be sure is supporting it for comparison).


Thu, 10 Dec 2015 02:13:53 GMT - alas: attachment set

server gedit screenshot (default encodings enabled server side)


Thu, 10 Dec 2015 02:14:46 GMT - alas: attachment set

client side gedit screenshot (--encodings=rgb)


Thu, 10 Dec 2015 02:15:59 GMT - alas: attachment set

compare of server gedit screenshot and client (--encodings=rgb) gedit screenshot


Thu, 10 Dec 2015 02:33:35 GMT - alas:

Well, that wasn't easy.

Scrot supports screenshots dilineated by a mouse, but compare (unsurprisingly) will only compare images that are exactly, pixel for pixel, the same size. I was able to mouse screenshots on server and client which were same pixel dimensions, but had no faith that I hadn't introduced some ghosting into the compare by being a pixel off in one or the other.

Using xdotool, though, I was able to get a window id and then use import -window [window id] screenshot.png to get a precise shot server side.

Unfortunately, Windows client side, the screen shot added a windows frame around the gedit window... so I was left to mouse it until I got it right.

In the process of trying to track the [window id] for the server-side screenshot, though, I noticed that xpra info was indicating window[3].pixel-format=BGRA.

Double checking with both NativeGUI_info.exe and gui.py, both indicated antialias.orientation : RGB.

Triple checking with the link provided at the head of the ticket, http://www.lagom.nl/lcd-test/subpixel.php, I wasn't sure one way or the other.

Quadruple checking with a search which turned up http://www.tomshardware.com/reviews/dell-p2815q-4k-monitor,3950-3.html, though, I think I'm at least mostly convinced that this monitor is RGB.

Did I misunderstand the import of that bit of xpra info?

In any case, it looks like the pixels mostly match from server to client (I suppose I will have to use a linux client to run a local gedit so I can compare both server and client shots... which should be esaier without the windows frame addition.)


Compare of server-side and client-side gedit window (I also uploaded the server and client side screenshots, in case you'd like to look/compare):

compare of server gedit screenshot and client (--encodings=rgb) gedit screenshot


Thu, 10 Dec 2015 05:23:07 GMT - Antoine Martin: owner changed

Did I misunderstand the import of that bit of xpra info?


Yes. BGRA is the pixel format used on the virtual framebuffer. There is no relation between how components are arranged as bytes in memory and how they get converted into light. Think of it this way: a video cable might have the red cable to the left of the blue one, what matters is that the red is in the right place (that red is shown as red, and not blue: #937).


As I have mentioned several times now, but clearly failed to get through: we do not want to compare server and client side. We want to see what effect the anti-aliasing and sub-pixel settings have on the rendering of applications on the server. A client should not be needed at all and in fact only makes things more difficult.

If there are ANY differences between the server and the client (assuming lossless compression is used, or that lossless refresh is given time to kick in), then this is a separate bug, which is very unlikely to have anything to do with the issues in this ticket.


Fri, 11 Dec 2015 00:36:17 GMT - alas: attachment set

server screenshot with all encodings "compare"d against fedora 21 local screenshot


Fri, 11 Dec 2015 00:37:12 GMT - alas: attachment set

--encodings=rgb server screenshot "compare"d against fedora 21 local screenshot


Fri, 11 Dec 2015 00:39:17 GMT - alas: attachment set

--encodings=webp server screenshot "compare"d against fedora 21 local screenshot


Fri, 11 Dec 2015 00:49:37 GMT - alas: owner changed

OOOHHHH... now I feel a bit idiotic. More than a little bit maybe...

Not sure why I kept insisting on trying to insert a client into the mix, but the next step I had meant to take was to also get a local screenshot to run a compare against.

Unfortunately, installing gedit on windows is no good, because the different sized frames make pixel comparisons fail.

It took some time to set up a fedora 21 local machine with a gedit window of the same pixel count... which actually (just in case anyone tries this at home) also required that I use the same process to take screenshots in order for the compare to not be just plain all red.

Oddly enough, the version of the gedit seems to be slightly different between the server vm and the local machine I was using... which is why the title bar and the footer of the screenshots are such a distorted red mess.

That said, once the windows were all the same pixel counts and I did the compares (one with all encodings enabled on the server, once with rgb, once with webp ... though maybe even that distinction was also something you were trying to tell me wasn't important... in any case, I got the shots) - the actual contents match perfectly.

Since they all match the same, I'll just post one of them - with all encodings enabled on the server shot.

server screenshot with all encodings


With a variety of straighter and rounder characters there, they all match perfectly, given the same font... so it looks like the anti aliasing has been working as hoped all along (or at least as of late).


Wed, 23 Dec 2015 16:16:57 GMT - Antoine Martin: owner, milestone changed

I failed at explaining this properly, again. We do not want to compare with a locally running application either!

We want to see what anti-aliasing and subpixel rendering do on the server. For that you need to run the same application (gedit or any application that is supposed to take those settings into account when rendering), and use different settings. (connect with different client settings)

Re-scheduling, I am not going to make any changes to this code for 0.16

See also #1086


Wed, 16 Mar 2016 05:40:46 GMT - Antoine Martin: milestone changed


Tue, 12 Jul 2016 16:52:22 GMT - Antoine Martin: milestone changed

Milestone renamed


Sun, 17 Jul 2016 19:02:58 GMT - Antoine Martin:

See A small post about the v40 interpreter: FreeType? 2.7.0 will ship the new v40 TrueType? instructions interpreter version enabled by default. It finally brings DirectWrite/ClearType?-like rendering to the screen, or "subpixel hinting" as some FreeType? code calls it. Actually, there is no subpixel hinting. Read on.


Tue, 19 Jul 2016 12:00:31 GMT - Antoine Martin:

From the comments section post 24 (must read in full): The only rendering-related patch that Ubuntu applies is to enable subpixel rendering that FreeType? disables by default because it's still patented. .. Subpixel rendering is still patented and therefore disabled in default FT builds, except on Ubuntu and I think Debian Link: On Slight Hinting, Proper Text Rendering, Stem Darkening and LCD Filters

See also #1086 and #1155


Fri, 14 Jul 2017 16:05:14 GMT - Antoine Martin: status changed; resolution set

1 year old, closing. Will follow up in #1580.


Thu, 03 Aug 2017 16:55:44 GMT - Antoine Martin:

(I've deleted lots of attachments that weren't helping)

#1580 now contains a test application which should make this a bit easier. Linux doesn't enable sub-pixel rendering it seems, only gray anti aliasing.


Thu, 10 Aug 2017 15:45:13 GMT - Antoine Martin:

Despite forcing the sub-pixel attributes as per archlinux wiki: font configuration:

gnome.Xft/DPI:  122880
Xft.hinting:    1
Xft.antialias:  1
Xft.rgba:       rgb
Xcursor.size:   24
Xft/DPI:        122880
Xft.dpi:        120
Xft.hintstyle:  hintfull
Xcursor.theme:  Bluecurve

And also modifying the system-wide fontconfig symlinks:

cd /etc/fonts/conf.d/
rm 10-hinting-slight.conf
rm 10-hinting-slight.conf
ln -sf /usr/share/fontconfig/conf.avail/10-hinting-full.conf
ln -sf /usr/share/fontconfig/conf.avail/10-sub-pixel-rgb.conf

Neither our test app (#1580) nor Xft (ie: Why is Xft ignoring font settings?) are honouring the sub-pixel hinting. ie:

pango-view --backend=xft --font='Mono, 12' -t 'Xft - X FreeType interface library'
pango-view --backend=cairo --font='Mono, 12' -t 'Xft - X FreeType interface library'

Sat, 07 Jul 2018 06:26:59 GMT - Antoine Martin:

Follow up: #1086


Wed, 31 Oct 2018 12:12:57 GMT - Antoine Martin:

See Fedora Enables ClearType Subpixel Font Rendering Thanks To Microsoft


Sat, 23 Jan 2021 04:59:21 GMT - migration script:

this ticket has been moved to: https://github.com/Xpra-org/xpra/issues/559