xpra icon
Bug tracker and wiki

Opened 6 years ago

Closed 5 years ago

#507 closed defect (fixed)

OSX clipboard fails in latest builds

Reported by: alas Owned by: alas
Priority: blocker Milestone:
Component: client Version: 0.11.x
Keywords: Cc:

Description

As of 0.11 r4998 the osx clipboard works as expected, by r5144 however, copying client-side does not register any change at all with clipboard, primary, or secondary, as indicated by the output of the gtk_clipboard_tool... and obviously will not paste into the xpra session apps at all.

Testing was all done with 0.11 r5257 fedora 19 server.

Attachments (8)

ticket507-clipboardtest2.txt (1.1 KB) - added by alas 6 years ago.
basically empty -d clipboard output, REBUILD r5451
Screenshot_xpra-REBUILD-5451_mouse-offset.png (324.2 KB) - added by alas 6 years ago.
screenshot of offset mouse with GUI launcher, REBUILD r5451
ticket507-r5870-1.txt (13.5 KB) - added by alas 6 years ago.
test of clipboard with r5870 and -d clipboard
507serverclipboard.txt (47.8 KB) - added by J. Max Mena 5 years ago.
-d clipboard output of the server.
507clientclipboard.txt (1.1 KB) - added by J. Max Mena 5 years ago.
-d clipboard output of the client. Copied and pasted from the terminal window.
ticket507-clientlog1.txt (14.1 KB) - added by alas 5 years ago.
client logs, 0.13.8 r6939 against fedora 20 0.14.0 r6936, clipboard failure
ticket507_client-issues-with-google1.txt (91.2 KB) - added by alas 5 years ago.
comparison of command-v pasting on working page vs. (non-working) google.com
ticket507_client-issues-with-google2.txt (27.8 KB) - added by alas 5 years ago.
comparison of non-working command-v pasting vs. working right-click menu pasting, google.com

Download all attachments as: .zip

Change History (43)

comment:1 Changed 6 years ago by Antoine Martin

Owner: changed from Antoine Martin to alas

Please bisect it, only 146 revisions should not take more than 7 steps.

Also, XPRA_CLIPBOARD_DEBUG=1 may shed some light.

Last edited 6 years ago by Antoine Martin (previous) (diff)

comment:2 Changed 6 years ago by Antoine Martin

Priority: majorcritical

Raising priority: I would like to release 0.11.2 tomorrow with a fix for this one.

FYI: OSX only has CLIPBOARD, no PRIMARY or SECONDARY.

Last edited 6 years ago by Antoine Martin (previous) (diff)

comment:3 Changed 6 years ago by alas

Well, some good news. Testing with your osx builds I can't find any sign of this regression, up to and including r5266.

It's not you, it's us. I'll see about tracking which of our versions is an issue.

comment:4 Changed 6 years ago by Antoine Martin

Milestone: 0.12
Owner: changed from alas to afarrr
Priority: criticalblocker

Can I close this ticket?

Or are there new clipboard problems? If so, can you bisect?

comment:5 Changed 6 years ago by alas

The same clipboard problem persists in our builds (at least as of our 0.12 r5828 build).

Went through bisection with Smo, the problem came up between r4998 and r4999 as I recall. We still haven't been able to pin down what happened though.

comment:6 Changed 6 years ago by alas

Checking through your beta osx builds - I find that all of them also have the broken clipoard functionality as well now.

Checking through the stable, I find that -

  • 0.11.3 r5451 has a working clipboard
  • 0.11.4 r5643 has a broken clipboard (which breaks in the same way as the other beta builds, and our builds).

comment:7 Changed 6 years ago by Antoine Martin

That narrows it down a bit.
Unfortunately, looking at the changes in the v0.11.x branch between r5451 and r5643:

$ svn log -r5451:5643 tags/v0.11.x | egrep -v "1 line|---" | sort -u

add latest fix to release notes
fix video compatibility with ancient clients
r5473 for v0.11.x branch: fix nvenc GPU memory leak
r5602 for v0.11.x branch: handle all encodings and not just h264
r5624 (and much more) for v0.11.x branch: report correct rgb encoding to client. \
    this change required backporting of parts of r5605
r5628 for v0.11.x branch: allow all encodings for the tray
r5630 for v0.11.x branch: only remove alpha modes from rgb_formats if we don't handle transparency\!
r5632 for v0.11.x branch: tell the server which rgb formats we handle
r5637 for v0.11.x branch: scale icon before calling set_from_pixbuf, \
    seems to prevent crashes when the icon is very small
r5641 for v0.11.x branch: fix webp loading error handler
stable version bump
updated 0.11.3 release notes
update patch context
update release notes with latest fix

And here is the link to all these changesets: r5451 r5453 r5474 r5586 r5587 r5603 r5604 r5629 r5631 r5633 r5634 r5635 r5639 r5640 r5642 r5643.

I got those using:

svn log -r5451:5643 tags/v0.11.x | grep "^r.* | .* | .* | .* line" | awk '{print $1}' | xargs

There is nothing there even remotely related to OSX or clipboard, so I suspect that the problem is build environment related.


I've uploaded a fresh OSX 0.11.3 build in the beta area (with the suffix "REBUILD" to make it easier to distinguish).
Does this one also fail?


r5846 fixes a bug introduced in r5827 (in trying to tidy up, I made things worse!), which would have prevented the OSX clipboard from working properly, exactly as described... but that's long after the problem got spotted (by about 200 changesets), and only affects trunk (0.12.x), so it may just be another bug with the same symptom.
In any case, I've also uploaded a new beta 0.12.0 build with this change included, does it help?

The changeset above also adds a simple command line tool to make it easier to test the clipboard change detection code.
You can run it with:

python xpra/platform/darwin/osx_clipboard.py

From a jhbuild shell or using the Python interpreter found in the DMG.

I works fine here..
If that also works for you, please try running the client with "-d clipboard" and post the output.

Last edited 6 years ago by Antoine Martin (previous) (diff)

comment:8 Changed 6 years ago by alas

Owner: changed from afarrr to alas

Testing with what seems to be the latest 0.12 beta (r5846), as well as with our own latest build, the clipboard problems continue to persist.

Testing with your 0.11.3-REBUILD (r5451) with -d clipboard on, the clipboard problems also persist, and there doesn't seem to be any output in the log (I copied something within in the session firefox window, then pasted it, then copied something from a local chrome browser, tried unsuccessfully to paste it {pasted the same unchanged xpra clipboard contents}, then copied something else on local chrome and again failed to paste it to the xpra firefox window). I'll include the tiny log file.

Even worse, testing with your 0.11.3-REBUILD (r5451) using the GUI launcher, rather than command line... not only does the clipboard problem persist, but now there is a mouse offset problem as well.

I'm attaching a screenshot which displays what was highlighted when I attempted to highlight the word in the orange box. I'm not sure if you want me to collect output with a -d mouse for you (or even if there is a -d mouse option supported, or if I am even able to collect such a log when launching with the GUI (using the control channel wouldn't allow collection of any client data, correct?).

Changed 6 years ago by alas

basically empty -d clipboard output, REBUILD r5451

Changed 6 years ago by alas

screenshot of offset mouse with GUI launcher, REBUILD r5451

comment:9 Changed 6 years ago by Antoine Martin

  • First, regarding the "mouse issues": it looks to me like you hit the bug fixed by r5852. This should have been obvious in the client output, which would have contained large stacktraces. If that is not the case, please move all the mouse issues to a new "critical" or "blocker" "0.12" ticket as this has nothing to do with the clipboard. r5859 adds -d mouse to some client code, but the mouse position is *extremely* unlikely to be reported wrong, the server-side window position is likely the problem (visible via xpra info).
  • the -d whatever option was added in 0.12: #411, so this won't do anything with 0.11 or older builds. Older builds only support the old debugging technique. ie: using the XPRA_CLIPBOARD_DEBUG environment variable for clipboard
  • please post the output of the test script as per comment:7
  • we only tested 0.11.3-REBUILD to verify that the problem is build or packaging related, rather than a bug introduced in the code. Since the problem occurs with two builds of the exact same code revision, this confirms that the problem is indeed build or packaging related. To investigate further:
    • copy the Xpra.app from the old DMG somewhere and replace the xpra code (Xpra.app/Contents/Resources/lib/python/) with the one from the REBUILD DMG
    • as above but the other way around (REBUILD DMG with older code)

Once we isolate the bug with 0.12, we can worry about fixing 0.11 builds, with a backport if necessary.

  • a missing PyObjC library would result in OSX failing the tests for #492 too, r5858 adds an alternative ctypes implementation for the clipboard changeCount monitoring code. If the clipboard now works and #492 does not, then we know for sure that the problem is with PyObjC packaging. If that's the case, it might be worth considering re-writing the PyObjC code using ctypes (see for example: pyglet's objc_runtime)
Last edited 6 years ago by Antoine Martin (previous) (diff)

comment:10 Changed 6 years ago by alas

  • Mouse issue seems to be gone, testing with GUI launcher with r5870, rather than r5851, so I believe it was the issue that was fixed in r5852 (though, when attaching with the launcher, there is no output client-side... and the issue didn't manifest at all with the CLI, oddly).
  • I should've realized that the 0.11 wouldn't support the new -d clipboard. Ran again with our new r5870 build, and I'll attach that client output.

Quick summary: the 'edu' content that it shows the clipboard picking up was a copy from within the xpra firefox window... I tried to copy twice after from local environment .txt file, which seems to correspond to the two local_clipboard_changed() greedy_client=False outputs. In both cases, when I subsequently tried to paste within the xpra firefox window, the output was that same xpra window copy - 'edu'.

I'll try the copy/swap of the Xpra.apps and post further...

Changed 6 years ago by alas

Attachment: ticket507-r5870-1.txt added

test of clipboard with r5870 and -d clipboard

comment:11 Changed 6 years ago by alas

Ok, that was quicker than I'd feared.

  • Swapping the Xpra.app/Contents/Resources/lib/python between the two 0.11.3 r5451 builds (the old from your dists/osx/x86 directory into the REBUILD from the beta, and vice-versa) ... the old/original 0.11.3 clipboard still works as expected (with the python from the REBUILD), and likewise the REBUILD still breaks as before the swap of the python.

What's the next piece to swap?

comment:12 Changed 6 years ago by Antoine Martin

One bit is still missing:

please post the output of the test script as per comment:7

python xpra/platform/darwin/osx_clipboard.py

(run the script, copy something to the clipboard whilst it's running)
Direct download link: browser/xpra/trunk/src/xpra/platform/darwin/osx_clipboard.py

In the log I see:

ctypes pasteboard access success, current change count=58, setting up timer to watch for changes

So the new ctypes clipboard code has been loaded OK, and seems to detect clipboard changes:

timer_clipboard_check() was 58, now 58
...
2014-03-20 16:12:03,198 clipboard CLIPBOARD set to 'edu'
change count now at 59

At least the ones *we* make.


I also see this:

clipboard_toggled((<XpraClient object at 0x14a37490 (xpra+client+gtk2+client+XpraClient at 0x941200)>,)) \
    enabled=True, server_supports_clipboard=True

Do you use the menu to toggle the clipboard? why?

Last edited 6 years ago by Antoine Martin (previous) (diff)

comment:13 Changed 6 years ago by Antoine Martin

Summary: OSX clipboard reversion in 0.11 between r4998 and r5144OSX clipboard fails in latest builds

Updating bug title to better reflect where the issue is (this is not a regression in the xpra source code).

Last edited 6 years ago by Antoine Martin (previous) (diff)

comment:14 Changed 6 years ago by Antoine Martin

clipboard_toggled..


Can be ignored, I think this happens no matter what when we setup the global menu after connecting.


Here's something that might shed some light on this issue: run the 0.11.3 client (both the OK and REBUILD versions) with XPRA_CLIPBOARD_DEBUG=1 and compare the log output. (remember to turn everything else off: no sound, etc)

comment:15 Changed 6 years ago by alas

I tried to run the osx_clipboard.py tool, after downloading and moving to the Xpra.app/Contents/Resources/lib/python2.7/xpra/platform/darwin directory, and couldn't seem to succeed in running it.

At first it was failing to import gobject, some work to make sure it was using the packaged python in the Helpers directory only succeeded in producing this error:

Traceback (most recent call last):
  File "osx_clipboard.py", line 9, in <module>
    log = Logger("clipboard", "osx")
TypeError: __init__() takes at most 2 arguments (3 given)

... I suppose maybe it should go in a different directory? Or maybe Logger is too recent and I should run it from a 0.12 (maybe r5870?) Xpra.app directory, while running the 0.11.3 r5451 (original and REBUILD), in order to compare?

Anyway, spent too much time on this to try with the XPRA_CLIPBOARD_DEBUG=1 today, will try as soon as I get the chance.

comment:16 Changed 6 years ago by Antoine Martin

and couldn't seem to succeed in running it


If you download this script from trunk (0.12) then you usually need to run it against trunk.

comment:17 Changed 6 years ago by Antoine Martin

Owner: changed from alas to Smo

Just had a quick look at the older builds:

  • 0.11.3 uses GTK 2.24.16
  • later (broken) versions have GTK 2.24.21

So, it is definitely worth trying to rebuild everything with 2.24.16 and see if that fixes the problem. Also worth trying 2.24.23 (the latest) or even the "unstable" modulesets https://git.gnome.org/browse/gtk-osx/tree/modulesets-unstable.

Assuming that at least one version does work... then this probably needs to be bisected down to the problematic change. We don't want to be stuck on an old GTK version.
Another possibility is that newer versions have improved osx clipboard support, and maybe we can drop some of our ugly osx clipboard workarounds. Answering comment:14 should clarify this.

If not:

  • inspect the gtk-osx changelog
  • look for other components that may have changed (pygobject, etc)

comment:18 Changed 6 years ago by Antoine Martin

Any progress?
I've noticed a new clipboard warning I cannot remember seeing before:

unhandled format 0 for clipboard data type NONE

r6153 silences it. (use -d osx or -d clipboard to see it)

This hints that there have been clipboard related changes in recent GTK versions.

comment:19 Changed 6 years ago by Smo

Tried with older GTK by putting

branches["gtk+"] = "http://ftp.gnome.org/pub/gnome/sources/gtk+/2.24/gtk+-2.24.16.tar.xz"

in .jhbuildrc-custom then extracting it manually when it fails on the hash

Still no luck will try latest next

comment:20 Changed 5 years ago by Antoine Martin

See also ticket:533#comment:23 about gtk build updates.

Please try a gtk build older than 2.24.14 as this bug seems relevant GtkClipboard unnotified on change of OS X pasteboard owner (this bug is mentioned in the releases notes for both 2.24.14 and 2.24.15..) - maybe this fix is what broke our workaround (we re-use the same change counter). If that fixes it then at least we know where to look.

comment:21 Changed 5 years ago by Smo

Owner: changed from Smo to alas

jhbuild now uses gtk+-2.24.21

Built with this and sent it to afarr for testing i'll reassign this ticket to him for comments.

comment:22 Changed 5 years ago by J. Max Mena

After retesting with 13.6 r6783 OSX and Fedora 20:

  • Copying from OSX to Xpra works at first
  • Copying from Xpra to OSX works fine as well
  • After copying from inside Xpra, local copies are no longer registered by the Xpra clipboard. That is, any subsequent OSX copies can not be pasted into Xpra. While copies in Xpra are registered and can be pasted into Xpra or OSX.

comment:23 Changed 5 years ago by Antoine Martin

What could be helpful:

  • try with a revert of this osx pasteboard change - I believe this will fix the problem. If so, a minimal diff between the version that works and the one that doesn't would help me figure out how to workaround the workaround.

If not:

  • minimal -d clipboard output comparing a version that does work with one that do not
  • comparing the output of this script version.py when run via the Xpra.app/Contents/Helpers/Python for versions that work and versions that don't
Last edited 5 years ago by Antoine Martin (previous) (diff)

comment:24 Changed 5 years ago by J. Max Mena

Ran some quick tests with -d clipboard with an 11.0 r4998 client against an 13.6 Fedora 20 server. I copied some things into gedit through Xpra, then something different back out into Textdit a few times then ended the session. I'll attach the logs shortly.

Last edited 5 years ago by J. Max Mena (previous) (diff)

Changed 5 years ago by J. Max Mena

Attachment: 507serverclipboard.txt added

-d clipboard output of the server.

Changed 5 years ago by J. Max Mena

Attachment: 507clientclipboard.txt added

-d clipboard output of the client. Copied and pasted from the terminal window.

comment:25 Changed 5 years ago by Antoine Martin

Can you please post the client log again using the new beta 0.13.8 osx build I have just posted here: http://xpra.org/beta/osx/x86/

If that still fails (likely), please re-assign to smo so he can try to:

  • make a build with the pasteboard revert (as per comment:23)
  • build older versions of gtk (as per comment:20)

Bearing in mind what I've just discovered during #533: it may be necessary to wipe the whole gtk installation directory to ensure that there are no remnants of the earlier builds (this can be scripted to run overnight and deliver multiple gtk installations directories for testing)

comment:26 Changed 5 years ago by alas

Owner: changed from alas to Smo

Sure enough, testing with the 0.13.8 beta osx client (against the fedora 20 server from your beta dist, 0.14.0 r6936) ... the clipboard works as before.

Attaching client log.

(I launched with an xterm and gedit start-child, typed "123" into the gedit, copied it and successfully pasted locally, then copied something locally - the random string "ting" - and tried to paste it into the gedit. Clipboard pasted the same "123" that had previously been its contents, rather than the new string which it should've acquired from local clipboard.)

Re-assigning to smo.

Changed 5 years ago by alas

Attachment: ticket507-clientlog1.txt added

client logs, 0.13.8 r6939 against fedora 20 0.14.0 r6936, clipboard failure

comment:27 Changed 5 years ago by Antoine Martin

As part of #533, please try with this patch reverted (on top of latest gtk): pasteboard change.

Raising as blocker.

comment:28 Changed 5 years ago by Smo

https://git.gnome.org/browse/gtk+/patch/?id=bc3f1893aa26761c0009ddc993b48623bcfbe4ed

I believe is the commit I successfully reverted this and I will get it out to testing.

comment:29 Changed 5 years ago by alas

Testing... fails. Clipboard still behaving as before (clipboard works within xpra session until a copy is made in the local environment... first copy in local updates the clipboard within the xpra session, but subsequent copying doesn't update clipboard within the session).

Let me know if any logging is of use (might try some as a test of #627 bug reporting tool).

comment:30 Changed 5 years ago by Antoine Martin

Owner: changed from Smo to Antoine Martin
Status: newassigned

Damn, taking this ticket back.

comment:31 Changed 5 years ago by Antoine Martin

Owner: changed from Antoine Martin to alas
Status: assignednew

Does r7231 fix things? Looks like it here. Please confirm and I will backport.
I've uploaded a new beta build here: https://xpra.org/beta/osx/x86/


If so, then we got completely sidetracked by this statement from the bug description:

As of 0.11 r4998 the osx clipboard works as expected, by r5144 however, copying client-side..


And then we went off in completely the wrong direction (comment:6 onwards) because I thought we had confirmed that the problem was environment related and not in the xpra code (by rebuilding supposedly OK versions and finding they don't work either).

As can be seen in osx_clipboard.py @ rev 4998, the same bug existed in the revision that was originally supposed to be bug-free. How can it ever have worked?
Either it never worked properly, or another bug was somehow hiding this one? (maybe a bug that causes the server to constantly request the clipboard contents? unlikely as it should have been spotted in log samples, but who knows..)

Something went very wrong here and we all wasted a lot of time on this ticket, it would be good to understand what we did wrong so we can avoid making the same mistake in the future.
Problem is that I don't see what we did wrong!

Last edited 5 years ago by Antoine Martin (previous) (diff)

comment:32 Changed 5 years ago by alas

First - the good news. osx client 0.14.0-r7231 (your beta) against fedora 20 0.14.0-r7189:

  • The Clipboard Worked !!

And now for some odd news... it only mostly worked.

For some reason, when using the command-v to paste onto the google.com start/search page... the client woudn't paste. Using the right-click mouse drop-menu to select paste did work though. And the command-v worked to paste everywhere I tried (except google.com start page) on both firefox and chrome.

I collected some client-side -d clipboard logs. One with a comparison of command-v paste success on another page against command-v fails on google.com, the other with just a comparison of command-v paste failure on google.com against right-click menu paste success at same place on same page. I can't figure out what log info corresponds to what though... so I'll just give you a synopsis of what I did and hopefully that will be enough to elucidate. If not, let me know and I'll try to narrow it further.

ticket507 ... test1:

connected to running session
copied text in xpra session (google) (command-c)
pasted locally (command-v)
pasted within same tab (command-v)
copied from local text file (command-c)
pasted in same non-google.com tab (command-v) … works
pasted in google.com search bar (command-v) - failed
pasted in google.com address bar (command-v) - failed
pasted in google.com search bar (right-click mouse drop menu) - worked (but appeared in address bar)
copied new text from local text file (command-c)
pasted into non-google.com window (command-v) - works
pasted into google.com address bar, next to previous pasting (command-v) - failed
pasted into google.com address bar, next to previous pasting (right-click mouse drop menu) - worked.

test 2:

re-connected to same session
copied text from local text file (command-c)
pasted into google.com address bar (command-v) - failed
pasted into google.com address bar (right-click mouse drop menu) - works
  • As for how we got sidetracked, I'm mystified as well. I distinctly remember doing reversion tests to see where, after our r4998 version, the clipboard broke... and I remember being extremely surprised that all the later versions I tried were broken - including our versions of some revisions of yours which actually did work. I think we may even have both a broken and a fixed version of r4998 lying around on a mini-mac somewhere, in case that might be useful for some sort of forensics?

Changed 5 years ago by alas

comparison of command-v pasting on working page vs. (non-working) google.com

Changed 5 years ago by alas

comparison of non-working command-v pasting vs. working right-click menu pasting, google.com

comment:33 Changed 5 years ago by Antoine Martin

The Clipboard Worked !!


Backport in r7256.


And now for some odd news... it only mostly worked.
worked to paste everywhere I tried (except google.com start page)


That sounds like a different bug, maybe a keyboard bug or a window focus problem.
That's why the Testing the clipboard wiki page uses xclip rather than adding the complexity of an application layer.

If you can reproduce a problem with xclip, then we keep this ticket open.
If the problem is specific to an application, then it needs to go in a different ticket.


I distinctly remember doing reversion tests to see where, after our r4998 version, the clipboard broke...
(..)
including our versions of some revisions of yours which actually did work


I still do not understand why that would be...

I think we may even have both a broken and a fixed version of r4998 lying around (..)

You can find older versions here too (all versions have that clipboard bug):
http://xpra.org/dists/osx/x86/

comment:34 Changed 5 years ago by alas

  • Testing again with 0.14.0-r7257 against the same fedora 20 0.14.0-r7189 ... command-c & command-v are working as expected on google.com page (and as expected with xclip as well).

Not sure what was happening, but it doesn't seem to be doing it anymore.

Just to be thorough, tested again with 0.14.0-r7232 against same server (same running server session, in fact)... and command-c and command-v were working as expected there as well.

Maybe it was just a bad internet day? Whatever the case, I think this ticket can be closed at long last.

  • As to the r4998 with fixed vs. broken clipboards ... would you like a copy of that older one that worked to run a diff?

comment:35 Changed 5 years ago by Antoine Martin

Resolution: fixed
Status: newclosed

Not sure what was happening, but it doesn't seem to be doing it anymore
(..)


As to the r4998 with fixed vs. broken clipboards ... would you like a copy of that older one that worked to run a diff?


I have hundreds of builds here already.

Bug. Finally. Closed.

Note: See TracTickets for help on using tickets.