Xpra: Ticket #660: stricter pre-release test plan

What we have right now does not work, we have too many tickets causing regressions (#614), regressions that don't get tested (#658, #655 / #656), then I ended up messing up #651.

We need to find all these issues before the release, not after.

I think I will cut the version from trunk roughly 2 weeks before the actual release date, make a release-candidate build, raise all outstanding tickets to release-blockers and create a tracker ticket to ensure a number of test scenarios are tested properly (platforms, OS / Python / toolkit versions, backwards compatibility, encodings, etc..) - see wiki/Testing.

Sat, 23 Aug 2014 05:41:47 GMT - onlyjob:

I'd like to suggest introducing some post-build unit tests. Long time ago I expressed some concerns about automated testing in #177 so perhaps it could be useful to run at least some tests on every build. We can use xvfb-run to run tests that require X.

Sat, 23 Aug 2014 05:42:18 GMT - onlyjob: cc set

Sun, 24 Aug 2014 04:42:24 GMT - Antoine Martin:

The problem with the tests is that very few of the problems we encountered recently would have been found using automated tests or unit tests.

That said, it can't hurt. So I am going to try to deal with #177.

Sat, 14 Mar 2015 03:38:00 GMT - Antoine Martin: owner, status changed

This one should have been caught: #822 (just start shadow and attach)

Thu, 30 Apr 2015 15:15:41 GMT - Antoine Martin: owner, status changed

The more complete integration tests will be dealt with in #847, #177 is now closed.

I have tagged 0.15.x in r9202 and I am building updated beta packages. Unless something major shows up, I will release 0.15.0 in about 5 days.

Of all the remaining 0.15 tickets, there is only one "blocker" (#770) and even this one will not hold up the release.

@afarr / testers: please take a look at milestone/0.15 and add to this ticket with the various end-to-end tests that you've run to verify that the new features do work, and that we haven't borken any platforms or existing functionality. One tool that is worth using during testing is the "box painting": #760

I've done quite a bit of testing today between total power cuts.. including:

etc, and I'll do more over the next few days. But ultimately, I would like the go-ahead decision to be out of my hands.. (you can call me a coward!)

Fri, 01 May 2015 07:52:04 GMT - onlyjob:

I don't want to hold release but 5 days (not even 5 working days) is too little even to schedule testing... I have most of the next week already booked so it looks like I will be testing 0.15 only after release (which is OK I just wish the notice wouldn't be so short)... Congratulations for new milestone and good luck with release, Antoine. Thanks for your much appreciated hard work on Xpra.

Tue, 05 May 2015 12:28:52 GMT - Antoine Martin:

Actually, I am quite happy to delay things a bit more to ensure more tickets get closed. Tagging 0.15 has allowed me to look at new things in the meantime, which is also very helpful in identifying bugs. And we even have one ticket re-opened: #626

Tue, 28 Jul 2015 06:53:52 GMT - Antoine Martin: status changed; resolution set

Not heard back, closing - will follow up in #847.

Sat, 23 Jan 2021 05:02:14 GMT - migration script:

this ticket has been moved to: https://github.com/Xpra-org/xpra/issues/660