xpra icon
Bug tracker and wiki

Opened 3 years ago

Closed 20 months ago

Last modified 16 months ago

#660 closed task (fixed)

stricter pre-release test plan

Reported by: Antoine Martin Owned by: alas
Priority: major Milestone: 0.15
Component: core Version: trunk
Keywords: Cc: onlyjob@…

Description

What we have right now does not work, we have too many tickets causing regressions (#614), regressions that don't get tested (#658, #655 / #656), then I ended up messing up #651.

We need to find all these issues before the release, not after.

I think I will cut the version from trunk roughly 2 weeks before the actual release date, make a release-candidate build, raise all outstanding tickets to release-blockers and create a tracker ticket to ensure a number of test scenarios are tested properly (platforms, OS / Python / toolkit versions, backwards compatibility, encodings, etc..) - see wiki/Testing.

Change History (8)

comment:1 Changed 3 years ago by onlyjob

I'd like to suggest introducing some post-build unit tests. Long time ago I expressed some concerns about automated testing in #177 so perhaps it could be useful to run at least some tests on every build. We can use xvfb-run to run tests that require X.

comment:2 Changed 3 years ago by onlyjob

Cc: onlyjob@… added

comment:3 Changed 3 years ago by Antoine Martin

The problem with the tests is that very few of the problems we encountered recently would have been found using automated tests or unit tests.

That said, it can't hurt. So I am going to try to deal with #177.

comment:4 Changed 2 years ago by Antoine Martin

Owner: changed from Antoine Martin to Antoine Martin
Status: newassigned

This one should have been caught: #822 (just start shadow and attach)

comment:5 Changed 2 years ago by Antoine Martin

Owner: changed from Antoine Martin to alas
Status: assignednew

The more complete integration tests will be dealt with in #847, #177 is now closed.

I have tagged 0.15.x in r9202 and I am building updated beta packages. Unless something major shows up, I will release 0.15.0 in about 5 days.

Of all the remaining 0.15 tickets, there is only one "blocker" (#770) and even this one will not hold up the release.

@afarr / testers: please take a look at milestone/0.15 and add to this ticket with the various end-to-end tests that you've run to verify that the new features do work, and that we haven't borken any platforms or existing functionality.
One tool that is worth using during testing is the "box painting": #760

I've done quite a bit of testing today between total power cuts.. including:

etc, and I'll do more over the next few days.
But ultimately, I would like the go-ahead decision to be out of my hands.. (you can call me a coward!)

Last edited 16 months ago by Antoine Martin (previous) (diff)

comment:6 Changed 2 years ago by onlyjob

I don't want to hold release but 5 days (not even 5 working days) is too little even to schedule testing... I have most of the next week already booked so it looks like I will be testing 0.15 only after release (which is OK I just wish the notice wouldn't be so short)... Congratulations for new milestone and good luck with release, Antoine. Thanks for your much appreciated hard work on Xpra.

comment:7 Changed 23 months ago by Antoine Martin

Actually, I am quite happy to delay things a bit more to ensure more tickets get closed.
Tagging 0.15 has allowed me to look at new things in the meantime, which is also very helpful in identifying bugs. And we even have one ticket re-opened: #626

comment:8 Changed 20 months ago by Antoine Martin

Resolution: fixed
Status: newclosed

Not heard back, closing - will follow up in #847.

Note: See TracTickets for help on using tickets.