Changes between Version 28 and Version 29 of Testing
- Timestamp:
- 11/28/15 10:07:10 (5 years ago)
Legend:
- Unmodified
- Added
- Removed
- Modified
-
Testing
v28 v29 36 36 {{{#!div class="box" 37 37 == Automated Performance and Regression Testing == 38 The [ http://xpra.org/trac/browser/xpra/trunk/src/tests/xpra/test_measure_perf.py xpra.test_measure_perf] script can be used to run a variety of client applications within an xpra session (or optionally vnc) and get statistics on how well the encoding performed. The data is printed out at the end in CSV format which you can then import into any tool to compare results - you can find some examples generated using [http://www.sofastatistics.com/home.php sofastats] [http://xpra.org/stats/ here] and [/ticket/147#comment:11 here]. There is a facility for generating charts directly from the CSV data, using a script that we now provide, described below.38 The [/browser/xpra/trunk/src/tests/xpra/test_measure_perf.py xpra.test_measure_perf] script can be used to run a variety of client applications within an xpra session (or optionally vnc) and get statistics on how well the encoding performed. The data is printed out at the end in CSV format which you can then import into any tool to compare results - you can find some examples generated using [http://www.sofastatistics.com/home.php sofastats] [http://xpra.org/stats/ here] and [/ticket/147#comment:11 here]. There is a facility for generating charts directly from the CSV data, using a script that we now provide, described below. 39 39 40 40 It can also be useful to redirect the test's output to a log file to verify that none of the tests failed with any exceptions/errors (looking for exception messages in the log afterwards). 41 41 42 At the moment it does not have a command line interface, and all the options have to be edited directly. However, we have improved that process by splitting out the configuration data into a separate file: [ http://xpra.org/trac/browser/xpra/trunk/src/tests/xpra/perf_config_default.py xpra.perf_config_default.py].42 At the moment it does not have a command line interface, and all the options have to be edited directly. However, we have improved that process by splitting out the configuration data into a separate file: [/browser/xpra/trunk/src/tests/xpra/perf_config_default.py xpra.perf_config_default.py]. 43 43 44 44 Note: to take advantage of iptables packet accounting (mostly for comparing with VNC which does not provide this metric), follow the error message and setup iptables rules to match the port being used in the tests, ie: by default: … … 57 57 {{{#!div class="box" 58 58 == Generating Charts from Performance Test Results == 59 To create multiple output files which can be used to generate charts, using [ http://xpra.org/trac/browser/xpra/trunk/src/tests/xpra/test_measure_perf_charts.py xpra.test_measure_perf_charts]:60 * Build a config class by taking a copy from [ http://xpra.org/trac/browser/xpra/trunk/src/tests/xpra/perf_config_default.py xpra.perf_config_default.py], then making changes as necessary.59 To create multiple output files which can be used to generate charts, using [/browser/xpra/trunk/src/tests/xpra/test_measure_perf_charts.py xpra.test_measure_perf_charts]: 60 * Build a config class by taking a copy from [/browser/xpra/trunk/src/tests/xpra/perf_config_default.py xpra.perf_config_default.py], then making changes as necessary. 61 61 * Determine the values of the following variables: prefix: (a string to identify the data set), ID: (a string to identify the variable that the data set is testing, for example '14', for testing version 14), repetitions: the number of times you want to run the tests. 62 62 * The data file names you will produce will then be in the format: prefix_id_rep.csv. … … 73 73 The optional arguments "1 14" and "2 14" pass the rep and ID values in to the script, which will store them in the "Custom Params" column of the data. 74 74 75 After running your script, open your copy of [ http://xpra.org/trac/browser/xpra/trunk/src/tests/xpra/test_measure_perf_charts.py xpra.test_measure_perf_charts] and change the variables at the top of the file so the script will point to the series of data files that you have generated.75 After running your script, open your copy of [/browser/xpra/trunk/src/tests/xpra/test_measure_perf_charts.py xpra.test_measure_perf_charts] and change the variables at the top of the file so the script will point to the series of data files that you have generated. 76 76 77 77 These variables are described in the script. Once this has been done, run the script on the command line, and an HTML file will be generated containing your charts.