Rate this Entry

Benchmarking (Part 5): Linux Hardware/Software Testing and Validation with Phoronix

Posted June 11th, 2008 at 09:26 by chrisplough

Hello!

Okay - so we all know the drill -- I'm a sucker for benchmarking and base a lot of the decisions here at MavenWire on the results (hardware/software/OS directions, performance recommendations, etc). Since we've based our Hosting infrastructure on Linux, we needed a way to quickly and reliably test the performance of various servers and also validate new servers before putting them into service. Luckily, another company (Phoronix Media) had the exact same need and created test suite specifically for this purpose.

You can read the full details in their press release here:

http://www.phoronix-test-suite.com/d...s_release.html

and more details about the test suite here:

http://www.phoronix.com/scan.php?pag...=pts_100&num=1

You can view all of the included tests here

http://tests.phoronix-test-suite.com/

Finally, you can compare your results to other users across the globe here:

http://global.phoronix-test-suite.com/

Now - there are a lot of good features to this test suite - it's comprehensive, repeatable, expandable, etc - but what I like best is that it's easy to setup and implement. As always, this fits well into the MavenWire Benchmark Toolkit, because it is FREE and works very well.

First, some links you'll need:

Phoronix Test Suite
http://www.phoronix-test-suite.com/

To get started, download and the Phoronix Test Suite (PTS). On Red Hat AS Linux 5.x, I had to install the following packages as dependencies: php-common, php-cli, php-gd, freeglut-devel, xorg-x11-proto-devel

Go ahead and install PTS to /opt/benchmarks/phoronix-test-suite/. Then, run the following command to download and install all of the base test packages and dependencies:

Code:

[root@perftest01 phoronix-test-suite]# ./phoronix-test-suite install universe-cli

Once that is complete, navigate to /opt/benchmarks/phoronix-test-suite/ and then run:

Code:

[root@perftest01 phoronix-test-suite]# ./phoronix-test-suite benchmark universe-cli

It will then prompt you for a name to save the results

Code:

Enter a name to save these results: mavenwire-tests

Then it will show you any previous test results that you've saved under that name. In this instance, we've already run mwtest01 and mwtest02 and are inputing mwtest03 as the name of the current series of tests.

Code:

Enter a unique identifier for distinguishing this series of tests: mwtest03

The benchmark will take several minutes to run.

Code:

====================================
Test Suite: universe-cli
====================================


====================================
LAME MP3 Encoding (Run 1 of 3)
====================================

...
...
...

####################################

Results Saved To: /root/.phoronix-test-suite/test-results/mavenwire-tests/composite.xml
Do you want to view the results in your web browser (y/N)? y

Once it is complete, check the output for errors. Once you're sure that you're benchmark runs are clean, then you can reliably run the benchmark.

To ensure I have solid, repeatable results - I run the benchmark 3 times and create a final score by averaging the results of each. If I notice a significant statistical difference from run to run, then I'll invalidate the results and analyze the system to ensure that nothing else is running or that there isn't a problem with the benchmark. For instance, when running this test on one of our medium-sized Hosting servers, the following results were returned:

LAME MP3 Encoding v3.97 (78MB WAV to MP3)
Run 1: 31.72
Run 2: 31.76
Run 3: 31.68
Averaged Result: 31.72
(lower is better)

Ogg Encoding v1.2.0 (78MB WAV to Ogg)
Run 1: 17.60
Run 2: 17.61
Run 3: 17.58
Averaged Result: 17.60
(lower is better)

FLAC Audio Encoding v1.2.1 (78MB WAV to FLAC)
Run 1: 12.32
Run 2: 12.30
Run 3: 12.30
Averaged Result: 12.31
(lower is better)

Monkey Audio Encoding v3.99u4b5 (78MB WAV to APE)
Run 1: 23.45
Run 2: 23.44
Run 3: 23.48
Averaged Result: 23.46
(lower is better)

WavPack Audio Encoding v4.41.0 (78MB WAV to WavPack)
Run 1: 16.01
Run 2: 16.03
Run 3: 16.01
Averaged Result: 16.02
(lower is better)

Mencoder v1.0-rc2 (89MB AVI to LAVC)
Run 1: 20.87
Run 2: 20.88
Run 3: 20.98
Averaged Result: 20.91
(lower is better)

FFmpeg (89MB AVI to NTSC VCD)
Run 1: 19.53
Run 2: 19.52
Run 3: 19.44
Averaged Result: 19.50
(lower is better)

Timed Apache Compilation v2.2.8
Run 1: 5.80
Run 2: 5.79
Run 3: 5.78
Averaged Result: 5.79
(lower is better)

Timed PHP Compilation v5.2.5
Run 1: 32.23
Run 2: 32.20
Run 3: 32.20
Averaged Result: 32.21
(lower is better)

ImageMagick Compilation v6.4.0
Run 1: 39.51
Run 2: 40.56
Run 3: 40.13
Averaged Result: 40.07
(lower is better)

Timed Linux Kernel Compilation v2.6.25
Run 1: 266.60
Run 2: 266.63
Run 3: 266.87
Averaged Result: 266.70
(lower is better)

7-Zip Compression
Run 1: 12844.00
Run 2: 12994.33
Run 3: 12978.66
Averaged Result: 12939.00
(higher is better)

Timed Gzip Compression (512MB)
Run 1: 25.01
Run 2: 24.99
Run 3: 25.17
Averaged Result: 25.06
(lower is better)

Parallel BZIP2 Compression v1.0.2 (512MB)
Run 1: 17.71
Run 2: 17.70
Run 3: 17.42
Averaged Result: 17.61
(lower is better)

Timed LZMA Compression (128MB)
Run 1: 129.50
Run 2: 128.55
Run 3: 129.07
Averaged Result: 129.04
(lower is better)

IOzone v3.291 (write)
Run 1: 288.24
Run 2: 289.02
Run 3: 284.75
Averaged Result: 287.34
(higher is better)

GnuPG v1.4.9 (1GB)
Run 1: 19.40
Run 2: 19.42
Run 3: 19.51
Averaged Result: 19.44
(lower is better)

Super PI v2.0
Run 1: 15.17
Run 2: 15.24
Run 3: 15.08
Averaged Result: 15.16
(lower is better)

OpenSSL v0.9.8g (RSA 4096-bit)
Run 1: 273.77
Run 2: 274.65
Run 3: 274.72
Averaged Result: 274.38
(higher is better)

SQLite v3.5.9 (2500 Inserts)
Run 1: 1.86
Run 2: 1.88
Run 3: 1.88
Averaged Result: 1.87
(lower is better)

PTS Grapher v1.00
Run 1: 83.78
Run 2: 83.11
Run 3: 83.90
Averaged Result: 83.60
(lower is better)

RAMspeed v2.5.1 (Integer BatchRun Add)
Run 1: 2606.28
Run 2: 2612.16
Run 3: 2610.79
Averaged Result: 2609.74
(higher is better)

Bandwidth v0.13 (RAM Sequential Read)
Run 1: 10324.40
Run 2: 11184.80
Run 3: 11184.80
Averaged Result: 10898.00
(higher is better)

Tandem XML v1.0 (Write Performance)
Run 1: 37.16
Run 2: 37.11
Run 3: 37.00
Averaged Result: 37.09
(lower is better)

Tandem XML v1.0 (Read Performance)
Run 1: 24.05
Run 2: 23.84
Run 3: 24.06
Averaged Result: 23.98
(lower is better)

Below is the full PDF created from running the test 3 times against one of our medium-sized Hosting servers:

In order to test other servers, just package up and transfer the /opt/benchmarks/phoronix-test-suite/ directory -- that way there's no need to re-download all of the tests. If you're testing multiple versions of Linux, then I'd go through the whole installation process each time, to ensure that library dependencies are met and the binaries are compiled correctly.

I hope this helps.

--Chris

Other related Posts: