Performance Testing @ the Frontline

A hidden world where small things make a big difference

OpenSource PT tools lack in features

Posted by Kim on Tuesday, April 22, 2008

I’ve been investigating some of the OpenSource performance test tools available on the market, and it seems that most of them are lacking in features that are essential for performance testing in enterprise environments. A good list of both functional and performance tools can be found at www.opensourcetesting.org.

The biggest problem is that the tools are not designed to give out analyzable data in exported form, or even in viewable and understandable form on the screen. An example is soapUI that allows you to run functional test cases directly in multiple threads (to create load) but there is no advanced view of the results, nor can you export the results to anything for later analysis, thus making the feature mostly useless.

What I would need when I do performance testing on a system is a detailed transaction log of every request made from the load-generators point of view, and monitoring results from the target servers (CPU and memory minimum). All these results should then be combined together based on a global time-index to be able to compare them. Without this data it is usually very difficult to determine where the problems are in large-scale enterprise system where you have multiple web and application servers + database servers.

Advertisements

2 Responses to “OpenSource PT tools lack in features”

  1. dave said

    Kim,

    Seems we came to the same conclusion….

    That is why I started this SF project (http://sourceforge.net/projects/testcentral/)

    At the moment it is in a very early stage of development (ALPHA) and I don’t have a lot of time to work on it, like most people I have a full time job, family commitments etc.

    The intention is that this project would provide a repository if you like for automated test results, the results page is live as in as soon as the results are sent to the TC server. Being focused on performance testing the it’s designed for the results from a load test from any load test tool, but should be able to be used by a functional test tool too.

    In your example as SoapUI can perform soap transactions it should be easy to post the response times to the TC server. I have planned to create another service for the load generation machines to use to upload their logs to the Test Central server at the end of the test, but haven’t got to that bit yet.

    So the good news is that there is a solution in the works…..

    Dave.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

 
%d bloggers like this: