In one of my recent projects I stumbled upon an interesting problem situation with the HTTP Authentication mechanism.

I had a Machine to Machine (M2M) interface, where clients used HTTP authentication to identify themselves to the server while sending data. In this scenario there was no HTTP 401 response from the server, because the client included the authentication info in the initial request.

I discovered that LoadRunner does not do the same. It waits for the HTTP 401 response before actually sending the authentication information. This is correct behavior if we would be simulating a web service meant for humans, but in my case this lead to a situation where LoadRunner was doing 1 extra POST request which had catastrophic results since I now did two POST requests instead of just one!

The negative effect was that I was POST:in the double amount of data over the network and doubling the amount of requests to the server, thus effectively doubling the throughput and connections (load) to the server!

To solve this problem I had to include the Authorization header in the initial request. To do that I used my Base64 Encoder to produce the needed Basic HTTP Authorization header information and then add it to the request using the web_add_header() function.

Below is code snippet from the project that shows how to create and use the custom made authentication header:

	// Must remove std auth mechanism since this causes 2 POSTS instead of one
	// web_set_user( "{Username}", "{Password}", "{Domain}:8000" );

	// Create Base64 encoded string
	b64_encode_string( "{Username}:{Password}", "BasicAuth" );

	// Add HTTP Authorization header "Authorization: Basic XXXXXXXXXXXXXXXXXX==\r\n"
	web_add_header("Authorization", lr_eval_string("Basic {BasicAuth}"));

	lr_start_transaction("Custom_HTTP_Auth");

	web_custom_request("Custom_HTTP_Auth",
		"Method=POST",
		"EncType=text/xml; charset=\"UTF-8\"",
		"URL=http://{Domain}/{Path}/{Document}"
		"BodyFilePath=XML.xml",
       LAST);

	lr_end_transaction("Custom_HTTP_Auth", LR_AUTO);

And I also wish you a Merry X-Mas everybody!

Recently I had the need to decode a Base64 string received from a server and in the process I created an include file that contains both Encode and Decode functions for base64. Use & Abuse as you see fit! Enjoy!
Below is an example of how to use the base64.h include file

# include "base64.h"
vuser_init()
{
 int res;
 // ENCODE
 lr_save_string("ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789","plain");
 b64_encode_string( lr_eval_string("{plain}"), "b64str" );
 lr_output_message("Encoded: %s", lr_eval_string("{b64str}") );

 // DECODE
 b64_decode_string( lr_eval_string("{b64str}"), "plain2" );
 lr_output_message("Decoded: %s", lr_eval_string("{plain2}") );

 // Verify decoded matches original plain text
 res = strcmp( lr_eval_string("{plain}"), lr_eval_string("{plain2}") );
 if (res==0) lr_output_message("Decoded matches original plain text");

 return 0;
}

And here’s the actual base64.h include file

/*
Base 64 Encode and Decode functions for LoadRunner
==================================================
This include file provides functions to Encode and Decode
LoadRunner variables. It's based on source codes found on the
internet and has been modified to work in LoadRunner.

Created by Kim Sandell / Celarius - www.celarius.com
*/
// Encoding lookup table
char base64encode_lut[] = {
'A','B','C','D','E','F','G','H','I','J','K','L','M','N','O','P','Q',
'R','S','T','U','V','W','X','Y','Z','a','b','c','d','e','f','g','h',
'i','j','k','l','m','n','o','p','q','r','s','t','u','v','w','x','y',
'z','0','1','2','3','4','5','6','7','8','9','+','/','='};

// Decode lookup table
char base64decode_lut[] = {
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0,62, 0, 0, 0,63,52,53,54,55,56,57,58,59,60,61, 0, 0,
0, 0, 0, 0, 0, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9,10,11,12,13,14,
15,16,17,18,19,20,21,22,23,24,25, 0, 0, 0, 0, 0, 0,26,27,28,
29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,
49,50,51, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, };

void base64encode(char *src, char *dest, int len)
// Encodes a buffer to base64
{
  int i=0, slen=strlen(src);
  for(i=0;i<slen && i<len;i+=3,src+=3)
  { // Enc next 4 characters
    *(dest++)=base64encode_lut[(*src&0xFC)>>0x2];
    *(dest++)=base64encode_lut[(*src&0x3)<<0x4|(*(src+1)&0xF0)>>0x4];
    *(dest++)=((i+1)<slen)?base64encode_lut[(*(src+1)&0xF)<<0x2|(*(src+2)&0xC0)>>0x6]:'=';
    *(dest++)=((i+2)<slen)?base64encode_lut[*(src+2)&0x3F]:'=';
  }
  *dest=#0; // Append terminator
}

void base64decode(char *src, char *dest, int len)
// Encodes a buffer to base64
{
  int i=0, slen=strlen(src);
  for(i=0;i<slen&&i<len;i+=4,src+=4)
  { // Store next 4 chars in vars for faster access
    char c1=base64decode_lut[*src], c2=base64decode_lut[*(src+1)], c3=base64decode_lut[*(src+2)], c4=base64decode_lut[*(src+3)];
    // Decode to 3 chars
    *(dest++)=(c1&0x3F)<<0x2|(c2&0x30)>>0x4;
    *(dest++)=(c3!=64)?((c2&0xF)<<0x4|(c3&0x3C)>>0x2):'';
    *(dest++)=(c4!=64)?((c3&0x3)<<0x6|c4&0x3F):'';
  }
  *dest=#0; // Append terminator
}

int b64_encode_string( char *source, char *lrvar )
// ----------------------------------------------------------------------------
// Encodes a string to base64 format
//
// Parameters:
//		source    Pointer to source string to encode
//		lrvar     LR variable where base64 encoded string is stored
//
// Example:
//
//		b64_encode_string( "Encode Me!", "b64" )
// ----------------------------------------------------------------------------
{
	int dest_size;
	int res;
	char *dest;
	// Allocate dest buffer
	dest_size = 1 + ((strlen(source)+2)/3*4);
	dest = (char *)malloc(dest_size);
	memset(dest,0,dest_size);
	// Encode & Save
	base64encode(source, dest, dest_size);
	lr_save_string( dest, lrvar );
	// Free dest buffer
	res = strlen(dest);
	free(dest);
	// Return length of dest string
	return res;
}

int b64_decode_string( char *source, char *lrvar )
// ----------------------------------------------------------------------------
// Decodes a base64 string to plaintext
//
// Parameters:
//		source    Pointer to source base64 encoded string
//		lrvar     LR variable where decoded string is stored
//
// Example:
//
//		b64_decode_string( lr_eval_string("{b64}"), "Plain" )
// ----------------------------------------------------------------------------
{
	int dest_size;
	int res;
	char *dest;
	// Allocate dest buffer
	dest_size = strlen(source);
	dest = (char *)malloc(dest_size);
	memset(dest,0,dest_size);
	// Encode & Save
	base64decode(source, dest, dest_size);
	lr_save_string( dest, lrvar );
	// Free dest buffer
	res = strlen(dest);
	free(dest);
	// Return length of dest string
	return res;
}

And so ends the summer & vacations and work begins … Back at the desk solving some the performance problems of the world.

A well performed Golf shot

Yes it’s true, I’m on vacation and not really doing very much at all … at least no real performance testing work!

But I’m active with planning the future for my company, and so we changed our Celarius.com look to more match the new comapny vision. In the near future I will start writing blog articles on the celarius pages as well as here, but they will be more from a commercial standpoint. This blog still remains as my private projects/discoveries etc blog..

And as it is summer and and I performed outstandingly on the golf-course on Friday I just have to include my opening shot on a 150 meter PAR 3 hole. The ball landed 15 cm (6″) from the flag, and my birdie was assured! The Birdie-whiskey we enjoyed afterwards was better than usual!

I hope you all have had a good summer and recharged your battires for a potentially demanding fall’09 !

I’ve had my poll for “Which LoadTesting tool do you use most?” open for some time now, but unfortunately there has been a very low level of participation, and therefore I now ask you to please participate and let us know what tools you are using!

Sometimes it is necessary to process parameters in LoadRunner after they have been “captured” using web_reg_save() or similar methods. There is no built-in function to do this, and I’ve seen multiple horrific attempts at different kinds of Search & Replace methods over the years.

In light of this I decided to post the source for my working solution that does not eat memory, is fast and is LoadRunner parameter friendly. It can be used by anyone who has worked with LoadRunner before!
Examples:

// Store a string into "MyPar" parameter
lr_save_string("This is a string", "MyPar");

// For examples sake, convert it to URL encoded format
web_convert_param( "MyPar",
                   "SourceEncoding=PLAIN",
                   "TargetEncoding=URL", LAST); 

// Output the current result
lr_output_message("%s", lr_eval_string("{MyPar}"));

// Replace the ? characters with %20
lr_replace("MyPar", "+", "%20" );

// Output new result
lr_output_message("%s", lr_eval_string("{MyPar}"));

The following source can be copied into the VUSER_INIT action, at the end or stored as a separate file to be included in the script (your choice!)

// ----------------------------------------------------------------------------
//
// Description:
//    Search for and replace text within a string.
//
// Parameters:
//    src (in) - pointer to source string
//    from (in) - pointer to search text
//    to (in) - pointer to replacement text
//
// Returns:
//    Returns a pointer to dynamically-allocated memory containing string
//    with occurences of the text pointed to by 'from' replaced by with
//    the text pointed to by 'to'.
//
// Notes:
//    Do not use this directly in scripts unless you are a more advanced
//    user who knows C and string handling. See below for the function you
//    should use!
//
// ----------------------------------------------------------------------------
char *strReplace(const char *src, const char *from, const char *to)
{
  char *value;
  char *dst;
  char *match;
  int size;
  int fromlen;
  int tolen;

  // Find out the lengths of the source string, text to replace, and
  // the replacement text.
  size = strlen(src) + 1;
  fromlen = strlen(from);
  tolen = strlen(to);

  // Allocate the first chunk with enough for the original string.
  value = (char *)malloc(size);

  // We need to return 'value', so let's make a copy to mess around with.
  dst = value;

  // Before we begin, let's see if malloc was successful.
  if ( value != NULL )
  {
    // Loop until no matches are found.
    for ( ;; )
    {
      // Try to find the search text.
      match = (char *) strstr(src, from);
      if ( match != NULL )
      {
        // Found search text at location 'match'.
        // Find out how many characters to copy up to the 'match'.
        size_t count = match - src;

        // We are going to realloc, and for that we will need a
        // temporary pointer for safe usage.
        char *temp;

        // Calculate the total size the string will be after the
        // replacement is performed.
        size += tolen - fromlen;

        // Attempt to realloc memory for the new size.
        //
        // temp = realloc(value, size);
        temp = (char *)realloc(value, size);

        if ( temp == NULL )
        {
          // Attempt to realloc failed. Free the previously malloc'd
          // memory and return with our tail between our legs.
          free(value);
          return NULL;
        }

        // The call to realloc was successful. But we'll want to
        // return 'value' eventually, so let's point it to the memory
        // that we are now working with. And let's not forget to point
        // to the right location in the destination as well.
        dst = temp + (dst - value);
        value = temp;

        // Copy from the source to the point where we matched. Then
        // move the source pointer ahead by the amount we copied. And
        // move the destination pointer ahead by the same amount.
        memmove(dst, src, count);
        src += count;
        dst += count;

        // Now copy in the replacement text 'to' at the position of
        // the match. Adjust the source pointer by the text we replaced.
        // Adjust the destination pointer by the amount of replacement
        // text.
        memmove(dst, to, tolen);
        src += fromlen;
        dst += tolen;
      }
      else // No match found.
      {
        // Copy any remaining part of the string. This includes the null
        // termination character.
        strcpy(dst, src);
        break;
      }
    } // For Loop()
  }
  return value;
}

// ----------------------------------------------------------------------------
//
// Description:
//    Find and replace text within a LoadRunner string.
//
// Parameters:
//    lrparam (in)    - pointer to LoadRunner Parameter Name
//    findstr (in)    - pointer to text top search for
//    replacestr (in) - pointer to text to use as replacement
//
// Returns:
//    Returns an integer. 0=Error, 1=Success.
//
// Example:
//    lr_save_string( "This is a small test of the search and replace function", "LRParam");
//    lr_replace( "LRParam", "a", "-x-" );
//    lr_output_message( "%s", lr_eval_string("{LRParam}") );
//
// ----------------------------------------------------------------------------
int lr_replace( const char *lrparam, char *findstr, char *replacestr )
{
  int res = 0;
  char *result_str;
  char lrp[1024];

  // Finalize the LR Param Name
  sprintf( lrp, "{%s}", lrparam);

  // Do the Search and Replace
  result_str = strReplace( lr_eval_string(lrp), findstr, replacestr );

  // Process results
  if (result_str != NULL )
  {
    lr_save_string( result_str, lrparam );
    free( result_str );
    res = 1;
  }
  return res;
} // EOF

Sometimes LoadRunner does not provide all the needed features that one needs when doing performance testing. One option is to write DLL’s to handle the needed stuff and writing your own custom DLL’s is really easy once you get the hang of it!

I needed to do a few HTTP calls “under the radar” from a script so using Delphi 2009 I wrote my own HTTPClient DLL that allows me to do a HTTP GET to a specific URL without adding to the statistics (Pages and Hits/Sec stats).

Creating DLL’s in Delphi is really simple. One of the things I had to keep in mind was that D2009 uses unicode internally so I had to take care to convert any internal strings to non-unicode before returning them to LR. I opted for using the Indy 10 component TIdHTTP as the base HTTP client, adding some Cookie handling and Compression support (gzip,deflate) and making sure the DLL was thread safe (as many client threads/process would be using it). I also soon realized I needed HTTP Proxy Support so in the end I added that too.

Finally had a DLL that exported the following methods:

• http_Initialize( VUserID: Integer )
• http_WebProxy( VUser: Integer; Host:PAnsiChar; Port: Integer )
• http_WebGet( VUserID: Integer; URL: PAnsiChar; DestBuf: PAnsiChar; BufSize: Integer )
• http_Finalize( VUserID: Integer )

The VUserID is the identifier for the connection since the DLL supports keep-alive. Calling Finalize() will destroy all cookies and disconnect the client from the server.

I also made a small sample script that utilizes all the methods:

vuser_init()
{
    int ret, VUserID;
    char buf[10240];

	// Get VUserID
    VUserID = atoi(lr_eval_string("{VUserID"));

	// Load the DLL
    lr_load_dll("HTTPClient.dll");

	// Initialize the VUser object inside the DLL
    http_Initialize( VUserID );

	// Set the HTTP Proxy
	http_WebProxy( VUserID, "127.0.0.1", 8080);

	// Clear buffer, and get the URL's response
	memset(buf,0,sizeof(buf));
    ret = http_WebGet( VUserID, "http://www.celarius.com", buf, sizeof(buf) );

	// Check for error
	// Returns
	//  0 != Insufficient Buffer (Returned value is needed size)
	if (ret != 0) lr_error_message("Error: RetCode=%d", ret)
	else lr_output_message("%s", buf);

	// Finalize the VUser (free the VUser object inside the DLL)
    http_Finalize( VUserID );

	return 0;
}

An additional good thing with this is that the DLL can be loaded under ANY protocol, so now HTTP calls can be made in any script type!

Click here to download v1.0.1.36 of the DLL (295KB)

The last few days I’ve spent optimizing an application written in Delphi 2009 that talks to a Firebird database using the standard TIBDatabase, TIBTransaction and TIBSQL components  The application needs to do alot of reading/writing to the database, and was rather slow so I had to optimize it a little …The application is basically an Indy based Web-Server that receives simple GET parameters, does a thing or two in the DB and returns with the Success/Fail and a possible result from the DB.

My need was to have more than 400 simultaneous connections to the web-server (keep-alive=true) and still maintain good performance in regards to reads/writes. For each command I have to do 3 selects and optionally one delete or insert. Further the business logic dictates that in a certain case only one (1) client can do a SELECT+DELETE and thus guarantee to retrieve a unique row from the table.

After several trial/errors I finally got to a situation where I was able to get 50 hits/sec with 400 active connections, and still retain response times of under 0.3 sec/insert and <1 sec for select+delete operations. Each connection has its own TIBDatabase component, and each SELECT or DELETE had it’s own transaction. Adding more load to the application only resulted in longer response times, but no more than 50 GET commands/sec.

The DB server was a QuadCore (4 x 3.0 GHz XEON) machine and the app is running on the same machine. Local connection to Firebird was with TCP. Firebird was configured to use all 4 cores and HashSlots was increased to 499. The machine avg CPU was around 55%-60%. My guess is that the Disk I/O was the limiting factor.

As soon as I was able to (the download surge killing Win7 downloads prevented it for a while) I downloaded and installed 2 copies of the Win7 beta. One was on an old 3GHz HT Intel with 2Gb mem and the other on a Virtual PC inside my QuadCore Vista ultimate machine.

The installation was smooth on both, took about the same time and I was amazed at the lack of problems along the way. Both recognized all hardware components without problems and everything seemed fast on the GFX level even when I was using a VGA mode for both (no seeing any GFX specialties).

I next installed Firebird v2.1 (latest stable) and it insalled without problems as well on both machines. I didn’t have much time to play around with the OS but the little I did showed me a stable well working system with very little or no hichups at all.

In my personal opinion this Win7 will be a hit-OS in the future. It feels fast and looks good and I see no reason for me to start looking into alternative OS’s in the future. If drivers and programs are made compatible with this Beta I suspect they will work well with the official release and if MS adds WinFS or other enhancements to it later they will likely not break the compatibility in any way.

I for one will surely use it in the future! And a s bonus it seems my Delphi 2009 programs are also working flawlessly on it!

I recently had the opportunity to test a SAP Enterprise Portal (web interface) with several different tools. I used both LoadRunner and QALoad, but in the end the customer chose LR for this, since scripting was somewhat easier in LR.

The performance of the EP portal is fairly good when it comes to communicating with the server on a HTTP protocol level, but boy was the customer in for a surprise when they actually used the interface. The whole EP interface is built around a huge amount of JavaScript (we are talking thousands of lines of code for every page) and is very complicated. It turns out that the client machines rendering the interface was the problem, along with an AntiVirus software that scanned the content. Sometimes drawing the interface on-screen took as much as 20 seconds while the actual transfers of data where in the range of less than 3 seconds.

So all in all, the performance reports shows a good positive thing, whilst the users experienced something totally different. I once again have been reminded that performance testing is not just measuring the servers responses but measuring the end-user experience of the system under test!