Powered by discountASP.NET
referal ID: sdtref
Why recommend discountASP.NET?

Archives
Steve Trefethen Steve's RSS Feed Subscribe or via email
What's this?
Contact me Send mail to the author(s)
About Me
View my LinkedIn profile

Add to Google
Subscribe with Bloglines
MCP Microsoft Certified Professional

Falafel Software
ActiveFocus Project Management Solution by Falafel Software
Online or OnSite TestComplete Training
Blogroll
Recent Comments
My Online Tools
Stats
Total Posts: 440
This Year: 45
This Month: 1
This Week: 0
Comments: 1526
Tags
Disclaimer
The posts on this weblog are provided �AS IS� with no warranties, and confer no rights. The opinions expressed herein are my own personal opinions and do not represent my employer�s view in any way.
 Friday, January 11, 2008

HTTP Load Testing with AutomatedQA's TestComplete 6 and Remote Agent

Posted @ 12:54AM by Steve Trefethen

Categories: Performance | Testing | Tools

Tags:  |  | 

Part of my job at Falafel Software is training on AutomatedQA’s TestComplete. Recently, I was teaching a class where the client was interested in HTTP load testing for which Test Complete has great support. Test Complete ships with a tool called Remote Agent that can be installed separately and allows for HTTP load testing using virtual users on multiple machines all controlled by a single instance of TestComplete. In fact, you can load test with up to 250 virtual users allowing a single tester to easily leverage an entire network of computers.

The following outlines the steps I took to demonstrate how easy it is to setup and execute HTTP load tests.

Setting up an HTTP Load Test ProjectTestComplete Project Manager

The first step is to create a new HTTP Load Test project.
  1. Select File | New Project
  2. Click the HTTP Load Testing Template
  3. Click Ok
  4. The Project Wizard appears and you can simply click Finish which will give you a project that looks like the image to the right.

As you can see the project has several nodes by default.

  • Stations: represents the machines that will be conducting the tests. Master is my current machine, the one I'm running TestComplete on.
  • Tasks: is the HTTP traffic you’re interested in testing.
  • Tests: allows you to assign tasks to various stations for test execution.
  • Scripts: is for manually writing test automation.

Setting up Remote Agent

For this example, I’m using a virtual machine running Windows XP. I’ve installed TestComplete 6.0 and I’m running Remote Agent which runs in a console window on the desktop. If you’re going to try and duplicate my setup be sure that your VPC network adapter is not set for Shared networking (NAT).

image The next step is to setup the machines you’ll use for load testing under the Stations node of your project.

  1. Right click the Stations node and select Add New... (you should see the dialog to the right)
  2. Enter the IP address of your Remote Agent machine, in my case that’s 192.168.1.104.
  3. Click Ok

This will give you a new node under Stations, "VPC" which is your VM, or the machine running Remote Agent.

Record an HTTP Task

The next step is to create the HTTP traffic you want to load test your server and the easiest way is using TestComplete’s recording facilities. For HTTP load testing this requires setting TestComplete to be a proxy for your web browser.
  1. Configure Internet explorer to use TestComplete as a proxy allowing it to monitor and record HTTP traffic. For IE you can do that from the Connections tab of the Internet Options dialog setting the proxy to localhost on port 9999. For Firefox open the Tools|Options dialog and look on the Advanced page, under the Network tab.
  2. Click the record button on the TestComplete toolbar and once recording begins click the "Record and HTTP Task" button (circled in red below):
    image
  3. This will bring up the Select Load Testing dialog where you can decide to record a new HTTP traffic or append it to an existing task.
  4. Once you click OK you can begin recording HTTP traffic by simply using your browser. TestComplete will record the HTTP traffic and add it to the task you specified.

Once recorded you can modify the web requests to fit your needs. Here is an example of what the recorded HTTP traffic looks like:

image

Running HTTP Load Testing

The next step is to configure individual Stations to perform the Tasks you’ve created. To execute your tests, TestComplete sends the recorded HTTP traffic to the web server bypassing the browser which means clients hosting Remote Agent will have no UI appear on screen other than the agent itself.

image

As you can see I’ve configured one virtual user to sent HTTP traffic starting half a second after the Master with a User-Agent of IE6.

Running HTTP Load Testing

The final step is to run your HTTP load tests and examine the output. To do that you can simply right click your test under the Tests node in the Project Explorer. The results at first will appear overwhelming but undoubtedly provide you with a detailed report of how you server performed.

image

Wrap up

As you can see TestComplete makes it quick and easy to create and run HTTP Load Tests so if this is something you’re looking for you might want to give it a try. Fwiw, Falafel Software offers TestComplete at a 10% discount in case you’re interested as well as online and onsite training. Feel free to contact me for more information.

[UPDATED: Jan, 17 2008] Fix a few typos, and clarify adding a new Stations node.
 Sunday, August 05, 2007

Bandwidth speed consistently faster in IE than Firefox is a USB drive to blame?

Posted @ 10:19PM by Steve Trefethen

Categories: Hardware | Home | Performance

Tags:  |  | 

linksys wrt150n
I recently purchased a LinkSys Wireless-N router ($99 on sale at Radio Shack) so I can roam around the house with my laptop. As a result I decided to do some bandwidth testing using broadbandreports.com's speed test as there is a server in Palo Alto, ~26 miles from my home, and it seemed like the results I was getting from Firefox were a bit slow. I started up IE to do some comparisons and found a not insignificant difference. As you can see on the graphic ComCast is my ISP which is easily three to four times as fast as what I was getting with my old DSL connection.

Results Using Internet Explorer 7.0.6000.16473

Results Using Mozilla Firefox 2.0.0.6

These results have been fairly consistent across a number of tests. I'm wondering if there is perhaps a performance penalty for running Firefox from a USB drive? All of these tests were conducted on my laptop.

I'll have test again at the office (also wireless) tomorrow and see how things compare. Of course, I'll probably install Firefox for a true apples-to-apples comparison.

 Tuesday, July 17, 2007

Vista's Windows Experience Index and the Apple MacBook Pro

Posted @ 11:32PM by Steve Trefethen

Categories: Performance | Vista

Tags:  | 

I thought readers might find this interesting so here is a screen capture of the Windows Experience Index for my MacBook Pro (my work machine). Btw, the more I use this machine the more I like it, Apple has done a very nice job getting the drivers working for all of this machines features. I have yet to run into any sort of hardware related problem, everything just works. If you ask me, these numbers look pretty good. I'll have to ping Mark Edington and see if I can get him to get me a screen capture of my old "mainframe box" at CodeGear for comparison (apple to oranges, er Dell's but still).Windows Vista Experience Index MacBook Pro
 Tuesday, February 27, 2007

Community contributions improve Delphi 2007 RTL performance

Posted @ 5:52AM by Steve Trefethen

Categories: Delphi | Performance

Tags:  | 

One of the areas I've worked on in the Delphi 2007 release is the inclusion of Delphi Community RTL contributions. Now, including community contributions is not new to Delphi. In fact, there is a project called FastCode which is dedicated to providing highly optimized, high quality replacements for existing RTL routines. With Delphi 2007 we've replaced the following RTL routines with code from our user community including the FastCode project under an MPL 1.1 license which is consistent with previous community contributions:

Contributor(s) Routine
FastCode and Pierre le Riche SysUtils.CompareStr
FastCode and Pierre le Riche SysUtils.StrLen
FastCode and John O'Harrow
SysUtils.LowerCase
FastCode and John O'Harrow
SysUtils.UpperCase
Pierre le Riche
System._LStrCmp

Since this release is interface compatible with BDS 2006 we had somewhat less flexibility to take RTL replacements but the door will swing back open with our next release and I'll continue integration of these great contributions.
 Saturday, February 24, 2007

Delphi 2007 desktop switching performance improvements

Posted @ 10:46AM by Steve Trefethen

Categories: Delphi | Performance

Tags:  | 

The Delphi/BDS IDE supports saving of desktops, or window layouts, via a dropdown list available from the main toolbar. These desktops allow you to control which windows are visible and arrange their size, position and docked location. For example, here is my design-time layout:

Delphi IDE design layout

As you can see I use the "undocked IDE" meaning my code editor is free floating, not docked into the top main toolbar window. I actually like the docked IDE but I prefer to do everyday work using the undocked layout to make sure it's always getting tested.

Now About the Perf...

The other day Mark Edington our performance cop recruited me to help with tuning the performance of switching desktop which I mentioned was keeping me busy on Tuesday. This morning I checked my email and Mark had done some timing of my new code and found "The general average seems to be that things are approximately 30% faster". Cool! Couple this with the flicker work I've been doing and I think the Delphi 2007 experience is going to be much nicer.
 Thursday, February 08, 2007

Delphi IDE and RTL/VCL performance improvements

Posted @ 11:12PM by Steve Trefethen

Categories: Delphi | Performance | RTL | VCL

Tags:  |  |  | 

Lately, there's been a lot of effort put into improving the Delphi IDE's UX and I wanted to comment briefly that in the past week alone we've made some great progress. For example, there have been recent improvements in:

  • IDE flicker (now nearly flicker free)
  • Desktop switching performance
  • GDI resource management
  • Moving controls on the form designer via the keyboard
  • Form designer grid repaint issues
  • Compiler time stamp access (thanks to Andreas Hausladen)
  • VCL control painting (thanks to Pierre le Riche)
  • VCL control data handling performance
  • FastCode routines added to the RTL (btw, a huge thanks to the whole FastCode crew who contribute to this great project benefiting us all)
One of the best things about a number of these fixes is that they're in the RTL/VCL so your applications will benefit as well. Btw, several of them came from QC so it definitely pays off to get your bugs logged. Anyway, we're cranking away and there's more to come but I'm pleased with the progress and thought it was worth mentioning.
 Sunday, February 04, 2007

Using the WS_EX_COMPOSITE window style to eliminate flicker on Windows XP

Posted @ 10:38PM by Steve Trefethen

Categories: Delphi | Performance | Programming | VCL

Tags:  |  |  | 

As I continue to work through the various flicker issues on individual controls in the VCL I thought it worth mentioning a bit more "global" way to tackle the "flicker" problem. If you're running Windows XP there is a window style called WS_EX_COMPOSITED which you'll find documented on MSDN under CreateWindowEx here. This style bit instructs the OS to apply double buffered painting for you. Here is how you would add this to a Delphi VCL form:
1 type 2 TMyForm = class(TForm) 3 protected 4 procedure CreateParams(var Params: TCreateParams); override; 5 end; 6 7 ... 8 9 procedure TMyForm.CreateParams(var Params: TCreateParams); 10 begin 11 inherited; 12 // This only works on Windows XP and above 13 if CheckWin32Version(5, 1) then 14 Params.ExStyle := Params.ExStyle or WS_EX_COMPOSITED; 15 end; 16

You should be able to use this on any 32-bit version of Delphi as long as your app is running on Windows XP.

The downside is that depending on your application this change could have rather serious performance implications during a window resize particularly with aligned controls. The upside of course is that it's flicker free! :-)

I've read that there may be some issues related to using this window style with GDI+ so be sure to look for that if it affects you.

There is no comment on the MSDN as to how this works under Windows Vista but I'd guess there may be some differences.

 Friday, April 21, 2006

Slow compile times with readonly files in Delphi 2006 fixed in Update 2

Posted @ 9:00AM by Steve Trefethen

Categories: Delphi | Performance

Tags:  | 

The problem I mentioned here has been addressed with the release of Update 2. If you're in a situation where you're using readonly files that are part of your Delphi project this could significantly speed up your compile times.

Thanks to Mark Edington for the leg work on this one!

 Monday, February 27, 2006

BDS 2006, slow compile times and the cost of exceptions

Posted @ 9:25AM by Steve Trefethen

Categories: Delphi | Performance | Quality

Tags:  |  | 

While browsing the newsgroups I ran into a thread where a user had commented that BDS 2006 compiled his application incredibly slowly. A while ago I'd had a discussion with Mark Edington our resident performance maverick about this issue and it turns out that it's a good reminder of the cost of raising an exception or in this case lots of them.

Since Mark debugged this issue he kindly provided me with a short write up of the specifc problem, it's solution and a workaround.

Mark writes:
In BDS 2006 in Delphi a performance issue arises when compiling projects in the IDE when the source files in the project are marked as read-only on disk. The more files in the project the more significant the delay. The problem only affects projects that have read-only source files. The root of the problem, is a callback procedure which is called by the compiler when compiling in the IDE. The callback is designed to open the file and return an IStream interface:
function OpenOSFile(const Name: string): IStream;
begin
{ try opening existing file so that read/write operations are
possible }
Result := nil;
if FileExists(Name) then
begin
try
Result := TOSFileStream.Create(Name, fmOpenReadWrite or
fmShareDenyNone);
except
on EFOpenError do
Result := TOSFileStream.Create(Name, fmOpenRead or
fmShareDenyWrite);
end;
end;
end;

Notice the try/except block, since there is no parameter to the function that specifies what access rights to use when opening the file, the procedure tries to be accommodating and provide read/write access first and if that fails it resorts to to opening the file as read-only. Unfortunately, as written, it introduces a serious performance penalty for anything other than small projects. Interestingly enough, the performance issue isn't from the fact that it takes 2 attempts to get a read-only file opened, it is actually caused by the exception that is raised when the first (read/write) TOSFileStream.Create call fails.

In the test case that was submitted for diagnosing this problem the RaiseException procedure (which is a Windows API call), was accounting for nearly 40% of the total compile time. Under a profiler it took nearly 50 seconds to execute that procedure about 1100 times. That works out to around 45 milliseconds per call which may not sound like that much, but when executed 1100+ times it adds up.

The solution was very straightforward: Rewrite the routine and remove the exception based fallback algorithm. The new version looks like this:

function OpenOSFile(const Name: string): IStream;
var
Code: Integer;
begin
Result := nil;
Code := GetFileAttributes(PChar(Name));
if (Code <> -1) and (FILE_ATTRIBUTE_DIRECTORY and Code = 0) then
begin
if (FILE_ATTRIBUTE_READONLY and Code = 0) then
Result := TOSFileStream.Create(Name, fmOpenReadWrite or
fmShareDenyNone)
else
Result := TOSFileStream.Create(Name, fmOpenRead or
fmShareDenyWrite);
end;
end;

This version first checks to make sure the file is not marked as read-only before attempting to open it as read/write. The FileExists call that was replaced in the original version of the procedure is actually implemented using GetFileAttributes anyway, so there was no extra overhead introduced to make the check.

The moral of the story is avoid writing code that raises exceptions as part of the normal course of program execution. Exceptions are horrifically expensive to raise and handle and should be reserved for the truly "exceptional" events. If you have a medium to large Delphi project that has files marked as read-only the workaround for this issue is to mark them as read/write until this fix is made publicly available.

 Thursday, December 08, 2005

Looking for feedback on Delphi 2006 performance

Posted @ 9:38AM by Steve Trefethen

Categories: Performance

Tags:

Since I've had trouble convincing Mark Edington to start a blog I figured I'd get his post to the Delphi.ide.general newsgroup out here for him.

Date: Thu, 08 Dec 2005 15:19:47 -0800
From: "Mark Edington (Borland)"
Newsgroups: borland.public.delphi.ide.general
Subject: Delphi 2006 Performance Feedback Wanted

We have made quite a few improvements in the performance of the IDE for Delphi 2006. Many common operations are noticeably faster than in Delphi 2005 but there are probably still areas where folks would like to see additional improvements (myself included).

We have been talking quite a bit about improving quality in Delphi and while I would like to think we have made great strides in Delphi 2006 we do recognize that for us achieving higher quality levels will be a journey and not a destination.

So, in that spirit, I am soliciting feedback on *specific* operations (such as opening a file or switching from code editor to form designer) that people perceive as being sluggish or noticeably slower than in Delphi 7 and previous releases.

If you have Delphi 2006 and have had some time to work with it I would love to hear your "top 3" or "top 5" or even "top 10" things you want to see made faster in the IDE.

If you have comments regarding the IDE startup time you might want to hang on to those for a little while. We have improved it quite a bit but there are definitely more gains to be made in that area and we will be looking into that. I'll probably solicit feedback on that specific topic at some point in the near future. For the purposes of this thread I would like to get feedback on everything *besides* startup times.

I would prefer to keep the discussion here in the forum, but if for some reason you have some valuable feedback and do not want to post it here you are welcome to email me. You will need to substitute a certain company name for the placeholder in my reply-to address (all you smart people here would have figured that out anyway though, right? :0)

Mark Edington
Delphi Quality Architect
Borland Software Corporation