We did share how to use Functions as variables and first class functions in javascript to achive similar goals in Test Complete.

Custom controls are a big part of the WPF draw, and something I hope to address in a couple of posts.

Ranorex does intrigue me and I’ll have to grab a trial license once I have a better idea of what makes a testing tool the bees knees. It is about 1/2 the cost of TestComplete and if it has all the features I need, then great!

Regarding development variable names in code – there’s no way around that I think. If it were a manual test, you would be tied to the the text that shows up on the button instead of the name of the button. In my experience, there’s little difference. However, where automation wins big-time over manual procedures is in code reuse. In a manual procedure, you wouldn’t want to send the tester back to some page where you define the process to save a single file over and over again. But, in automation, you can define a procedure to save a file and use it over and over again (and test half a dozen points along the way, to ensure consistency in the application). My hope is, before long, you will have a slew of test helpers that let you do anything in the application you could want. Maintenance will be as simple as debugging one test helper. Writing tests will be as simple as chaining a few helper methods together. Getting there is a lot of work, but well worth it.

The biggest problem with automating system testing that I’ve encountered hasn’t been performance or maintenance, but state (though state can be tied to maintenance). Unit tests are easy and quick because your state is usually one or two objects – a relatively small chunk of memory that can be easily disposed and recreated. In system testing, your state is the entire machine. Resetting the entire machine is often not easy or even possible. What’s more, a good system test shouldn’t count on the machine being in a consistent state. It’s the border cases where the machine is busy getting a Windows update or something equally strange where applications tend to fail miserably. Writing your tests to handle every version of the application successfully executing an operation can be extremely challenging. I’m not sure what a test would look like which handles this gracefully. Perhaps some sort of “optional” flag that can be passed to a method like ConfirmOverwriteDialog so that, just in case the file already exists, your test will go ahead and save over it. There’s definitely some room for high-level design with tests.