Let us review our strategy, especially referencing our usage of White. I find it interesting that Test Complete makes it so hard to “do the right thing” as propounded by both Scripting GUI Tests in Ruby and by White. And just what is that?

The responsibility of the test script is to

• perform a series of actions and verifications.
• allow someone to read the script and understand what the hell is going on.

The first is a gimme. The second is interesting. When I have my developer hat on, I believe code is read more often then it is written. Wearing my manual tester hat, I believe the scripts should be clear and show intent. Tricky scripts are trouble. They are also a pain to maintain. So, if automated testing the meeting of these two worlds, the clarity and intent of the scripts must be sacrosanct. The libraries the scripts use, on the other hand, just need to be treated like code and kept maintainable.

What do I want my test scripts to look like? There are two options, DSL, or clearly abstracted test code.

First, lets look at DSLs, Baby! WAHOOO! Tests that double as manual tests!

open sekrit with New Document
insert image "picture.png" into sheet
first image item should match stored image "stored_0001.png"

I could do that. I could use some functional programming language. I could also use something like cucumber and code behind my feature files. But what’s the point? All I need is clarity, so the auditors / business people understand. I LIKE dsls, but this buys us nothing right now.

Option two uses Objects to clarify the application.

sek = Sekrit.open( :New_Document )
doc = sek.document

doc.add_image("picure.png")
Assert.equal doc.images[0].as_bitmap, Stored.images["0001.png"]

This is at the proper level of abstraction for a slightly motivated business user to understand. What they don’t care about at this level is the control hierarchy and control manipulation. Unless the requirements say, “via a popup menu” or some such.

If you’re going to be technical, this is a facade. It’s a wrapper to the complex underbelly of the white framework. It allows for a higher level use of the collected objects and re-use as we don’t have to repeat the code for complex actions and interactions. It also decouples our tests from changes in our application. I’ve had to chase that down in our current automation set, and I hate it.

How do we build up this abstraction? To be compatible with White’s recommended practices, I’m going to call this facade a “Screen”. We’re going to write a test for an application for which we have no source. Download blu and warm up your tweetin’ fingers. We’re going to log in!

# our screens
class Blu
  def initialize(application, window)
    @app = application
    @win = window
  end
  def login_screen
    return LoginScreen.new(@app, @win)
  end
end
class LoginScreen
  def initialize(application, window)
    @app = application
    @win = window
  end
  def login_user( username, password, remember = false)
    @win.get_textbox("UsernameTextBox").enter username
    @win.get_textbox("PasswordTextBox").enter password
    rem = @win.get_checkbox("Rememberage")
    if rem.checked != remember
      rem.enter " "
    end
    @win.get_button("LoginButton").click
  end
end

You noticed that the BluScreen has a LoginScreen? Our application has regions and concepts to which we wish to refer. For example, we may want to represent a ribbon control, a toolbar, different types of documents, or even the clipboard.

How do we use these screens? Let’s flesh out this little story in Dan North “Story” format.

 As I malicious user, so I can't defame Brian's good twitter name,
   Given blu is open
   And no one is logged in
   When I attempt to log in using a bad password
   Then I should see a warning that my login information was incorrect

app = Application.attach("blu")
win = app.get_window("blu")

blu = Blu.new(app, win)
login = blu.login_screen
login.login_user( "Myotherpants", "NoWai!" )

# A test would have assertion here...

I’m holding out on you. The truth of the matter is, I started with the test steps first, then made screens that abstracted those steps. But there’s magic in the screens, they have magic strings passed around like @win.get_checkbox("Rememberage"). Where did that come from? I got it from UISpy. I’ve read about Woodstock, Snoop, and a few other tools, but UISpy is the only tool I used for this post. It is in the Windows SDK. UISpy shows a tree of controls. When you click on one, it hilights it with a red box. It also displays information such as the automation Id, those magic strings we have scattered in our code. Before my two picture tutorial on UISpy, let me note, those strings are what’s likely to break your script in the future as your application grows.

UI Spy looking at a textbox

Blu when UISpy is indicating a textfiled

what else is there?

These aren’t tests, they are automation. I’ll eventually wrap the automation with RSpec and get some real tests. I’ll get to Cucumber. I’ll publish examples. I promise.

But next, I want to address three types of custom controls:
User controls, Inhereited and Templated Controls, and Framework Elements. Another problem is finding un-named controls. I’ll see you next week.

One last bit, here’s what the white ruby library is starting to look like.

white_loc = (File.dirname(__FILE__) + "\\White_Bin\\")
$LOAD_PATH.push(white_loc)
require "White.Core.dll"

Application = Core::Application
Button = Core::UIItems::Button
CheckBox = Core::UIItems::CheckBox
TextBox = Core::UIItems::TextBox
Window = Core::UIItems::WindowItems::Window
SearchCriteria = Core::UIItems::Finders::SearchCriteria

class Window
  def get_button(*args)
    self.method(:get).of(Button).call(*args)
  end
  def get_button_labeled(label)
    self.get_button(SearchCriteria.by_text(label))
  end
  def get_checkbox(*args)
    self.method(:get).of(CheckBox).call(*args)
  end
  def get_textbox(*args)
    self.method(:get).of(TextBox).call(*args)
  end
end

My install of white is not in the GAC, so I’ll be requireing DLLs from a directory. DLLs are looked for in ruby’s $LOAD_PATH, just like ruby libraries. It’s an array. You’ll notice that White suggests also including an NUnit dll, but I’ve not needed that with RSpec. And, I want to shorten the namespace stuff. So let’s start with white.rb and we’ll add to it later.

white_loc = "C:\\Projects\\Libs\\White_Bin_0.18"
$LOAD_PATH.push(white_loc)
require "White.Core.dll"
Application = Core::Application

So, next we just start using the application. I want to launch my application, conviniently called “sekrit.” I need to find the window titled “Sekrit (version 0.5)”. Then I need to call the C# method win.Get<Button>("Open") but that uses generics that ruby doesn’t have. Using what I learned from a Stack Overflow post how to invoke an overloaded generic method in ironruby, we can work around that in IronRuby 0.5.

require 'white'
app = Application.launch("c:\\projects\\sekrit\\build\\sekrit.exe")
win = app.get_window("Sekrit (version 0.5)")
win.focus
win.method(:get).of(Core::UIItems::Button).call("Open")

Oh, lawdy! Did you see that last line? It ain’t pretty. This is something we’ll be doing a lot of, so let’s wrap that up. I need to add, essentially, a non-generic extention method to wrap the generic call. But, do it with Ruby to extend a .NET class. Back to white.rb for some open class magic.

class Core::UIItems::WindowItems::Window
  def get_button(*args)
    self.method(:get).of(Core::UIItems::Button).call(*args)
  end
end

Now that cleans up our earlier program.

require 'white'
app = Application.launch("c:\\projects\\sekrit\\build\\sekrit.exe")
win = app.get_window("Sekrit (version 0.5)")
win.focus
win.get_button("Open")

Okay, one more problem. I talked about RSpec, but that’s because I hacked my install. There’s a known bug in IronRuby 0.5’s implementation of File.expand_path(file_name) that uses the .NET System::IO::Path methods that throws execptions when you try to expand a path with “:8″ on the end to help track line numbers. I don’t know yet if it’s been fixed. If not, that may be another post.

So, what’s left? RSpec doesn’t run out of the box. I need to wrap some stuff up so I can say app.open_file("some_file.ext") and other higher level “user action” concepts (those shouldn’t be in white.rb, but sekrit.rb). I need to handle finding controls quickly and easily. I’ll take a look at that this weekend and see what I can find. Especially, since I’ll be hanging with a WPF MVP both this evening at IndyALT.NET and on Saturday.

I’ve been using TestComplete for this.  T.C. is good stuff, but there are some things about it I don’t like.  It’s meant for recording and playback of tests.  But, to automate in a regulated situation, I don’t want to see javascript (T.C.’s CSharp script is a horrible trap, don’t go there) like the following

var p1;
var w1;
p1 = Sys.Process('MyProcess');
w1 = p1.Window('HWind: MainWindow').MainWindow.Base.WPFObject('Grid', '', 1).ScrollView;
w1.x = 0;
w1.y = 0;
w1.WPFObject('RowItem', '', 2).StackPannel.WPFObject('Button', 'Cancel', 1).Click(3,15);

What the hell is my intent here?  Also, I’m crawling the visual tree, which may be brittle.  The renaming of internal variables, like “StackPannel” to something more revealing like “VisibleFlarns,” I’d have to go edit this test.  And a buch of other tests that traverse that point in the tree.  We kind of saw this coming and have rewritten our tests to a degree.

var process = Sys.Process('MyProcess');
VisibleFlarn(process, 2).WPFObject('Button', 'Cancel', 1).Click();

This is slightly more clear.  The main benefit is that we’re hiding the traversal of the tree to the visible flarns.  That means we only have one place to hunt down our changes to the tree.  But it doesn’t take full advantage of our environment, JavaScript.  Foreign objects, like the process, windows, and buttons can’t be extended with helper methods.  So, I’ve started the next round of testing with a bunch of decorators and strict reuse.

var process = new MyProcess();
process.GetVisibleFlarn(2).Cancel();

That’s starting to get to good.  The last piece is that regulatory oversight loves positive passing of tests, not just error logs.  T.C. doesn’t have that.  T.C. doesn’t even really have asserts.  You can only error, or log.  In fact, I’ve yet to find a test framework that does that out of the box.   Even those that assert, don’t log positive asserts.  Why is this important?  I want to be able to say Req-123 was tested 7 times, and passed each one.  That’s what the oversight wants.  They fear, “Well, it’s didn’t show up in the error logs, so I’m assuming it worked.”  So, I wrote some asserts that take explicit requirements and “Why” I’m asking about that.

Now, on to prong two, of my assault.

I recently picked up “Scripted GUI Testing with Ruby” by Ian Dees as I visited my local book store. I like the book. It covers using Win32API to grab window handles and throw around events. Oh, it’s also got some crap about Jruby and an apple-talk bridge if you want to go that route. Basically, it covers testing on all the platforms that aren’t X11 or .NET. That doesn’t detract from the book’s handling of GUI testing strategy, but I want to fill in that gap.

The CLR has a pretty good testing story, for WPF at least, in the White testing framework from ThoughtWorks.  It uses Automation Ids to identitfy and find all of your controls.  So, the pieces I’m looking at are, “What are Automation Ids,” and “What are their best practices?”  Couple that with a running example of using IronRuby to get my hands on the application and bang it around. The big deal is our current project doesn’t use Automation Ids (accessibiltiy isn’t a huge driving need), so I need to show some benefit and low risk.

What am I going to have to do, for this experiement?  I’m, currently, only interested in rSpec and don’t need cucumber.  I’ll start by using White directly, but I’m guessing I’ll be wrapping it some. I’m also interested in alternatives to AutomationIds.  But I’ll need to wrap the standard asserts with something that positively checks asserts against requirement numbers.

Will the regulators go for this?  I don’t know.  When working in a regulated environment, you can be innovative with how you handle testing as long as the auditors are comfortable with your ability to be repeatable and cover what they find important.

Should be fun.