You are currently browsing the monthly archive for December 2010.

Recently I was working on a project utilizing SVG/HTML5 and I wanted users to be able to move shapes around within the browser window.

My first reaction was to write a test that did something like the following:

  • Create shape
  • Simulate a mouse down on shape
  • Simulate a mouse move with controlled event clientX and clientY values
  • Test that position of shape had moved by those amounts.

What Is That Really Testing Anyway?

But then it occurred to me that while this would be valid for the System Under Test (SUT), I don’t have any information beyond my own use and environment.  How do I really know a user in their environment will generate those expected inputs I use to make things go green?

See, here’s the rub with the way most Agile testing starts:

  1. A team works with a PO/PM and collectively they decide on a set of functionality.  Depending on the organization, questions might range from “what do you want the application to do?” to “what value is the user wanting to get out of the application?”.
  2. They come to an agreement on a certain set of exhibited functionality characteristics.  These represents the collective best guess at what will work when it is put in front of users.
  3. The team creates a system that exhibits those functionality characteristics.
  4. The work is delivered to the user, who may or may not achieve the success envisioned by the system creators.  They may be hampered by “usability” or “environment” issues.

When I write traditional TDD, I have validated that the system works as I expect it to work, but I have not validated that the user can use the system to bring about the expected result.   One can see how those in the testing community like Jon Bach suggest that:

The Agilista’s main vehicle for testing always seems to be “checking”, not investigation. It’s weird that they are quick to say there are no silver bullets, but imply that checking-via-automation is the only way to kill werewolves.

Put simply, a functionally correct system (as determined by tests) may still turn out to be unusable or not work or otherwise not achieve its intended goals. Read the rest of this entry »