Monday 15 August 2011

Test result trend not appearing in Jenkins for NUnit plugin?

This was driving me crazy for a week. Some jobs were showing the trends graph, others were not.

Eventually, i solved the problem; your job's name must begin with a lower case if you there are any other upper case characters in the first word. Confused? Here's an example;

OvernightBuild_Test (graph will not appear)


overnightBuild_Test (graph will appear)


Overnight_Build_Test (graph will appear)

If you rename, it will appear (including all previous builds)


Monday 23 May 2011

Lessons from a WATIR to WATIR-Webdriver port - #5 Finally real Grid support!

My port from Watir to Watir-Webdriver has taken a bit of a back seat recently, and instead I have been working on a new project using Selenium-Webdriver (AKA Selenium 2).  I think a prefer Selenium mainly because I am writing in Java and I get compilation errors and better IDE support! Also, who can forget Selenium Grid? Watir has been crying out for a grid, and yes, I know there is "Watirgrid" but let's be honest, it's doesn't hold a candle to Selenium Grid.  To use "watirgrid" you have to re-write tests to take advantage of it.

Although Selenium-Webdriver does not work with Selenium Grid, I have been able to use a pre-beta version of Selenium Grid 2, and it's great. Easy to use, and with minimal effort I achieved things which previously had taken a lot of effort with Watir.

However, back to promoting Watir! With my Selenium Grid 2 up and running, I thought "mmm, if Watir-Webdriver uses the same Webdriver code as Selenium-Webdriver, can I use the grid with Watir?"  The answer is yes!

This great overiew of Watir-Webdriver (http://watirmelon.com/2010/12/14/watir-webdriver-a-detailed-introduction/) mentions running via Remote Webdriver Server.  Although the blog entry refers to the Remote Webdriver Server for headerless testing is can still be used for browsers.

Assuming you had started a locally hosted Remote Webdriver Server on port 4444, a test for firefiox would look something like this:


require "rubygems"
require "watir-webdriver"
require "watir-webdriver/extensions/wait"
b = Watir::Browser.new(:remote, :url => "
http://127.0.0.1:4444/wd/hub", :desired_capabilities => :firefox)
b.goto "
www.google.com"www.google.com"
b.text_field(:name => "q").set "Watir-WebDriver"
b.button(:name => "btnG").click
b.div(:id => "resultStats").wait_until_present
puts "Displaying page: '#{b.title}' with results: '#{b.div(:id => "resultStats").text}'"
b.close

Here comes the good part! If I change the URL to a Selenium Grid 2 hub machine, then my test actually runs on the grid. I could have 10 tests running in paralell on my machine, but the browser under test would actually be on different machines for each test.
Looking back on this entry, I really have explained this all quite poorly. In my defence, it is getting late.

Instead of listening to me, try the following;

1. Goto http://selenium-grid.seleniumhq.org/ , understand what a grid gives you and get jealous that it is only for Selenium!

2. Think about how great your Watir tests would be if you had a grid.

3. Get the Selenium 2 Grid source code, build it and set up a simple grid

4. Run a Watir-Webdriver test against your grid

5. Get very excited

Obviously you still need a parallel running solution to help you leverage the grid to the full potential of distributed test runs . As my new project is in Java, I'm using TestNG which comes with "free" multi-threading!

Wednesday 9 February 2011

Lessons from a WATIR to WATIR-Webdriver port - #4 When assert_true is no longer assert_true

I wanted to record problems found (and perhaps solved?) while attempting to port a test automation framework from WATIR to WATIR - Webdriver, and a simple blog entry seemed to be the easiest way.


This problem may be just faced by myself and the caused by the versions of TestUnit, Watir & Webdriver I am using, but just in case other people are having the same issue, here it is for google to find!


For the same test, Assert_True was passing when using WATIR, but failing when using Watir-Webdriver.


After tearing my hair out for a while I realised that when using WATIR, assert_true comes from more-assertions.rb in a gem called s4t-utils, where as when I ran the test as WATIR-Webdriver, the assert_true from the testunit gem is used.


The assert_true in TestUnit gem is far stricter in that the value must be boolean true to pass, where as the other assert_true takes advantage of the fact that in Ruby, anything which isn't nil is true.


Personally, I like the version which insists the value is boolean true, so have decided to change the tests to follow this, although obviously I could have overridden it with the less strict version.


As I said, this problem will not affect everyone, perhaps it's only me, but I wanted it recorded!



Lessons from a WATIR to WATIR-Webdriver port - #3 Ruby meets strong typing

I wanted to record problems found (and perhaps solved?) while attempting to port a test automation framework from WATIR to WATIR - Webdriver, and a simple blog entry seemed to be the easiest way.


WATIR is far less strict than Webdriver about the classes you use to access objects in the html, presumably this because Webdriver is written in Java(???) and that is derives its tag_name from teh class itself.


If I have a visible object which looks like a button but in the html is actually a link, then WATIR is far more forgiving.


For example;


 <a href="http://www.google.com" id="goog">Goto Google</a>


Using id, WATIR allows me several ways to access the object, such as;


@browser.button(:id, "goog")
@browser.link(:id, "goog")


However, with Webdriver, I have to use the class define in the html.  Therefore only the following is valid.


@browser.link(:id, "goog")  NB. I believe Link is an alias of "a", therefore "a" is valid too


If you were to use "button", you would get the following error;


Watir::Exception::UnknownObjectException: unable to locate element, using {:tag_name=>"button", :id=>"goog"}


This is obviously not a big problem, but as I said, exposes mistakes which WATIR allowed and requires changes if those mistakes were made



Tuesday 8 February 2011

Lessons from a WATIR to WATIR-Webdriver port - #2 Add_checker can still be my friend

I wanted to record problems found (and perhaps solved?) while attempting to port a test automation framework from WATIR to WATIR - Webdriver, and a simple blog entry seemed to be the easiest way.

A colleague advised me that tests ported from WATIR ran extremely slow on IE, and that this was caused by our use of AddCheckers (http://rubydoc.info/gems/watir-webdriver/0.1.7/Watir/Browser#add_checker-instance_method)

As a result, it was assumed that;

add_checker + webdriver = BAD


However, this is not true.  The reason the add_checkers appeared to be slow on IE, was because we used the Text method on the Browser class to scan for certain text on the page.  This is extremely slow in IE for webdriver.


Therefore, add_checkers can be used in webdriver as long as you do not use browser.text

Monday 7 February 2011

Lessons from a WATIR to WATIR-Webdriver port - #1 When it comes to Firefox, size matters

I wanted to record problems found (and perhaps solved?) while attempting to port a test automation framework from WATIR to WATIR - Webdriver, and a simple blog entry seemed to be the easiest way.


Slow Firefox Startup


I was finding that it was taking upwards of 30 seconds to launch Firefox at the start of a test.  This compared badly with WATIR, where the same test was a few seconds.

The problem;

As part of Selenium::WebDriver::Firefox::Launcher, the FF profile is copied to a temporary folder.  This uses FileUtils.cp_r and appears to be slow.

The solution:

Make sure the firefox profile used on your test run machine is as small as possible.  I removed Firebug, EditCookies and a few other add-ons, and this reduced the start up time to a few seconds and comparable to WATIR