Archive for the ‘Automated Tests’ Category.

STPCon session submission. What do you think?

An Automated Test is Real Code!

Session Information

An automated test is real code! We should follow good programming practices even when automating tests. Use Session-Based Test Management to explore you system under test. Model your automation design before you get started. Write test first unit tests to help build your page objects. After your page objects are built, make them portable to use in different test suites such as functional, conformance, and cucumber tests. Organize your continuous integration jobs to tell you pinpointed information about your system under test.

The Selenium Fury gem is ready! Page object factory for Selenium and Ruby.

I have been working on converting our page object factory to open source for a few weeks. Now it is time to launch the  HomeAway sponsored open source project under the Apache 2.0 license.  It is a furiously quick way to implement test automation.  I am planning to add more configuration options in the future.

This project started when I had to test a page with 300+ check boxes and I did not want to enter them by hand.  I used the page object generator to build a page of ruby variables with Selenium locators. Everything was great until some number of the check boxes changed their ids and my tests started failing.  I needed  a quick way to find out how many changed so I could update the locators by hand or regenerate the page.  This is where the validators came in.  I used Ruby’s support of reflection to open a class, navigate to the url of the page and use Selenium to validate the locators and return a list of missing locators on the page.  It worked perfectly for my page of 300+ check boxes.  I had over 40 that changed I quickly regenerated the page.

Install with:

  • gem install selenium_fury

Checkout the home page and examples at https://github.com/scottcsims/SeleniumFury

Thanks to HomeAway for sponsoring this project.

Advanced Test Automation – Generate your own css locators using nokogiri

Some of the pages that I test have hundreds of check boxes. I needed a way to generate the locators to use in selenium and create classes to use those locators.
This is an example of a class that contains selenium css locators for navigation links

class Navigation
  def initialize * browser
    @location =  "css=a:contains(\"Location\")"
    @amenities =  "css=a:contains(\"Amenities\")"
    @photos =  "css=a:contains(\"Photos\")"
    @contact =  "css=a:contains(\"Contact\")"  
    @further_details =  "css=a:contains(\"Further Details\")"
  end
  attr_accessor :location, :amenities, :photos, :contact, :further_details
end

To use this class I would new it up and use standard selenium commands. This is how I would navigate to the amenities page

navigation = Navigation.new
browser.click amenities

The spec below demonstrates how to get data you need using nokogiri. I want to click an anmenity check box with the label beach whose input name is “amenity_1_1_6”. I could click this checkbox with this command browser.click “amenity_1_1_6”, but I want to have a way to store all of the amenities on the page with the variable named by their label.

require "spec"
require "nokogiri"
describe "Find elements to use as locators in a web test" do

  it "should find the value of name for the input element and the text in span" do
    sample_html = ''
    attribute_name=""
    attribute_value=""
    doc = Nokogiri::HTML(sample_html)
    doc.css("label").each do |check_box|
      attribute_name =  check_box.text
      attribute_value = check_box.children.css("input")[0]["name"]
      attribute_name = attribute_name.strip
      attribute_name = attribute_name.to_s.downcase

    end
    attribute_name.should=="beach"
    attribute_value.should=="amenity_1_1_6"
  end
end

To take this further I can print the contents of my class file to generate my locators from this spec.

puts "@#{attribute_name} = \"#{attribute_value}\""

Don’t miss this important note if your planning to run tests against Internet Explorer using Selenium Grid

I tried to run tests in parallel against IE using the selenium grid for a few days before I found this important message on the selenium grid FAQ.
Picture 9

If you try to run multiple selenium remote controls for IE on one machine it will work for a while.   Your test will execute without any problems, but then you start getting sporadic error messages from IE.  I assumed that the xpath expressions I was using for the tests in Firefox were throwing errors in IE.  I tried different configurations for security settings in IE for a while and then I finally found out that it was a memory sharing issue that prevents the selenium remote from launching more than one IE instance.  For now I am just requesting more VMs to run a single IE remote on each.  I use ruby with deep test to run test in parallel on the grid.  I configure deep test to run 6 tests at a time, so I would only need 6 VMs for IE to run.

It puzzling to me that Watin tests run in parallel against IE without any problems on the same machine, but selenium tests cannot.  For now I will add more VMs and wait for Selenium Grid 1.2  to fix the IE problem.

Essentials of getting started with Selenium Grid and Ruby.

I have been running test/unit scripts on the grid for a while now and I was trying to remember where I got my test/unit syntax from.  I was looking to create a live template to create new test scripts in Rubymine. You actually have to make sure that you create the Selenium Driver object in a specific way or your scripts will not run on the Selenium Grid or they will not run in parallel correctly.  Your test class must also always have a setup and tear down, even if you call a method out side of the class to setup the driver.

There are 2 sources of information that you must understand and monitor for updates.  First is the Selenium Grid home page.  Recently when Firefox 3.5 became available, I needed to update the version of the grid software I was using.  The other site is the site for the selenium client gem. You can find documentation here on how to structure your spec and test/unit scripts so that they run on the grid.  The gem and the grid  are both maintained by the same person.  I revisit the sites often to pick up tips on how to structure rake files and get tips for running scripts in parallel. In working with this software, I have become very familiar with the files in the rspec, deeptest, and selenium-client gems.

Use custom Firefox profiles with selenium grid.

I started using custom Firefox profiles so that I could get around the self signed cert error in firefox.  I found a way to easily maintain one configured profile that is used across all Selenium remote control environments running on OS X and multiple windows versions.

  1. Download the selenium grid software.  I am using selenium-grid-1.0.4
  2. Create a new directory called SeleniumFireFoxProfile  in your selenium-grid-1.0.4 directory.
  3. Start your remote control using the custom profile. rake all:start SELENIUM_ARGS=”-firefoxProfileTemplate SeleniumFireFoxProfile”
  4. Create a test script to open a selenium client driver and navigate to the site that is getting the ssl cert error. Use a debugger to pause the execution of your script at the browser.open command
  5. Execute your script to the break point and accept the cert in your Firefox browser.
  6. Look for the creation of the cert_override.txt file in your user directory.   With Firefox 3.0 I found it on my mac in /private/var/folders.  I cd to the contained directory and run a find command to locate it sudo find . -name cert_override.txt. With Firefox 3.5 the profile is stored in the user directory ~/Library/Application Support/Firefox/Profiles.
  7. You should see an entry in your cert_override.txt file after you accept the certificate. Now copy the cert_override.txt from your user directory to your SeleniumFireFoxProfile directory in your selenium grid directory.
  8. Restart your remote control now using the profile with the cert_override.txt file in it.  Rerun your test and break on the open command now you should see your page render instead of the security warning.

Just a warning about the cert_override.txt file.  It is very white space sensitive, it is better to copy the whole file into your custom profile directory rather then copying a new line into an existing file.

I check in the override text file along with my selenium software.  This way I can add a cert on my local machine and then run a SVN update on the distributed grid hosts to get the new cert information. This works for both windows and OS X.

Using the selenium IDE and Rubymine

This post explains how to debug a Selenium test script in Rubymine like you would in the selenium IDE. It is common to have a missing element failure in a recorded selenium script.  A quick way to debug this in the Selnium IDE is to put a break point in your script and use the execute command button to find the right code for your failing command.
Add breakpoints start and stop test execution

Add breakpoints start and stop test execution

In the case of a failing script you can see below that the failing line is highlighted in red and you can see the log message explaining the error.
Failing script

Failing script

So when you export your selenium IDE script to your favorite language, how do you trouble shoot failures like this easily in your development environment?
Test Script

Test Script

I generated an error in the script by changing the name of the google button from “btnG” to “btn” .  Trouble shoot this error by using the Rubymine debugger and evaluate expression tool.  I put a break point on the line of code that is failing.  Then I can select the text and open the evaluate expression tool to see if I can locate the button.
picture-15

When you get the expression correct, you will see a message, result = nil.  This is a great tool for testing xpath expressions.

Successful Evaluation

Successful Evaluation

If you noticed that the code I used in this example is different than what is exported from the Selenium IDE, it is because I use the selenium-client gem and execute my tests using the selenium grid.