Going to Software Test Professionals Conference next week.

I am looking forward to my first trip to Las Vegas http://www.stpcon.com/. I will have a chance to meet with other selenium grid users and test out my strategies in test automation patterns. I will be attending all of the topics on agile automation

Deep Test 2.0 prerelease is available.

Deep test allows you to run ruby rspec tests in parallel by providing a spec task that you include in your rake file. The 2.0 prerelease version of deep tests offers support for rspec 1.1.12. You can install it buy running: gem install deep_test_pre. Require it in your rake file like this:

gem “deep_test_pre”, “=2.0”
require “deep_test”

gem “deep_test_pre”, “=2.0”
require “deep_test”

There currently is a bug that occurs if you run Test::Unit tests with the deep test spec runner.  The tests will all run twice but are reported once.

Selenium Grid Projects

What I have already done to update the ruby example in the Selenium Grid repo. Code Link

  • Updated rspec from 1.1.8 to 1.1.12. I accomplished this by getting the most recent deep-test code  and adding it to the /lib directory. The new version of the deep test code supports rspec 1.1.12. I then just require it in in the rake file. require File.expand_path(File.dirname(__FILE__) + ‘/lib/deep_test/rake_tasks’)
  • The rake file does not need selenium-client 1.2.7. I removed the dependency by  adding the reporting files it was referencing into the lib/reporting  directory. report_formatter_path = “lib/reporting/selenium_test_report_formatter”
  • Updated selenium-client from 1.2.7 to 1.2.18. With the change to the rake file, I am now able to use the most recent version of selenium-client in my specs.

Other helpful tools/tricks I have added to selenium grid (Maybe useful to other Ruby/Grid users)

  • I start and stop the grid from the same rake file that I execute the tests from. Examples are grid:all_start and spec:run_in_parallel
  • Rake tasks can just run and create a HTML report or run and send test results to Rally
  • The tests and rake file runs on OS X, Cent OS, and Windows. (no parallelizaion on windows)
  • Specify command options to the rake tasks to switch between hostnames I am running against. (This is useful when you have more than one instance like test.yourdomain.com and stage.yourdomain.com)
  • Specify to only run tests for specific sites in the test suit. For example if you wrote tests for www.yourdomain.com and www.yourdomain.es.  You can specify which sites to run against
  • I use gem bundler with all grid projects to help control gem installation and versions.  (My Rakefile automatically installs needed gems if you don’t have them)
  • I have a framework for creating ruby classes with selenium locators automatically. I use nokogiri to help with this.
  • Spec test to open ruby classes with locators and create a report of all of the changed selenium locators.

Future Goals

  • Update Deep Test so that it uses the same version(1.2.8) of rspec as the latest version of Selenium-Client. Right now we are at rspec 1.1.12 and selenium client 1.2.18. Selenium-Client 1.2.18 can use rspec 1.2.8.
  • When rspec is updated, I will be able to use the selenium client to provide screen shots if needed
  • Figure out how to create a formatter that allows the Team City and Rubymine spec runners to provide the same test runner template as it does for test in sequence when you run test in parallel in deep test.
    Rubymine Spec Formatter

    Rubymine Spec Formatter

    Team City Test Report

Advanced Test Automation – Generate your own css locators using nokogiri

Some of the pages that I test have hundreds of check boxes. I needed a way to generate the locators to use in selenium and create classes to use those locators.
This is an example of a class that contains selenium css locators for navigation links

class Navigation
  def initialize * browser
    @location =  "css=a:contains(\"Location\")"
    @amenities =  "css=a:contains(\"Amenities\")"
    @photos =  "css=a:contains(\"Photos\")"
    @contact =  "css=a:contains(\"Contact\")"  
    @further_details =  "css=a:contains(\"Further Details\")"
  end
  attr_accessor :location, :amenities, :photos, :contact, :further_details
end

To use this class I would new it up and use standard selenium commands. This is how I would navigate to the amenities page

navigation = Navigation.new
browser.click amenities

The spec below demonstrates how to get data you need using nokogiri. I want to click an anmenity check box with the label beach whose input name is “amenity_1_1_6”. I could click this checkbox with this command browser.click “amenity_1_1_6”, but I want to have a way to store all of the amenities on the page with the variable named by their label.

require "spec"
require "nokogiri"
describe "Find elements to use as locators in a web test" do

  it "should find the value of name for the input element and the text in span" do
    sample_html = ''
    attribute_name=""
    attribute_value=""
    doc = Nokogiri::HTML(sample_html)
    doc.css("label").each do |check_box|
      attribute_name =  check_box.text
      attribute_value = check_box.children.css("input")[0]["name"]
      attribute_name = attribute_name.strip
      attribute_name = attribute_name.to_s.downcase

    end
    attribute_name.should=="beach"
    attribute_value.should=="amenity_1_1_6"
  end
end

To take this further I can print the contents of my class file to generate my locators from this spec.

puts "@#{attribute_name} = \"#{attribute_value}\""

Rubymine 2.0.2 and the gem bundler

I use bundler 0.9.6 with all of my projects. Checkout my earlier post on how to add bundled gems to your project.  I am excited about the updated bundler support in Rubymine 2.0.2 for a few reasons.

Rubymine can easily find my bundled gems in my Gemfile and attach gems to my project with suggestions. Now I can Go To Declaration for all of my attached gems.

Attech gems from the Gemfile

Attech gems from the Gemfile

I don’t have to navigate to my project through the terminal to unlock the Gemfile and add a new gem. I can use the bundler menu for all bundler functions.

Bundler 9.0 options

Bundler 9.0 options

I can get feedback from my bundler commands inside the Rubymine run window.

Unlocking through rubymine

Unlocking through rubymine

Selenium-Client 1.2.18 has excellent logging and error messages.

I am impressed with the new logging and stability features of  Selenium-Client 1.2.18. One of the most useful improvements is the automatic failure when a page is showing service unavailable.

I used to put a line like this after every browser.open command.

browser.text?("Http/1.1 Service Unavailable").should(be_false)

Now I get a nice error message from the Selenium-Client.
Selenium::CommandError: Response_Code = 503 Error_Message = Service Unavailable

Deploy your Ruby project with out the need to install gems.

I have been using the gem bundler lately with my ruby projects to deploy to team city build agents.  My project needed to run on multiple OS X and Cent OS systems without having to manage ruby gems manually.  The bundler gem repository is located on github at http://github.com/carlhuda/bundler.  There are a few different ways to use it so I decided to write about how use it.  The short answer is that I lock and pack the gems I need, and fall back to sudo installed gems when needed. I am using bundler 0.9.6. The prerequisite is that you gem install bundler.

Use a begin / rescue to try to require the environment file.
If the file is not found then execute the bundle install command

begin
  # Try to require the preresolved locked set of gems.
  require File.dirname(__FILE__) + "/../.bundle/environment"
rescue Exception=>e
  # Fall back on doing an unlocked resolve at runtime.
  if (!system("bundle install"))
        puts $?
  end
end
require File.dirname(__FILE__) + "/../.bundle/environment"

On the first run the bundle install command will execute
This operation will create a .bundle directory in the root of your project that will contain an environment.rb file that will contain all of your gem configuration information.
All your files need to do, is include this environment file and they will have access to your bundled gems. If you run the bundle install command twice in a row it will not install again if your gems have already been installed.

Three step process to setup your bundler.
Create a Gemfile that will list what gems to install. Lock your gem file and then pack your gems.
Check out the information on github to see the format of the gemfile. You can use the bundle init command to create the Gemfile for you in the base of your project. Now run the bundle lock command. A Gemfile.lock file will be created you will check this in to your version control system. Next you need to pack the gems. Run the command bundle pack. You will see that a vendor/cache directory will be created. You will need to check in the vendor cache directory into your VCS. Now when your deploy your code will self execute the bundle install command making all of the gems you need available too your project.

Working with compiled gems
I really like using the Nokogri gem but I foud out that when I execute my project with in the Team City execution environment with a Cent OS build agent, that Nokogiri cannot build correctly because it cannot access the system files it needs. The work around is to list nokogiri in your Gemfile, but don’t check in the gem into the gem cache. Install it as the user that will be executing your scripts so it will be available to your execution environment. The bundler will resolve the dependency to your system installed gem.

That was a quick rundown of what you can get using the bundler and how to set it up. I will continue to post updates as I come across more challenging situations. For now I still have to touch every machine that needs odbc ruby configuration and nokogiri.

Test navigation in Rubymine (control +T )

I like to navigate through Rubymine using only keyboard shortcuts if I can. One of the most useful techniques I have found is mapping a keyboard shortcut to enable you to run the test that your cursor is on. This way you can navigate up and down tests using ctl + up or down and then run the test your interested in with your key stroke. Even though I am on a Mac I prefer running a test to be mapped to control + t. The default configuration for this is command + alt + F8. The keymap configuration is under the label Run context configuration, which is under the All Actions -> Other section in the Keymap configuration menu.

Picture 2

The other shortcut that goes well with this one is Go To Test. I can put my cursor on a class name and press alt+shift+t and I can view the tests implementation for this class. Then I can use the short cut I created ctrl + t to run a test I am interested in.

Don’t miss this important note if your planning to run tests against Internet Explorer using Selenium Grid

I tried to run tests in parallel against IE using the selenium grid for a few days before I found this important message on the selenium grid FAQ.
Picture 9

If you try to run multiple selenium remote controls for IE on one machine it will work for a while.   Your test will execute without any problems, but then you start getting sporadic error messages from IE.  I assumed that the xpath expressions I was using for the tests in Firefox were throwing errors in IE.  I tried different configurations for security settings in IE for a while and then I finally found out that it was a memory sharing issue that prevents the selenium remote from launching more than one IE instance.  For now I am just requesting more VMs to run a single IE remote on each.  I use ruby with deep test to run test in parallel on the grid.  I configure deep test to run 6 tests at a time, so I would only need 6 VMs for IE to run.

It puzzling to me that Watin tests run in parallel against IE without any problems on the same machine, but selenium tests cannot.  For now I will add more VMs and wait for Selenium Grid 1.2  to fix the IE problem.

Configure the Team City rake runner to launch Selenium Grid tests.

This article will describe how I was able to use the team city rake runner to launch selenium grid tests.  I learned a few tricks to get this to work correctly.   I first had to learn the makeup of a rspec specktask.  Next I needed to understand how to pass in spec options and lastly I needed to write a rake task that would execute in an exact order.  I needed to still generate the html report with the rake task so I had to add the formatter path to the rake runner

Setup the report formatter path in your rake runner like you do in your rake file.
See the code below to see how the formatter path is setup in the rake file. You can also see how the formatter path is used by the rake task.

# Make sure we pick up the reporter from the appropriate selenium-client
# install as RSpec runner --require does not discriminate between multiple
# selenium-client gems.
report_formatter_path = `gem which -q "selenium/rspec/reporting/selenium_test_report_formatter"`.chomp
report_formatter_path.gsub! /selenium-client-\d+\.\d+.\d+/, "selenium-client-1.2.7"
Spec::Rake::SpecTask.new("spec:run_in_parallel_report") do |t|
  t.spec_files = FileList['./*_spec.rb']
  t.deep_test :number_of_workers => 6,
              :timeout_in_seconds => 300
  t.spec_opts << '--color'
  t.spec_opts << "--require 'rubygems,#{report_formatter_path}'"
  t.spec_opts << "--format=Selenium::RSpec::SeleniumTestReportFormatter:./tmp/test_report.html"
  t.spec_opts << "--format=progress"
  t.fail_on_error = false
end

Setup the spec opts in the rake runner.
See the image below. I need to tell team city which formatter to use and where to put the html report. To pass in spec options to the rake runner in team city just put them in the text field labeled RSpec options(SPEC_OPTS): .
My full spec ops string is:

–require ‘/usr/lib/ruby/gems/1.8/gems/selenium-client-1.2.7/lib/selenium/rspec/reporting/selenium_test_report_formatter.rb’ –format=Selenium::RSpec::SeleniumTestReportFormatter:./tmp/test_report.html
RakeRunner

Run your rake task.
I made a rake task to take selenium grid environment variables as arguments.  If you wanted to run the task above, just add the task name  “spec:run_in_parallel_report” in your rake configuration. You don’t need any command line parameters for the spec:run_in_parallel_report task.