Selenium Fury 5.2 with custom generators has been released.

The latest release of Selenium Fury is ready for your install.  Checkout the new cucumber features at the project home  https://github.com/scottcsims/SeleniumFury. This version includes extended page element recognition, custom generators, and the git hub project includes support for bundler and rvm.

Install the new release:   gem install selenium_fury

https://rubygems.org/gems/selenium_fury

 

What I liked about “Perfect Software And Other Illusions About Testing” by Gerald M. Weinberg

Perfect Software And Other Illusions About Testing
Perfect Software And Other Illusions About Testing

I recently finished reading “Perfect Software and other illusions about testing” by Gerald Weinberg.  I wanted to elaborate and reflect on a few points I enjoyed about the book .  I recently attended the Software Test Professionals Conference and Gerald Weinberg was there signing his book. I had not heard of him at that point, but now I will be at the front of the line for the next book signing.

The first chapter was a home run, it was titled “Why do we bother testing?”  I found that this statement shed new light on an old subject for me.

“Common mistake 5: Believing testing can improve a product: Testing gathers information about a product; it does not fix things it finds that are wrong.  Testing does not improve a product; the improving is done by people fixing the bugs that testing has uncovered.  Often when managers say, “Testing takes too long” what they should be saying is, “Fixing the bugs is the product takes too long, ” — a different cost category. Make sure your accounting for effort and time under the correct cost category.”

Chapter four is titled  “Whats the difference between testing and debugging”.  I found two interesting answers about locating faults and pinpointing that I was not expecting.  I remember thinking at one point in my testing carrer that if I could find a defect and then locate in code the source of the problem that I would be considered an amazing tester. Who should pinpoint failures?

“Common mistake 4:  Demanding that testers pinpoint every failure:  Testers can help developers with this job, if there time has been scheduled for it, but it is ultimately a developer responsibility.  At least that’s what I’ve seen work best in the long run. “

“Common mistake 5:  Demanding that testers locate every fault:  This is totally a developer’s job, because developers have the needed skills.  Testers generally don’t have these skills, though at times, they may have useful hints.”

In chapter thirteen different strategies are discussed for determining significance.  Significance is the importance attached to the bug by the person who gets to decide what to do about that bug.  I liked these four categories for determining the significance of an issue from a testing point of view

Level 0: This issue is blocking other testing.
Level 1:  Our product cannot be used if this issue isn’t resolved
Level 3:  The value of our product will be significantly reduced if this issue isn’t resolved.
Level 4:  This issue will be important only if there are large numbers of similar issues when the product is shipped.

From the high level project view, think about your project and how important fixing is.   I learned that fixing is linked to an emotional value that is given to every fixable work item.  It is possible that every member of the team might assign a different emotional value to ever fixable work item.  In this case, how do you manage what gets fixed in what order if all members of your team members have different values assigned to your fixable work items? I like the summary of chapter thirteen to answer this question.

“Our emotions carry information about how important things are. If we pay attention to emotions, listen, and address important matters before unimportant matters, we’ll be doing the best we can with the data we have.”

In conclusion I highly suggest this book for anyone developing or testing software.  It will also be helpful to anyone making decisions about how software is released or built.

The Selenium Fury gem is ready! Page object factory for Selenium and Ruby.

I have been working on converting our page object factory to open source for a few weeks. Now it is time to launch the  HomeAway sponsored open source project under the Apache 2.0 license.  It is a furiously quick way to implement test automation.  I am planning to add more configuration options in the future.

This project started when I had to test a page with 300+ check boxes and I did not want to enter them by hand.  I used the page object generator to build a page of ruby variables with Selenium locators. Everything was great until some number of the check boxes changed their ids and my tests started failing.  I needed  a quick way to find out how many changed so I could update the locators by hand or regenerate the page.  This is where the validators came in.  I used Ruby’s support of reflection to open a class, navigate to the url of the page and use Selenium to validate the locators and return a list of missing locators on the page.  It worked perfectly for my page of 300+ check boxes.  I had over 40 that changed I quickly regenerated the page.

Install with:

  • gem install selenium_fury

Checkout the home page and examples at https://github.com/scottcsims/SeleniumFury

Thanks to HomeAway for sponsoring this project.

Use rspec partial mock to write a quick executing example.

I wanted to test an error message in my class that processes test results and sends them to Rally.  The problem was that I did not want to actually create a rally object and run the method to parse the results, just to test the error message.
This is what my class looks like.

class AutomationRun
  def send_results_to_rally
    @rally = RallyUtils.new(ENV['RALLY_WORKSPACE'])
    push_test_case_results
    raise("Test Case Results Were Not Parsed Correctly") if test_case_results.empty?
    test_case_results.each do |result|
      @rally.update_test_case_result(:tc_id =>result.test_case_number_from_spec_report, :build =>result.build_number, :verdict =>result.verdict, :notes => result.note.format_note)
    end
  end
end

I want to skip these steps to test the error message:

  • don’t instantiate RallyUtils.new  we don’t need @rally for the test
  • don’t parse the results so don’t call parse_test_case_results

This is what my test code looks like:

 it "should raise an exception if there are not test case results" do
    RallyUtils.stub!(:new).and_return(nil)
    automation_run=AutomationRun.new
    automation_run.should_receive(:push_test_case_results).and_return([])
    begin
    automation_run.send_results_to_rally
    rescue Exception=>e
    end
    e.message.should == "Test Case Results Were Not Parsed Correctly"
  end
  1. We use a rspec stub to intercept the new call to the RallyUtils class and return nil.  We could have also returned a mock object if we wanted to execute a call on the rally class. Now we are not creating a connection to Rally
  2. Now it is time to new up or AutomationRun class and setup our mock expectation to skip running the real push_test_case_results method.
  3. automation_run.should_receive tells us to intercept the symbol or method push_test_case_results
  4. and_return([]) will just return an empty array
  5. We setup a begin rescue block to capture the Exception object returned by our expected error message
  6. Now we call send_results_to_rally to get our error message
  7. Lastly we verify that the message attribute for our error is Test Case Results Were Not Parsed Correctly

So that is it, I created a partial mock  in that we are using a real automation run object but we stubbed out one of the methods.  We also stubbed out the new call to rally so we don’t create a connection.  Now I can focus my test on exactly the function I wanted to test, the error message.

Going to Software Test Professionals Conference next week.

I am looking forward to my first trip to Las Vegas http://www.stpcon.com/. I will have a chance to meet with other selenium grid users and test out my strategies in test automation patterns. I will be attending all of the topics on agile automation

Deep Test 2.0 prerelease is available.

Deep test allows you to run ruby rspec tests in parallel by providing a spec task that you include in your rake file. The 2.0 prerelease version of deep tests offers support for rspec 1.1.12. You can install it buy running: gem install deep_test_pre. Require it in your rake file like this:

gem “deep_test_pre”, “=2.0”
require “deep_test”

gem “deep_test_pre”, “=2.0”
require “deep_test”

There currently is a bug that occurs if you run Test::Unit tests with the deep test spec runner.  The tests will all run twice but are reported once.

Selenium Grid Projects

What I have already done to update the ruby example in the Selenium Grid repo. Code Link

  • Updated rspec from 1.1.8 to 1.1.12. I accomplished this by getting the most recent deep-test code  and adding it to the /lib directory. The new version of the deep test code supports rspec 1.1.12. I then just require it in in the rake file. require File.expand_path(File.dirname(__FILE__) + ‘/lib/deep_test/rake_tasks’)
  • The rake file does not need selenium-client 1.2.7. I removed the dependency by  adding the reporting files it was referencing into the lib/reporting  directory. report_formatter_path = “lib/reporting/selenium_test_report_formatter”
  • Updated selenium-client from 1.2.7 to 1.2.18. With the change to the rake file, I am now able to use the most recent version of selenium-client in my specs.

Other helpful tools/tricks I have added to selenium grid (Maybe useful to other Ruby/Grid users)

  • I start and stop the grid from the same rake file that I execute the tests from. Examples are grid:all_start and spec:run_in_parallel
  • Rake tasks can just run and create a HTML report or run and send test results to Rally
  • The tests and rake file runs on OS X, Cent OS, and Windows. (no parallelizaion on windows)
  • Specify command options to the rake tasks to switch between hostnames I am running against. (This is useful when you have more than one instance like test.yourdomain.com and stage.yourdomain.com)
  • Specify to only run tests for specific sites in the test suit. For example if you wrote tests for www.yourdomain.com and www.yourdomain.es.  You can specify which sites to run against
  • I use gem bundler with all grid projects to help control gem installation and versions.  (My Rakefile automatically installs needed gems if you don’t have them)
  • I have a framework for creating ruby classes with selenium locators automatically. I use nokogiri to help with this.
  • Spec test to open ruby classes with locators and create a report of all of the changed selenium locators.

Future Goals

  • Update Deep Test so that it uses the same version(1.2.8) of rspec as the latest version of Selenium-Client. Right now we are at rspec 1.1.12 and selenium client 1.2.18. Selenium-Client 1.2.18 can use rspec 1.2.8.
  • When rspec is updated, I will be able to use the selenium client to provide screen shots if needed
  • Figure out how to create a formatter that allows the Team City and Rubymine spec runners to provide the same test runner template as it does for test in sequence when you run test in parallel in deep test.
    Rubymine Spec Formatter

    Rubymine Spec Formatter

    Team City Test Report

Advanced Test Automation – Generate your own css locators using nokogiri

Some of the pages that I test have hundreds of check boxes. I needed a way to generate the locators to use in selenium and create classes to use those locators.
This is an example of a class that contains selenium css locators for navigation links

class Navigation
  def initialize * browser
    @location =  "css=a:contains(\"Location\")"
    @amenities =  "css=a:contains(\"Amenities\")"
    @photos =  "css=a:contains(\"Photos\")"
    @contact =  "css=a:contains(\"Contact\")"  
    @further_details =  "css=a:contains(\"Further Details\")"
  end
  attr_accessor :location, :amenities, :photos, :contact, :further_details
end

To use this class I would new it up and use standard selenium commands. This is how I would navigate to the amenities page

navigation = Navigation.new
browser.click amenities

The spec below demonstrates how to get data you need using nokogiri. I want to click an anmenity check box with the label beach whose input name is “amenity_1_1_6”. I could click this checkbox with this command browser.click “amenity_1_1_6”, but I want to have a way to store all of the amenities on the page with the variable named by their label.

require "spec"
require "nokogiri"
describe "Find elements to use as locators in a web test" do

  it "should find the value of name for the input element and the text in span" do
    sample_html = ''
    attribute_name=""
    attribute_value=""
    doc = Nokogiri::HTML(sample_html)
    doc.css("label").each do |check_box|
      attribute_name =  check_box.text
      attribute_value = check_box.children.css("input")[0]["name"]
      attribute_name = attribute_name.strip
      attribute_name = attribute_name.to_s.downcase

    end
    attribute_name.should=="beach"
    attribute_value.should=="amenity_1_1_6"
  end
end

To take this further I can print the contents of my class file to generate my locators from this spec.

puts "@#{attribute_name} = \"#{attribute_value}\""

Rubymine 2.0.2 and the gem bundler

I use bundler 0.9.6 with all of my projects. Checkout my earlier post on how to add bundled gems to your project.  I am excited about the updated bundler support in Rubymine 2.0.2 for a few reasons.

Rubymine can easily find my bundled gems in my Gemfile and attach gems to my project with suggestions. Now I can Go To Declaration for all of my attached gems.

Attech gems from the Gemfile

Attech gems from the Gemfile

I don’t have to navigate to my project through the terminal to unlock the Gemfile and add a new gem. I can use the bundler menu for all bundler functions.

Bundler 9.0 options

Bundler 9.0 options

I can get feedback from my bundler commands inside the Rubymine run window.

Unlocking through rubymine

Unlocking through rubymine

Selenium-Client 1.2.18 has excellent logging and error messages.

I am impressed with the new logging and stability features of  Selenium-Client 1.2.18. One of the most useful improvements is the automatic failure when a page is showing service unavailable.

I used to put a line like this after every browser.open command.

browser.text?("Http/1.1 Service Unavailable").should(be_false)

Now I get a nice error message from the Selenium-Client.
Selenium::CommandError: Response_Code = 503 Error_Message = Service Unavailable