Test creation

Let's consider a scenario testing Google search and use it as an example of test definition. Let's name our test 'GoogleSearch'. Name has to be upper case since it will be used as Ruby class name.

First let's assume that our 'Demo' project has been created and is properly referred in bubik.ini. The following steps of test definitions are:

• create a directory 'GoogleSearch' in project's Demo/tests/ parent directory
• in this directory create file 'GoogleSearch.rb'

Basic scenario

Let's consider the following test scenario:

• go to http://www.google.com
• type term "England" into input field
• wait for google to perform search and display results and result stats
• read the number of results
• assert that it's greater than 1000

Additionally let's declare that this test belongs to test groups (aka suites) 'regression' and 'stats', and define its short description.

The example content of test file would be:

#GROUPS: regression, stats
#DESC: A tutorial scenario for google search

driver = handler( :webdriver ).driver
driver.navigate.to "http://www.google.com"

element = driver.find_element( :name, 'q' )
element.send_keys "England"

wait = Selenium::WebDriver::Wait.new( :timeout => 10 ) 
wait.until { driver.find_element( :xpath, "//div[ @id='resultStats' ]" ) }

stats = driver.find_element( :xpath, "//div[ @id='resultStats' ]" ).text
# stats contains something like: "About 14,700,000 results (0.61 seconds)"

# need to extract the number and convert it from string to int
num = stats.match( /([\d\.,]+\s?)+/ )[0].strip.gsub( /,|\.| / , "" ).to_i

log( "found: #{num} results" )

assert( num >= 1000 , "Number of results should be greater than 1000" )

For those familiar with WebDriver this listing looks pretty straightforward. There are only few points that require some clarification:

• #GROUPS - assign current test to coma separate test groups
• #DESC - write short summary of what test is doing, can be used multiple times
• handler( :webdriver ) - returns specific handler object
• log( message ) - writes given message to log file specific for current test run
• assert( condition, message ) - an assertion, does NOT terminate test when not met

To execute this test and see the results in the console the following command should be used:

bubik -tGoogleSearch -rconsole

Introducing parameters

Looking at the scenario above we can see that it could be easily reused for different parameters: search term, and expected number of results. In the example there are hardcoded "England" and 1000). In order to introduce parameters to this test we need to create a CSV file which will hold parameter values, and then refer to them in test using param( 'param_name' ) function.

The CSV file has to be saved in the same directory as GoogleSearch.rb under name GoogleSearch.csv. Expected format: semicolon (;) as separator, quote as string delimiter ("), UTF-8 encoding with no BOM. And example parameters file for our test scenario could look like this:

term	min_results
England	1000
Polska	1000
Deutschland	2000
Россия	3000

Modified test that will be executed 3 times for each set of parameters (line from CSV file) should look like below. A single set of parameters (line from CSV file) is called a test variant. If there is no CSV file with parameters test has one variant (not parametrized).

#GROUPS: regression, stats
#DESC: A tutorial scenario for google search

driver = handler( :webdriver ).driver
driver.navigate.to "http://www.google.com"

element = driver.find_element( :name, 'q' )
element.send_keys param( 'term' )

wait = Selenium::WebDriver::Wait.new( :timeout => 10 ) 
wait.until { driver.find_element( :xpath, "//div[ @id='resultStats' ]" ) }

stats = driver.find_element( :xpath, "//div[ @id='resultStats' ]" ).text
# stats contains something like: "About 14,700,000 results (0.61 seconds)"

# need to extract the number and convert it from string to int
num = stats.match( /([\d\.,]+\s?)+/ )[0].strip.gsub( /,|\.| / , "" ).to_i

log( "found: #{num} results" )

expected_min = param( 'min_results' ).to_i
assert( num >= expected_min , "Number of results should be greater than #{expected_min}" )

As presented in the example hardcoded values have been replaced with param() function calls. Please bear in mind that values returned from this function are strings, so conversion to other types (number, bools, etc) might be needed. In the example above since value of 'min_results' is compared with an integer value in assert() call it has to be converted to integer before. Hence to_i appearing in the line above assert function call.

Introducing constants

Looking at this example one could see that there are still another two values hardcoded: the URL of the page to start from, and the XPath expression locating text with search stats on the page. Since these values might be used in other tests as well it makes sense to extract them to some place accessible for all code related to the project. Bubik provides this features using ini files that are saves in 'const' subdirectory. Since in bigger projects list of constants can grow significantly bubik supports grouping these constants into files. For our example let's locate start URL and XPath location in separate files.

If there's only one consts file (smaller projects) it should be named default.ini. When more files come the names don't matter - they just have to be saved in the same directory as default.ini

For our example default ini will look like this (section 'common' is required):

[common]
start_url = "http://www.google.com"

For XPath locator, which is semantically connected to UI let's create file ui.ini with the following contents:

[common]
locator.stats = "//div[ @id='resultStats' ]"

Now let's use function const() to access these files in the test:

#GROUPS: regression, stats
#DESC: A tutorial scenario for google search

driver = handler( :webdriver ).driver
driver.navigate.to const( 'start_url' )

element = driver.find_element( :name, 'q' )
element.send_keys param( 'term' )

wait = Selenium::WebDriver::Wait.new( :timeout => 10 ) 
wait.until { driver.find_element( :xpath, const( 'ui.locator.stats' ) ) }

stats = driver.find_element( :xpath, const( 'ui.locator.stats' ) ).text
# stats contains something like: "About 14,700,000 results (0.61 seconds)"

# need to extract the number and convert it from string to int
num = stats.match( /([\d\.,]+\s?)+/ )[0].strip.gsub( /,|\.| / , "" ).to_i

log( "found: #{num} results" )

expected_min = param( 'min_results' ).to_i
assert( num >= expected_min , "Number of results should be greater than #{expected_min}" )

Please keep in mind that constants from files other than default.ini need to be referenced with file specific prefix added to constant name.

Introducing environment specific constants

Very often it happens that one application is being tested on different environments. This might refer to different stages of development and testing (dev, integration, staging) of different localizations of our site. Let's imagine that we'd like to run out test not against google.com domain, but rather localized versions for UK, Poland, Germany and Russia. In order to do that we don't need to change neither the test script nor the parameters file. The environments are defined in const file, and referred in execution command.

In our case let's change the defaut.ini file in the way so that definition of 'start_url' would be different for different environments. The environments are defined as sections of the ini file:

[common]
start_url = "http://www.google.com"

[en]
start_url = "http://www.google.co.uk"

[pl]
start_url = "http://www.google.pl"

[de]
start_url = "http://www.google.de"

[ru]
start_url = "http://www.google.ru"

Once this file is saved the name of the environment that we'd like to use for test execution should be passed as parameter to bubik command. So to run the test for 'pl' and 'de' environments the command should look like this:

bubik -tGoogleSearch -rconsole -epl,de

Since there are 4 variants defined in CSV file and tests should be executed in 2 environments test will be executed 8 times. Each variant once per each environment.

Introducing environment specific parameters

Having multiple environment specific constants in default.ini file we might want to use them to define test variants specific to given environment. The most common scenario utilizing this feature is testing translations in internationalized websites. Let's extend our example scenario with assertion on one of page headers. It will be different for Polish, German and Russian site localizations, but using environments and CSV file we can test both three language versions without having to change the test scenario file.

Let's start from extending the CSV file with column for the translation that we expect to appear on the site, and want to assert on. Let's name this column 'search_label'. In order to associate variants (aka parameter sets, CSV rows) with environments a column '__env' has to be added. With these two columns CSV file will look like this:

__env	term	min_results	search_label
en	England	1000	Search
pl	Polska	1000	Wyszukiwarka
de	Deutschland	2000	Suche
ru	Россия	3000	Поиск

Now let's add the assertion on the search label to test script, so it looks like this:

#GROUPS: regression, stats
#DESC: A tutorial scenario for google search

driver = handler( :webdriver ).driver
driver.navigate.to const( 'start_url' )

element = driver.find_element( :name, 'q' )
element.send_keys param( 'term' )

wait = Selenium::WebDriver::Wait.new( :timeout => 10 ) 
wait.until { driver.find_element( :xpath, const( 'ui.locator.stats' ) ) }

stats = driver.find_element( :xpath, const( 'ui.locator.stats' ) ).text
# stats contains something like: "About 14,700,000 results (0.61 seconds)"

# need to extract the number and convert it from string to int
num = stats.match( /([\d\.,]+\s?)+/ )[0].strip.gsub( /,|\.| / , "" ).to_i

log( "found: #{num} results" )

expected_min = param( 'min_results' ).to_i
assert( num >= expected_min , "Number of results should be greater than #{expected_min}" )

search_label = driver.find_element( :xpath, "//div[ @id='ab_name' ]/span" ).text
assert( search_label.eql?( param( 'search_label' ) ), "Search labels have to matach" )

Let's run this test for 'en' and 'pl' environments using command:

bubik -tGoogleSearch -rconsole -een,pl

Unlike in previous example, where '__env' column was not set and all variants were executed as many times as many environments were passed to command, in this example each variant will be executed only once, for the environment it is assigned to by '__env' column.

That's a short introduction, please search for more information in other sections of this wiki.resultStats is compared with an integer value in assert() call it has to be converted to integer before. Hence to_i appearing in the line above assert function call.

Introducing constants

Looking at this example one could see that there are still another two values hardcoded: the URL of the page to start from, and the XPath expression locating text with search stats on the page. Since these values might be used in other tests as well it makes sense to extract them to some place accessible for all code related to the project. Bubik provides this features using ini files that are saves in