Using unit tests in a
test automation project brings a lot of benefits
- create short test automation scripts
- have independent test automation scripts
- have easy to maintain code
What follows is a very long article about why unit testing is
essential for test automation.
It includes real code that you can use by yourself.
If you have any questions on the code or the article, please leave them in the Comments section.
Enjoy!
TEST AUTOMATION WITHOUT UNIT TESTS
Lets start with a sample project that aims at automating 2 test cases.
Both test cases describe user scenarios for the Vancouver Public Library site:
Test Case 1
1. open the home page of the site (http://www.vpl.ca)
2. Check that the home page title is correct
3. Do a keyword search; the results page opens after the keyword search
4. Check that the results page title is correct
5. Click on the first result of the results page; the details page opens
6. On the details page, check that the book title is displayed and not empty
7. On the details page, check that the book author is displayed and not empty
Test Case 2
1. open the home page of the site (http://www.vpl.ca)
2. Check that the home page title is correct
3. Do a keyword search; the results page opens after the keyword search
4. Check that the results page title is correct
5. Check that the results count is displayed and is correct for page 1
6. click on page 2 of results
7. Check that the results count is displayed and is correct for page 2
See below the code of the project.
It uses
Java as the programming language.
//TEST CASE 1: book title and author are displayed and not empty
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.firefox.FirefoxDriver;
public class LibraryTestCases {
public static void main(String[] args) {
WebDriver driver = new FirefoxDriver();
//open the home page and check that the home page title is correct
driver.get("http://www.vpl.ca");
if (driver.getTitle().equalsIgnoreCase("Vancouver Public Library - Home") == true)
System.out.println("correct home page title");
else
{
System.out.println("incorrect home page title, stop application");
System.exit(0);
}
//search with a keyword, the results page is opened as result of the search
WebElement searchTextBox;
searchTextBox = driver.findElement(By.xpath("//input[@id='globalQuery']"));
searchTextBox.click();
searchTextBox.sendKeys("java");
WebElement searchButton;
searchButton = driver.findElement(By.xpath("//input[@class='search_button']"));
searchButton.click();
//check that the results page title is correct
if (driver.getTitle().equalsIgnoreCase("Search | Vancouver Public Library | BiblioCommons") == true)
System.out.println("correct results page title");
else
{
System.out.println("incorrect results page title, stop application");
System.exit(0);
}
//click on the first result, the details page is opened as a result
WebElement firstResult;
firstResult = driver.findElement(By.xpath("(//a[@testid='bib_link'])[1]"));
firstResult.click();
//check that the title of details page is correct
if (driver.getTitle().indexOf("Vancouver Public Library | BiblioCommons") >= 0)
System.out.println("correct details page title");
else
{
System.out.println("incorrect details page title, stop the application");
System.exit(0);
}
//check that the book title is displayed and not empty
WebElement bookTitle = driver.findElement(By.xpath("//h1[@id='item_bib_title']"));
if (bookTitle.isDisplayed() == true)
System.out.println("book title displayed");
else
{
System.out.println("book title not displayed, stop application");
System.exit(0);
}
if (bookTitle.getText().length() > 0)
System.out.println("the book title not empty");
else
{
System.out.println("book title empty, stop application");
System.exit(0);
}
//check that the book author is displayed and not empty
WebElement bookAuthor;
bookAuthor = driver.findElement(By.xpath("//a[@testid='author_search']"));
if (bookAuthor.isDisplayed() == true)
System.out.println("book author displayed");
else
{
System.out.println("book author not displayed, stop application");
System.exit(0);
}
if (bookAuthor.getText().length() > 0)
System.out.println("book author not empty");
else
{
System.out.println("book author empty, stop application");
System.exit(0);
}
//TEST CASE 2: browsing through results pages works
//go back to results page
driver.navigate().back();
//check that the number of results is displayed and correct
WebElement numberResultPageOne;
numberResultPageOne = driver.findElement(By.xpath("//span[@class='items_showing_count']"));
if (numberResultPageOne.isDisplayed() == true)
System.out.println("Results number on page 1 - displayed");
else
{
System.out.println("Results number on page 1 - not displayed, stop app");
System.exit(0);
}
if (numberResultPageOne.getText().indexOf("1 - 25") >=0)
System.out.println("Results number on page 1 - correct");
else
{
System.out.println("Results number on page 1 - not correct, stop app");
System.exit(0);
}
//navigating to page 2 of results
WebElement pageTwo = driver.findElement(By.xpath("//a[@testid='link_page2']"));
pageTwo.click();
//check that the number of results is displayed and correct
WebElement numberResultPageTwo;
numberResultPageTwo = driver.findElement(By.xpath("//span[@class='items_showing_count']"));
if (numberResultPageTwo.isDisplayed() == true)
System.out.println("Results number on page 2 - displayed");
else
{
System.out.println("Results number on page 2 - not displayed, stop app");
System.exit(0);
}
if (numberResultPageTwo.getText().indexOf("26 - 50") >=0)
System.out.println("Results number on page 2 - correct");
else
{
System.out.println("Results number on page 2 - not correct, stop app");
System.exit(0);
}
//close the browser
driver.quit();
}
}
The code goes through the following phases:
1.
create the browser driver object; the Firefox browser is loaded as a result
2.
Test Case 1 code; it goes from home page to results page and from the results page to details page
3.
go back from details page to results page; this is needed because test case 2 happens on results page
going back from details page to results page is done to avoid going through the home page for test case 2
4.
Test Case 2 code
5.
quit the browser driver object; the Firefox browser is closed as a result
Each test case has a few verifications.
Each verification logs information to the console panel.
If the verification passes, the code execution continues.
If the verification fails, the application is stopped.
WHAT IS WRONG WITH THIS SAMPLE PROJECT?
The sample code is not efficient for multiple reasons:
1.
if one of the test case 1 verifications fails, the application stops; test case 2 is completely ignored
2. to understand if the test cases passed or failed,
the test automation engineer needs to read all information from the console panel
3.
the code is long; its maintenance will not be easy
How do we fix these issues?
UNIT TESTING TO THE RESCUE
First, some unit testing basics.
We will continue with a code example shortly.
- Unit testing is being done with the help of unit testing frameworks.
- With a unit testing framework, developers create short unit test scripts that test their application code.
- The unit tests are code that tests the application code.
- Each unit test has the purpose of testing a single application entity (one class, one object).
- Each of the unit tests is independent of the other unit tests.
- Unit tests are being created in test classes.
- They can be identified from normal class methods through annotations.
- A unit test verifies if the tested entity works as expected. This is done using assertions.
- Developer can create multiple unit tests for his application code. Test runners are used for running both individual and multiple unit tests.
- Before executing unit tests, the environment may need to be set up. The environment should be cleaned up after the unit test executes.
- The set up and cleaning of the environment is being done through test fixtures.
TEST AUTOMATION WITH UNIT TESTS
I will refactor the sample project in a few different steps:
- break the code in unit tests
- use assertions for test case verifications
- use test fixtures for setting up and cleaning the test environment
The JUNIT framework will be used in the code examples.
The code can be changed easily to use the
Test NG framework.
1. BREAK THE CODE IN UNIT TEST SCRIPT
Lets see how the code can be broken down.
I will use a test class that does not have a main method and includes 2 unit test scripts (one for each test case).
Each unit test is prefixed with the @Test annotation.
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.firefox.FirefoxDriver;
public class LibraryTestCases {
//Test Case 1: book title and author are displayed and not empty
@Test
public void testResultDetails()
{
WebDriver driver = new FirefoxDriver();
//open the home page and check that the home page title is correct
driver.get("http://www.vpl.ca");
if (driver.getTitle().equalsIgnoreCase("Vancouver Public Library - Home") == true)
System.out.println("the home page title is correct");
else
{
System.out.println("the home page title is incorrect, stop the application");
System.exit(0);
}
//search with a keyword, the results page is opened as result of the search
WebElement searchTextBox = driver.findElement(By.xpath("//input[@id='globalQuery']"));
searchTextBox.click();
searchTextBox.sendKeys("java");
WebElement searchButton = driver.findElement(By.xpath("//input[@class='search_button']"));
searchButton.click();
//check that the results page title is correct
if (driver.getTitle().equalsIgnoreCase("Search | Vancouver Public Library | BiblioCommons") == true)
System.out.println("the results page title is correct");
else
{
System.out.println("the results page title is incorrect, stop the application");
System.exit(0);
}
//click on the first result, the details page is opened as a result
WebElement firstResult = driver.findElement(By.xpath("(//a[@testid='bib_link'])[1]"));
firstResult.click();
//check that the title of details page is correct
if (driver.getTitle().indexOf("Vancouver Public Library | BiblioCommons") >= 0)
System.out.println("the details page title is correct");
else
{
System.out.println("the details page title is incorrect, stop the application");
System.exit(0);
}
//check that the book title is displayed and not empty
WebElement bookTitle = driver.findElement(By.xpath("//h1[@id='item_bib_title']"));
if (bookTitle.isDisplayed() == true)
System.out.println("the book title is displayed");
else
{
System.out.println("the book title is not displayed, stop the application");
System.exit(0);
}
if (bookTitle.getText().length() > 0)
System.out.println("the book title is not empty");
else
{
System.out.println("the book title is empty, stop the application");
System.exit(0);
}
//check that the book author is displayed and not empty
WebElement bookAuthor = driver.findElement(By.xpath("//a[@testid='author_search']"));
if (bookAuthor.isDisplayed() == true)
System.out.println("the book author is displayed");
else
{
System.out.println("the book author is not displayed, stop the application");
System.exit(0);
}
if (bookAuthor.getText().length() > 0)
System.out.println("the book author is not empty");
else
{
System.out.println("the book author is empty, stop the application");
System.exit(0);
}
//close the browser
driver.quit();
}
//Test Case 2: browsing through results page works
@Test
public void testPaging()
{
WebDriver driver = new FirefoxDriver();
//open the home page and check that the home page title is correct
driver.get("http://www.vpl.ca");
if (driver.getTitle().equalsIgnoreCase("Vancouver Public Library - Home") == true)
System.out.println("the home page title is correct");
else
{
System.out.println("the home page title is incorrect, stop the application");
System.exit(0);
}
//search with a keyword, the results page is opened as result of the search
WebElement searchTextBox = driver.findElement(By.xpath("//input[@id='globalQuery']"));
searchTextBox.click();
searchTextBox.sendKeys("java");
WebElement searchButton = driver.findElement(By.xpath("//input[@class='search_button']"));
searchButton.click();
//check that the results page title is correct
if (driver.getTitle().equalsIgnoreCase("Search | Vancouver Public Library | BiblioCommons") == true)
System.out.println("the results page title is correct");
else
{
System.out.println("the results page title is incorrect, stop the application");
System.exit(0);
}
//check that the number of results is displayed and correct
WebElement numberResultPageOne = driver.findElement(By.xpath("//span[@class='items_showing_count']"));
if (numberResultPageOne.isDisplayed() == true)
System.out.println("The number of the results on the page 1 is displayed");
else
{
System.out.println("The number of the results on the page 1 is not displayed, stop the application");
System.exit(0);
}
if (numberResultPageOne.getText().indexOf("1 - 25") >=0)
System.out.println("The number of the results on the page 1 is correct");
else
{
System.out.println("The number of the results on the page 1 is not correct, stop the application");
System.exit(0);
}
//navigating to page 2 of results
WebElement pageTwo = driver.findElement(By.xpath("//a[@testid='link_page2']"));
pageTwo.click();
//check that the number of results is displayed and correct
WebElement numberResultPageTwo = driver.findElement(By.xpath("//span[@class='items_showing_count']"));
if (numberResultPageTwo.isDisplayed() == true)
System.out.println("The number of the results on the page 2 is displayed");
else
{
System.out.println("The number of the results on the page 2 is not displayed, stop the application");
System.exit(0);
}
if (numberResultPageTwo.getText().indexOf("26 - 50") >=0)
System.out.println("The number of the results on the page 2 is correct");
else
{
System.out.println("The number of the results on the page 2 is not correct, stop the application");
System.exit(0);
}
//close the browser
driver.quit();
}
}
The changes done so far are that
- the test class does not have a main method any longer
- the code of each test case is included in a separate unit test; each unit test uses the @Test annotations
- executing the test class will execute both unit tests; if one of the unit tests fails, the second unit test continues
- both unit tests have a similar structure:
- create the Selenium WebDriver browser driver object
- open the site
- implement the test case
- quit the browser driver object
It is because of this structure that both unit tests are independent.
The result of executing the unit tests is a green bar.
The unit tests are still logging information to the console panel about their verifications.
Executing the scripts shows a green execution bar meaning that all unit tests passed:
The green bar is displayed even if one of the unit tests has an error.
We need to use assertions so that the bar is red in case of an error.
2. USE ASSERTIONS FOR TEST CASE VERIFICATIONS
All verifications will be replaced next with assertions.
Assertions get a condition that needs to be verified.
The condition is evaluated.
If the condition's result is true, the assertion passes and the unit test continues.
If the condition fails, the assertion fails and the unit test fails as well.
Assertions do not log any information to the console.
An error message can be logged to the Failure panel.
The error message is optional.
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.firefox.FirefoxDriver;
public class LibraryTestCases {
//Test Case 1: book title and author are displayed and not empty
@Test
public void testResultDetails()
{
WebDriver driver = new FirefoxDriver();
//open the home page and check that the home page title is correct
driver.get("http://www.vpl.ca");
assertTrue("incorrect home page title",
driver.getTitle().equalsIgnoreCase("Vancouver Public Library - Home") == true);
//search with a keyword, the results page is opened as result of the search
WebElement searchTextBox = driver.findElement(By.xpath("//input[@id='globalQuery']"));
searchTextBox.click();
searchTextBox.sendKeys("java");
WebElement searchButton = driver.findElement(By.xpath("//input[@class='search_button']"));
searchButton.click();
//check that the results page title is correct
assertTrue("results page title incorrect",
driver.getTitle().equalsIgnoreCase("Search | Vancouver Public Library | BiblioCommons") == true);
//click on the first result, the details page is opened as a result
WebElement firstResult = driver.findElement(By.xpath("(//a[@testid='bib_link'])[1]"));
firstResult.click();
//check that the title of details page is correct
assertTrue ("the details page title is incorrect",
driver.getTitle().indexOf("Vancouver Public Library | BiblioCommons") >= 0);
//check that the book title is displayed and not empty
WebElement bookTitle = driver.findElement(By.xpath("//h1[@id='item_bib_title']"));
assertTrue ("the book title is not displayed", bookTitle.isDisplayed() == true);
assertTrue ("the book title is empty", bookTitle.getText().length() > 0);
//check that the book author is displayed and not empty
WebElement bookAuthor = driver.findElement(By.xpath("//a[@testid='author_search']"));
assertTrue ("the book author is not displayed", bookAuthor.isDisplayed() == true)
assertTrue ("the book author is empty", bookAuthor.getText().length() > 0);
//close the browser
driver.quit();
}
//Test Case 2: browsing through results pages works
@Test
public void testPaging()
{
WebDriver driver = new FirefoxDriver();
//open the home page and check that the home page title is correct
driver.get("http://www.vpl.ca");
assertTrue ("the home page title is incorrect",
driver.getTitle().equalsIgnoreCase("Vancouver Public Library - Home") == true);
//search with a keyword, the results page is opened as result of the search
WebElement searchTextBox = driver.findElement(By.xpath("//input[@id='globalQuery']"));
searchTextBox.click();
searchTextBox.sendKeys("java");
WebElement searchButton = driver.findElement(By.xpath("//input[@class='search_button']"));
searchButton.click();
//check that the results page title is correct
assertTrue ("results page title incorrect",
driver.getTitle().equalsIgnoreCase("Search | Vancouver Public Library | BiblioCommons") == true);
//check that the number of results is displayed and correct
WebElement numberResultPageOne = driver.findElement(By.xpath("//span[@class='items_showing_count']"));
assertTrue ("page 1 results number is not displayed",
numberResultPageOne.isDisplayed() == true);
assertTrue ("page 1 results number is incorrect",
pagenumberResultPageOne.getText().indexOf("1 - 25") >=0);
//navigating to page 2 of results
WebElement pageTwo = driver.findElement(By.xpath("//a[@testid='link_page2']"));
pageTwo.click();
//check that the number of results is displayed and correct
WebElement numberResultPageTwo = driver.findElement(By.xpath("//span[@class='items_showing_count']"));
assertTrue ("page 2 results number is not displayed",
numberResultPageTwo.isDisplayed() == true);
assertTrue ("page 2 results number is not correct",
numberResultPageTwo.getText().indexOf("26 - 50") >=0);
//close the browser
driver.quit();
}
}
Using assertions for test case verifications helps in the following ways:
- assertions do not log information to the console log
- the unit test continues to execute only if the assertion passes
- if the assertion fails, the unit test fails
- using assertions, the only thing that needs to be checked after executing the unit tests is if the progress bar is red or green
If the unit tests pass again, the green bar is displayed.
If one of the assertions fails, the unit test fails and we get a red bar:
The assertion error message is displayed in the Failure Trace:
3. USE TEST FIXTURES FOR PREPARING AND CLEANING THE TEST ENVIRONMENT
Each of the unit tests depends on the test environment to be set up before the unit test runs.
By setting up the test environment, I understand creating the browser driver object:
driver = new
FirefoxDriver();
After the unit test finishes execution, the test environment is cleaned up by closing the browser driver object:
driver.quit();
Currently, the browser driver object is created and closed in each of the test scripts.
We can use JUNIT fixtures for moving this code outside of the test scripts.
See below the refactored code:
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.firefox.FirefoxDriver;
public class LibraryTestCases {
WebDriver driver;
@Before
public void setUp()
{
driver = new FirefoxDriver();
}
@After
public void tearDown()
{
driver.quit();
}
//Test Case 1: book title and author are displayed and not empty
@Test
public void testResultDetails()
{
//open the home page and check that the home page title is correct
driver.get("http://www.vpl.ca");
assertTrue("the home page title is incorrect",
driver.getTitle().equalsIgnoreCase("Vancouver Public Library - Home") == true);
//search with a keyword, the results page is opened as result of the search
WebElement searchTextBox = driver.findElement(By.xpath("//input[@id='globalQuery']"));
searchTextBox.click();
searchTextBox.sendKeys("java");
WebElement searchButton = driver.findElement(By.xpath("//input[@class='search_button']"));
searchButton.click();
//check that the results page title is correct
assertTrue("the results page title is incorrect",
driver.getTitle().equalsIgnoreCase("Search | Vancouver Public Library | BiblioCommons") == true);
//click on the first result, the details page is opened as a result
WebElement firstResult = driver.findElement(By.xpath("(//a[@testid='bib_link'])[1]"));
firstResult.click();
//check that the title of details page is correct
assertTrue ("the details page title is incorrect",
driver.getTitle().indexOf("Vancouver Public Library | BiblioCommons") >= 0);
//check that the book title is displayed and not empty
WebElement bookTitle = driver.findElement(By.xpath("//h1[@id='item_bib_title']"));
assertTrue ("the book title is not displayed", bookTitle.isDisplayed() == true);
assertTrue ("the book title is empty", bookTitle.getText().length() > 0);
//check that the book author is displayed and not empty
WebElement bookAuthor = driver.findElement(By.xpath("//a[@testid='author_search']"));
assertTrue ("the book author is not displayed", bookAuthor.isDisplayed() == true)
assertTrue ("the book author is empty", bookAuthor.getText().length() > 0);
}
//Test Case 2: browsing through results pages works
@Test
public void testPaging()
{
//open the home page and check that the home page title is correct
driver.get("http://www.vpl.ca");
assertTrue ("the home page title is incorrect",
driver.getTitle().equalsIgnoreCase("Vancouver Public Library - Home") == true);
//search with a keyword, the results page is opened as result of the search
WebElement searchTextBox = driver.findElement(By.xpath("//input[@id='globalQuery']"));
searchTextBox.click();
searchTextBox.sendKeys("java");
WebElement searchButton = driver.findElement(By.xpath("//input[@class='search_button']"));
searchButton.click();
//check that the results page title is correct
assertTrue ("the results page title is incorrect",
driver.getTitle().equalsIgnoreCase("Search | Vancouver Public Library | BiblioCommons") == true);
//check that the number of results is displayed and correct
WebElement numberResultPageOne = driver.findElement(By.xpath("//span[@class='items_showing_count']"));
assertTrue ("page 1 results number is not displayed",
numberResultPageOne.isDisplayed() == true);
assertTrue ("page 1 results number is incorrect",
pagenumberResultPageOne.getText().indexOf("1 - 25") >=0);
//navigating to page 2 of results
WebElement pageTwo = driver.findElement(By.xpath("//a[@testid='link_page2']"));
pageTwo.click();
//check that the number of results is displayed and correct
WebElement numberResultPageTwo = driver.findElement(By.xpath("//span[@class='items_showing_count']"));
assertTrue ("page 2 results number is not displayed",
numberResultPageTwo.isDisplayed() == true);
assertTrue ("page 2 results number is not correct",
numberResultPageTwo.getText().indexOf("26 - 50") >=0);
}
}
A few changes have been done:
- the driver object is declared outside of the test scripts
- for the test environment set up, the setUp() method is created using the @Before annotation;
The browser driver object is created in this method.
Because of the @Before annotation, the setUp() method runs automatically before each unit test
- for the test environment clean up, the tearDown() method is created using the @After annotation;
The browser driver object is closed in this method.
Because of the @After annotation, the tearDown() method runs automatically after each unit test.
Using test fixtures, the execution order becomes:
- setUp()
- unit test()
- tearDown()
- setUp()
- unit test()
- tearDown()
If you liked this article, please share it with your friends.
