Experitest Blog

Selenium Webdriver Tutorial with JAVA: How to set up a TestNG maven based project from scratch.


 

What is TestNG Framework

 

TestNG is an automation testing framework. The NG-suffix abbreviation stands for “Next Generation”. The concept of TestNG is similar to the more familiar JUnit which uses the annotations (@).

 

TestNG Framework provides vast opportunities for test engineers for example:

  • Multiple executions of the same test are without looks are possible by using mechanism (annotation) called ‘invocation count.’
  • Failed tests can be re-run only by running the tests that are in TestNG-failed.xml file (test-output folder).
  • TestNG automatically handles uncaught exceptions without terminating the test prematurely. TestNG reports these exceptions as failed steps in the report.
  • Different tests are arranged into groups by placing them into the TestNG.xml file.
  • Cross browser testing allows the execution of multiple test cases on multiple browsers.
  • Reports are generated with respective skip, failed and passed percentages.
  • Annotations used in testing are self-explanatory ex: @BeforeMethod, @AfterMethod, @BeforeTest, @AfterTest (https://TestNG.org/doc/documentation-main.html#annotations).

 

In other words,

  • TestNG annotations are self-explanatory and easy to understand
  • Test cases can be organized in groups
  • Parallel testing is possible

 

How to build and set-up a project from scratch

 

To start, here is a video-snap for project creation from scratch in the IntelliJ IDEA based on java, maven, TestNG: https://take.ms/H8KxD  (video-snap).

cmd.exe

Important note: Before creating a project from scratch – please ensure that you have java and maven installed locally. ‘java -version’ and ‘mvn –version’ commands in the command line should provide exact versions installed.

IntelliJ Maven

Meanwhile, the maven-archetype-quickstart is supposed to be selected when creating a project from scratch.

IntelliJ

GroupId and ArtifactId should be given. Please see the format: https://stackoverflow.com/questions/3724415/maven-artifact-and-groupid-naming of the GroupId and ArtifactId.

 

Finalized Test Project Structure

 

This section provides a high-level overview of what the structure of test frameworks built over TestNG, Page-Object (using java and maven – as dependency manager for the project) should look like. Please see the video-snap: https://take.ms/L6ncY.

 

How to create tests using Page object-pattern

 

Pros of using the Page object:

 

The Page Object Pattern is a decoupling layer. To clarify, a Page object encapsulates web elements descriptions and methods operating on the described element. In contrary to the direct locators and web element references from a test scenario. Most importantly, the page object pattern approach makes a test developer’s life much easier especially when the HTML layout is liable to changes.

TestNG framework html test

Pic. 1. Schema of implementation of when a test script refers directly to web-elements locators. An example of bad test design: in case the HTML layout changes the test developer will have to look up changed (affected) web elements, and manually modify locators in different framework places.

TestNG framework html

Pic. 2. Schema of an implemented page object. In Selenium/Appium tests test developer refers to the instances of page objects. Every page object encapsulates elements and methods operating on the elements. In case a page changes the only modification to do is in Page Object, in a single place. Implemented tests remain unchanged.

 

Getting started with this implementation is actually easy and straightforward:

  1. Basic Selenium project setup (Java, Maven, Git).
  2. Perform analysis of the application under test
  3. Create page objects.
  4. Implement tests
  5. Organize implemented tests into XML test suites
  6. Set-up Selenium grid INFRA
  7. Set-up the CI/launch tests

Some examples of Page object are below including a Selenium test scenario. For demo purposes, we will open the Google landing page and do a simple search (query) there.

 

For the purposes of describing elements on the page – Google Chrome DevTools > Elements tab.

TestNG framework google

Pic.3. Using this tab and just entering the ‘Ctrl+F’ keys combo test gives developers the ability to isolate and find web elements covered and described in the Page Object pattern.

 

Firstly, let’s implement the base Page class that will be a parent for all the page entities. Click here for the full page object entities code source: https://GitHub.com/eugenepolschikov/TestNG-demo/blob/master/src/main/java/com/TestNGdemo/automation/pages/Page.java.

 

An important note here: We will also need this parent Page.java class for the purposes of initialization of page objects because page-object will not work unless we initialize the WebElements that we have annotated. So Page.java superclass holds this small but important piece of code.

 

Let’s move on to implementing a Page Object representing the Google landing page. Click here for the full code source: https://GitHub.com/eugenepolschikov/TestNG-demo/blob/master/src/main/java/com/TestNGdemo/automation/pages/GoogleLandingPage.java

 

After that, let’s consider that we’ve implemented a page object, and proceed with creating a test scenario.

 

Simple test scenario(-s)

 

For simplicity and demo purposes let’s consider the following use case (test scenario):

 

A user opens a Google landing search page, then the user enters ‘Experitest’ and ‘Appium’ in search input, and after that analyzes results.

 

First, we’re going to create a parent test class that will contain common stuff for test classes. Click here for the full code source: https://GitHub.com/eugenepolschikov/TestNG-demo/blob/master/src/test/java/com/TestNGdemo/automation/functional/TestSuitesBase.java

 

And test scenario implemented respectively. Click here for the full code source: https://GitHub.com/eugenepolschikov/testng-demo/blob/master/src/test/java/com/testngdemo/automation/functional/GoogleLandingPageFirstTest.java

 

As we can see, the steps implemented in the scenario are pretty self-explanatory. As a result, this implementation helps us keep tests more flexible to adjustments and more flexible in terms of maintenance.

 

Please also note how ‘Group’ annotations are implemented and the way how some specific test is assigned to the test group:

@Test(groups = { “REGRESSION.SUITE” } )

 

If needed, we may apply multiple ‘Group’ annotations, e.g:

@Test(groups = { “API.SUITE”, ” SMOKE.SUITE”, “MYGROUP.SUITE”, “WHATEVERGROUP.SUITE”})

 

Listeners

 

In TestNG framework, everything executes via the Suite. Subsequently, Test and Methods and the Listeners give us the ability to add specific action(-s) before and after of every Suite, Test, and Methods.

 

In technical terms, TestNG Listeners are interfaces that allow you to modify the TestNG framework’s behavior. Here are a few listeners:

  • IAnnotationTransformer
  • IAnnotationTransformer2
  • IHookable
  • IInvokedMethodListener
  • IMethodInterceptor
  • IReporter
  • ISuiteListener
  • ITestListener

For more details please see the TestNG spec https://TestNG.org/doc/documentation-main.html#TestNG-listeners.

 

How to rerun a failed test. Retry mechanism (implemented as Listener)

 

In the case of a test failure, test developers are often eager to ensure that the reason for the failure is a functional bug but not connection lag, non-stable environment or any kind of temporary abruption.

 

For these purposes, there is a great mechanism called the re-try mechanism. Simply speaking it is a mechanism that reruns the same test in case of the failure X-times and only after X-number of failures is the test considered as failed.

 

First, we need to implement the IRetryAnalyzer interface. Click here for the full code source: https://GitHub.com/eugenepolschikov/TestNG-demo/blob/master/src/test/java/com/TestNGdemo/automation/listeners/retrymechanism/Retry.java

 

At this point, we’re ready to implement the listener’s logic itself. The full source is here: https://GitHub.com/eugenepolschikov/TestNG-demo/blob/master/src/test/java/com/TestNGdemo/automation/listeners/retrymechanism/RetryListener.java.

 

How to invoke Listener

 

There are two simple options we may invoke listeners in: either in TestNG.xml or in test classes. Let us compare both options.

 

Listener in TestNG.xml

 

Click here for the full code source: https://GitHub.com/eugenepolschikov/TestNG-demo/blob/master/src/test/java/com/TestNGdemo/automation/TestNG_Configs/ParallelTestsSuiteWithListeners.xml

 

The full source for a listener in Test class (in java code): https://GitHub.com/eugenepolschikov/TestNG-demo/blob/master/src/test/java/com/TestNGdemo/automation/functional/GoogleLandingPageSecondTest.java

 

Given that listeners are added in test class there is no need to add them in the TestNG.xml test suite now.

 

Multi-thread (parallel) vs single-thread execution (different options in TestNG XML test suites). Examples of TestNG XML suites.

TestNG.xml suites contain a huge amount of parameters. Let’s consider the parallel parameter and the values it may take. In terms of TestNG and the annotations we have in TestNG.xml – we may do the parallel execution for tests, classes, and methods. The parallel attribute on the tag can take one of the following values:

<suite name=”Demo test suite” parallel=”methods” thread-count=”10″>

<suite name=”Demo test suite” parallel=”tests” thread-count=”10″>

<suite name=”Demo test suite” parallel=”classes” thread-count=”10″>

<suite name=”Demo test suite” parallel=”instances” thread-count=”10″>

 

Please have a look at the full code example here: https://GitHub.com/eugenepolschikov/TestNG-demo/blob/master/src/test/java/com/TestNGdemo/automation/TestNG_Configs/ParallelTestsSuiteWithListeners.xml

 

Here we have 2 tests, where 1 test represents sub-set of all test methods annotated with “SMOKE.SUITE”. Considering suite config and parameters are <suite name=”Parallel testNG tests suite for demo purposes” parallel=”tests” thread-count=”2″ verbose=”8″> that means that we have created 2 browser instances simultaneously. Use one browser instance for the first test (where the sub-set of all test methods annotated with “SMOKE.SUITE” is to be executed) and another browser instance – for the second test (where sub-set of all test methods annotated with “REGRESSION.SUITE” is to be executed).

 

To Summarize, before you start implementing the framework architecture, please understand whether you need to have tests executed in a parallel way and what kind of parallel testing you actually need. The TestNG framework gives us a solution for the parallel test implementation.

 

Sometimes it is crucial to have everything executed in a consecutive way (i.e in 1 thread). That may be accomplished by an easy flag value change to parallel=”false” .

 

<!DOCTYPE suite SYSTEM “http://testng.org/testng-1.0.dtd”>

<suite name=”Parallel testNG tests suite for demo purposes” parallel=”false” verbose=”8″>

…..

 

Appropriate Selenium GRID config for parallel execution support

TestNG framework selenium

Selenium GRID representation: Selenium HUB and Selenium nodes communicate with each other. Considering we have 2 parallel threads for execution in this demo project, we will need a hub and 2 nodes. Also, 1 thread will be executed in a Chrome browser instance and another thread – in a Firefox browser instance.

 

 

Click here to see the source code for JSON Selenium GRID config: https://GitHub.com/eugenepolschikov/TestNG-demo/tree/master/grid_json_config. Once this has all been prepared, the start of the grid is pretty straightforward. As we are working on a Windows machine I will provide Windows-specific commands for launching the grid:

 

  1. java -jar Selenium-server-standalone-3.11.0.jar -role hub -hubConfig hub.json
  2. java -jar -Dwebdriver.gecko.driver=geckodriver.exe -Dwebdriver.chrome.driver=chromedriver.exe Selenium-server-standalone-3.11.0.jar -role node -nodeConfig node1.json
  3. java -jar -Dwebdriver.gecko.driver=geckodriver.exe -Dwebdriver.chrome.driver=chromedriver.exe Selenium-server-standalone-3.11.0.jar -role node -nodeConfig node2.json

 

To ensure that everything has started correctly – please open: http://localhost:4444/grid/console:

selenium grid console

Maven configuration for test execution

 

A suite might execute via the following command: mvn clean test -DTestNG.suite.name=ParallelTestsSuiteWithListeners

 

Please note: this should be called from the project root. Alternatively, we may create a Maven run configuration in the IDEA:

TestNG framework rundebug

Take a look at the video-snap of execution and tests results analysis (of Allure HTML report): https://take.ms/WcbnC.

 

Experitest. Digital assurance lab for test execution and its benefits

 

Experitest is the world’s leading provider of quality assurance tools for mobile DevOps, including manual testing, performance testing, load testing and monitoring for mobile applications. Experitest tools support all mobile OS, iOS, Android, Windows Phone and Blackberry.

 

I got acquainted with Experitest products and one of my favorites is the Digital assurance lab. It allows access to hundreds of browsers & mobile devices (physical/emulated) Hosted in Experitest data centers, from anywhere at any time. There is a large pool of browsers, physical mobile devices & tablets, simulators and emulators available for selection. The lab includes a variety of manufacturers, models & OS versions, including newly released device models and OS beta versions.

 

I’ve managed to integrate it into my QA test automation environment with almost no effort (just following the manual provided). Another great benefit worth mentioning here is the possibility to run tests in parallel mode. Given that most projects demand a high percentage of functionality coverage and when test developers have hundreds and thousands of tests to execute – parallel execution is indeed very helpful and less time-consuming.

 

And also I like the result analytics dashboard – a cool feature of Digital assurance lab, providing test results and prettified HTML representation of the executed tests in a very detailed way (tests suites, groups, timelines, percentages, tests steps, etc).

 

Most importantly, as test developer that makes test debugging effortless.

 

Please see more details on the parallel browser and mobile execution: https://experitest.com/mobile-test-automation/grid-execution-mobile/ and https://experitest.com/mobile-cloud-testing/seetest-reporter/?am_force_theme_layout=desktop.

SeeTest

Reporting

 

For reporting purposes in the demo project, we will use Allure reports. To use Allure – we need to add a couple of dependencies in pom.xml (https://GitHub.com/eugenepolschikov/TestNG-demo/blob/master/pom.xml)

 

As soon as we added Allure dependencies we are eligible to use Allure annotations in the test classes: https://GitHub.com/eugenepolschikov/TestNG-demo/blob/master/src/test/java/com/TestNGdemo/automation/functional/GoogleLandingPageFirstTest.java.

 

Please consider that the Allure reporter also has a large number of settings, that are specified in the public spec: http://Allure.qatools.ru/.

 

Once all of the tests are executed, all of the results including the .xml file and screenshots (if any) are usually placed in the following folder: TestNGexample\target\Allure-results. So to generate a clear and well laid out HTML report based on the resulting XML, test developers should open the TestNGexample/target folder and execute the Allure serve command via the command-line Allure tool (https://docs.qameta.io/Allure/#_installing_a_commandline) :

 

Demo Code source

 

The Complete test projects used for demo purposes are placed in my GitHub repository: https://GitHub.com/eugenepolschikov/TestNG-demo. All are verified and working.

 

Summary

 

In short, this article provides how-to instructions for some of the cool features of the TestNG framework. These include organizing tests into separate groups, organizing tests and setting them up for parallel execution, simple steps how to create a test project from scratch based on Java and Maven as dependency manager and also introduces the Page Object pattern – one of best practices that are widely used in QA world nowadays.

 

Eugene PolschikovQA automation expert

 

Comments are closed.