October 26, 2020 Jonny Steiner

TestNG vs JUnit – Which is the Right Choice for your Enterprise

Share this knowledge!

Software test automation is not a single entity but encompasses an array of tasks performed using multiple tools and frameworks. There are different types of testing such as functional testing, non-functional testing, UI testing, and accessibility testing. Each testing type is again divided into different levels such as unit testing, end-to-end testing, and integration testing. Based on the target application, you can categorize them into mobile testing, web testing, and hybrid app testing. In addition, you need to generate reports and analyze test reports. JUnit and TestNG are two important test automation frameworks widely used by developers. As both frameworks mostly offer similar features, the TestNG vs JUnit debate has been going on for a while now.

Here is a comparison of the features and functionality of both tools:

An Overview

An Overview of JUnit

JUnit is a test automation framework written in Java that is specifically designed to run unit tests. With a motto of ‘test first before code’, JUnit enables developers to perform unit testing on code units so that they can test first and then start building the application. JUnit is an open-source framework that is helpful for test-driven development. It resides in the org.junit package and linked to the code at the runtime as a JAR file. In versions earlier than 3.8, it resided in the junit.framework. Annotations are an important feature of this tool. With annotations, the syntax becomes simple, elegant, and easily understandable. By using assertions, you can check if the test results generated as desired. JUnit 5 is the latest version.

Here is the link to the JUnit website: https://junit.org/junit5/

An Overview of TestNG vs JUnit

TestNG is a next-generation test automation framework for Java. Cedric Beust wrote it, taking inspiration from JUnit. However, the tool extends the JUnit functionality to support a wide range of testing methods such as Unit testing, integration testing, and end-to-end testing. The tool has quickly gained momentum as it offers powerful features such as Annotations, parameterization, and Data providers for Data-driven testing, flexible test execution, concurrent testing, and distributed testing. It comes out-of-the-box for popular IDEs such as Eclipse, NetBeans, and IntelliJ via plugins. With test reports generated in the HTML format, it allows you to easily analyze test cases.

The History

JUnit

Unit testing framework JUnit is a part of the family of unit testing frameworks collectively known as xUnit. All xUnit frameworks derive the functionality and structure from sUnit. The architecture includes Test Runner, test cases, test suites, fixtures, result formatter, assertions test executions. Kent beck designed xUnit in 1998 for the Smalltalk programming language. It allowed developers to create tests and run them using Smalltalk language. As xUnit heavily implements the object-oriented programming style, it also suited well for C# and Java. Today, this framework is widely used with most programming languages. xUnit frameworks follow a similar naming convention wherein the first letter replaces the programming language’s first letter. For instance, JUnit is for Java, NUnit is for .NET.

JUnit was developed by Kent Beck, Eric Gamma, Kris Vasudevan, and David Saff and was released in 2002. Within 10 years, it became the leading Java library with 30.7% of Java projects implementing JUnit, as per OverOps.

TestNG

Inspired by JUnit, Cedric Beust created the TestNG test automation framework in 2004 for Java. The name TestNG stands for Test Next Generation. This tool extends the testing functionality of JUnit to cover a wide range of test methods while resolving some of its deficiencies with multi-thread access, reuse of the same JVM, and objects accessing private/shared content.

Here is the website link: https://testng.org/

Installation

JUnit Installation

The installation of JUnit is simple and easy.

  1. JUnit 5 needs Java JDK 8.0 or higher. Download it from this link: https://www.oracle.com/in/java/technologies/javase-downloads.html
  2. A Java IDE (eg: Eclipse) is required. You can download Eclipse from this link: https://www.eclipse.org/downloads/
  3. Visit this link to download JUnit 4: https://junit.org/junit4/
  4. Download JUnit jar files (junit.jar and hamcrest-core.jar) and save them on your machine.
  5. Configure environment variables – Add jar files to the Classpath.
  6. Open your project and go to Build path configuration. Click on ‘Add external jars’ and add the downloaded jar files.

For JUnit 5, visit this link: https://junit.org/junit5/

TestNG Installation

  1. TestNG needs Java JDK 8.0 or higher. Download it from this link: https://www.oracle.com/in/java/technologies/javase-downloads.html
  2. A Java IDE (eg: Eclipse) is required. You can download Eclipse from this link: https://www.eclipse.org/downloads/
  3. You can download TestNG files from this link: https://dl.bintray.com/testng-team/testng-eclipse-release/
  4. Open Eclipse and go to Help -> Install New Software and provide the TestNG link and Eclipse will automatically download and add TestNG to Eclipse.

Features

When it comes to TestNG vs JUnit features, both tools offer more or less similar features, TestNG extends JUnit functionality to offer powerful features. Here is a comparison of the features of the two tools:

Annotations

Annotations are an important feature offered by both tools. The functionality is almost similar in both tools with minor differences in syntax.

TestNG vs JUnit

Here is a comparison list of annotations that show syntax differences:

table

Here is an example of annotations in the code:

import org.openqa.selenium.*;
import org.openqa.selenium.remote.BrowserType;
import org.openqa.selenium.remote.CapabilityType;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.openqa.selenium.support.ui.ExpectedConditions;
import org.openqa.selenium.support.ui.WebDriverWait;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.BeforeMethod;
import org.testng.annotations.Test;
import java.net.URL;

public class exampleScript {
    private static final String ACCESS_KEY = "Enter your key here";
    private RemoteWebDriver driver;
    private URL url;
    private DesiredCapabilities dc = new DesiredCapabilities();

   @BeforeMethod
    public void setUp() throws Exception {
        url = new URL("https://cloud.seetest.io/wd/hub");
        dc.setCapability(CapabilityType.BROWSER_NAME, BrowserType.CHROME);
        dc.setCapability(CapabilityType.PLATFORM, Platform.ANY);
        dc.setCapability("accessKey", ACCESS_KEY);
        dc.setCapability("testName", "Quick Start Chrome Browser Demo");
        driver = new RemoteWebDriver(url, dc);
    }

    @Test
    public void browserTestGoogleSearch() {
        driver.get("https://www.google.com");
        new WebDriverWait(driver, 10).until(ExpectedConditions.presenceOfElementLocated(By.name("q")));
        WebElement searchBar = driver.findElement(By.name("q"));
        searchBar.click();
        searchBar.sendKeys("Experitest");
        searchBar.sendKeys(Keys.ENTER);
    }

    @AfterMethod
    public void tearDown() {
        System.out.println("Report URL: "+ driver.getCapabilities().getCapability("reportUrl"));
        driver.quit();
    }
}

This program opens a Chrome browser from the Experitest cloud and opens a Google search website and searches for the keyword ‘Experitest’. Before executing this code, it will execute the @BeforeMethod code that assigns values to the RemoteWebDriver, site URL, and desired capabilities. Similarly, after the test code is executed, it will execute the code mentioned under @AfterMethod that involves generating a report and closing the browser.

testng vs junit

The same test script can be run using JUnit. The import option and the annotations will change.

 

package ExperitestJUnit;

import org.openqa.selenium.*;
import org.openqa.selenium.remote.BrowserType;
import org.openqa.selenium.remote.CapabilityType;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.openqa.selenium.support.ui.ExpectedConditions;
import org.openqa.selenium.support.ui.WebDriverWait;
import org.junit.*;
import java.net.URL;

public class ExperitestSearch1 {
	   private static final String ACCESS_KEY = "Enter your access key here";
		private RemoteWebDriver driver;
		private URL url;
		private DesiredCapabilities dc = new DesiredCapabilities();

@Before
 public void setUp() throws Exception {
       url = new URL("https://cloud.seetest.io/wd/hub");
 dc.setCapability(CapabilityType.BROWSER_NAME, BrowserType.CHROME);
	     dc.setCapability(CapabilityType.PLATFORM_NAME, Platform.ANY);
		    dc.setCapability("accessKey", ACCESS_KEY);
			  dc.setCapability("testName", "Quick Start Chrome Browser Demo");
			         driver = new RemoteWebDriver(url, dc);
			     }

     @Test
 public void browserTestGoogleSearch() {
	    for (int i = 0; I<7; i++) {
	 driver.get("https://www.google.com");
new WebDriverWait(driver, 10).until(ExpectedConditions.presenceOfElementLocated(By.name("q")));
	WebElement searchBar = driver.findElement(By.name("q"));
	searchBar.click();
	 searchBar.sendKeys("Experitest");
	 searchBar.sendKeys(Keys.ENTER);
		 }
			}
    @After
	public void tearDown() {
 System.out.println("Report URL: "+ driver.getCapabilities().getCapability("reportUrl"));
			         driver.quit();
 }
}

testng vs junit

Running Test Suites

A test suite comprises two or more test cases. When it comes to running test cases simultaneously, TestNG offers more flexibility. You can simply add the test cases in the XML file and directly run the XML file. TestNG executes tests in the order mentioned in the XML file. To change the order, you’ll have to set the attribute preserve-order=’false’.

Example script:

<suite name=”Experitest Suite1”>
    <test name=”Experitest1”>
        <classes>
           <class name=”ExperitestProject.Experitestsearch1″ />
            <class name=”ExperitestProject.Experitestsearch2″ />
        </classes>
    </test>

Running test suites was not available in JUnit earlier. However, they have introduced it later. The new JUnit 5 comes with this feature. However, you need to mention it in the code.

Example:

 

@RunWith(Suite.class)
 @Suite.SuiteClasses({
   Experitest1.class,
   Experitest2.class
   Experitest3.class
})

public class JunitTest5 {
   //code
}
</suite>

Parallel Execution

Parallel execution is about running test cases simultaneously instead of running them in sequence. TestNG offers an easy way to run parallel execution. You can simply select the option and choose whether to run parallel execution based on methods, classes, or tests. You can also specify the thread count. When you run the XML file, it will automatically execute the tests in parallel.

testng vs junit

However, JUnit doesn’t support parallel testing. So, TestNG gets an edge here in the TestNG vs JUnit battle.

Parameterization

Parameterization is an important feature that significantly benefits data-driven testing. It automatically feeds different values to the code every time it is executed. As such, it helps you to reduce code length and complexities. Both tools offer the parameterization feature.

JUnit

JUnit uses the following syntax:

import org.junit.Test;
import org.junit.runners.Parameterized;
import org.junit.runner.RunWith;

@RunWith(Parameterized.class)
public class ExperitestParameters{ 
     @Parameters
    public static Collection<Object[]> data() {
           /*provide the parameter list here */
    }

The ability to provide different combinations of data between arguments is one feature of JUnit that is not available in TestNG.

TestNG

In TestNG, using parameterization is pretty straightforward. You can simply provide parameter values in the code.

import org.junit.Test;
import org.junit.runners.Parameterized;
import org.junit.runner.RunWith;


@RunWith(Parameterized.class)
public class ExperitestParameters{ 
     @Parameters
  public static Collection<Object[]> data() {
           /*provide the parameter list here */
    }

However, you cannot run this code directly. You need to run the TestNG XML file.

Data Providers

Data Provides is a powerful feature in TestNG that comes built-in with the framework. It extends the functionality of parameterization. With the data providers feature, you can directly run the test script instead of running the XML file.

The syntax of data providers in TestNG:

@DataProvider (name = “name_of_dataprovider”)
public Object[][] dpMethod() {
return new Object [][] { values}
}

When it comes to Data providers, JUnit doesn’t support this feature. However, a few enthusiast developers have written a TestNG-style Data Providers function for Junit and uploaded it to Github. You can download it from this Github link:

https://github.com/TNG/junit-dataprovider

Reporting

For test automation projects, reports are a key requirement as they provide the test results to help you understand how each test is executed, where it failed, and why it failed. similarly, logs tell you about the test execution flow. The reporting feature comes in-built in TestNG. it automatically generates reports in HTML and XML format. In addition, you can create custom reports and logs. Moreover, you can easily share these reports with your team. However, JUnit does not generate reports by default. You need to install a 3rd party tool to generate reports. The test data resides in XML format, though.

Support

JUnit is older when compared with TestNG. As such, JUnit enjoys a larger user base. It has an active community that is always ready to help. Adequate documentation and user guides are also available which makes help for any tool-related issues available. TestNG has a smaller user base compared to JUnit. However, the community is active and offers good support. documentation and user guides are available as well.

Summary

The TestNG Vs JUnit debate has been going on for a long time in development circles. So, which is the right choice for your Enterprise?

JUnit and TestNG are two popular test automation frameworks that are widely used by developers and QA teams. Both tools offer similar features. However, TestNG is a more advanced framework as it offers additional features when compared with JUnit.

Here are some areas wherein TestNG has an edge over JUnit:

  • Parameterization in TestNG is easy to configure and requires less time and effort.
  • Parallel Execution is not available with JUnit.
  • TestNG offers special annotations such as @BeforeGroups, @AfterGroups, @BeforeSuite, @AfterSuite etc.
  • JUnit doesn’t support HTML reports.
  • JUnit doesn’t support dependency tests based on Methods or Groups while TestNG does. (eg: dependsOnGroups, dependsOnMethods).
  • Executing tests in a specific order is possible with TestNG.
  • JUnit comes in multiple modules while TestNG comes in a single module. As such, TestNG is easy to use.
  • TestNG is suitable to run Test Suites.

When you want to inject test methods input data, Junit fares well. Similarly, it supports assumptions which means you can skip tests by adding a condition. You can enable or disable tests easily. TestNG offers limited functionality in this segment.

For enterprises that deal with large projects, TestNG is a good choice as it allows you to run multiple test cases in parallel to save time and effort. It is easy to install and use. For projects that heavily deal with logic, JUnit serves a good purpose as it allows you to skip, enable, and disable tests based on specific conditions.

JUnit 5 is quickly evolving. So, the missing features of JUnit get added to the framework. So, the TestNG vs JUnit debate is not going to end any sooner.

Guy ArieliCTO

Share this knowledge!