What is a test automation framework?
The more software systems increase in complexity as new development practices emerge, the more the quality assurance and software testing requirements rise. Now, among a number of testing methods, automated testing is the most common candidate to enhance your quality assurance process. At first glance, automated testing may look like a very difficult and resource consuming choice. But, diving into the matter deeper, you may find out that there are a number of established practices, rules, tools and libraries that significantly simplify the process of designing your test cases – test automation frameworks. Basically, test automation frameworks are tools for empowering and simplifying automated testing.
Test automation frameworks provide a number of benefits: lower expenses, lower entry threshold for testers, more readable and reusable code.
However, there are a lot of test automation frameworks available. In this article I’ll cover the most popular ones, that have rightfully become test automation industry standards:
Each of these testing frameworks provides its own benefits and applies its own limitation to the test automation process. The differences include supported programming languages, test case organization, a level of applicability. I will describe the benefits and limitations of each framework in details below. The goal of the article is to make it clearer which tests automation framework will meet your needs.
JUnit is a test automation framework for the Java programming language. As one of the oldest, yet constantly evolving frameworks it holds a leading position in the world of automated testing. As it utilizes Java, it has one of the biggest benefits – cross-platform support.
In JUnit, test cases are represented by Java objects, where a simple annotation system manages the execution of test methods. Like in this code.
Here, the methods are executed in the following order:
@BeforeClass methods will be executed once before all the methods, annotated as @Test
@Before methods will be executed before each of the methods, annotated as @Test
@After methods will be executed after each of the methods, annotated as @Test
@AfterClass methods will be executed once after all the methods, annotated as @Test
Aside from annotations, that manage the order of test methods executions, there are annotations to parametrize tests and set up custom rules for additional flexibility.
Basically, JUnit will suit you in the following situations:
If you rely on a cross-platform solution.
If your testers have experience in Java.
NUnit is a port of JUnit to C# language. So, if you want a .NET solution, then you should consider it as one of the most viable choices. Basically, as a port, it allows you to write JUnit-style automated tests in C#, using similar techniques, such as annotations.
As you can see, NUnit is very similar to JUnit, it follows the same test design philosophy, utilizes a similar annotation mechanism. So, if you need a testing framework to organize automated testing in C#, then NUnit is the best choice.
TestNG is, again, a new approach to JUnit. Like JUnit, TestNG utilizes Java programming language, inheriting all the benefits of Java cross-platform support.
What is the differences between TestNG and JUnit? Well… TestNG is striving to be a more configurable framework, allowing some additional grouping features as well as more hook annotations. For example, in TestNG, you can group your test classes to Groups and then group Groups to Suites. Besides, it supports annotations such as @BeforeGroup/@AfterGroup, @BeforeSuite/@AfterSuite, and others.
Choosing between JUnit and TestNG is not an easy task, so, usually, it is a matter of personal preference.
Compared to all the previous examples, Cucumber is a different beast. Following aspects of the behavior-driven development paradigm, Cucumber describes high-level test scenarios in the Gherkin notation, very similar to a human language. For an example click here.
a href=”https://github.com/jonnysteiner/How-to-Select-the-Right-Test-Automation-Framework-for-you/blob/master/step%20definition%20java” target=”_blank” rel=”noopener noreferrer”>An example of step definition in Java:
Besides that, you can use Cucumber in conjunction with JUnit, TestNG or NUnit framework to enhance your test cases with additional features.
So, you may consider Cucumber if:
You are more interested in high-level test scenarios.
You have a team with different programming experience, so you can distribute tasks of writing Gherkin test cases and implementation.
However, you should be aware that Cucumber is not designed for more low-level tests such as unit tests and API tests.
All of the test automation framework from the examples above integrate with the SeeTest Platform in a simple manner. No matter what testing framework you have chosen, you can use Selenium and/or Appium drivers and specify access key and URL capabilities. Here is an example for Java programming language.
In order to make this code work, you should add a new system environment variable “SEETEST_IO_ACCESS_KEY” and set its value to your SeeTest access key. Then you can refer to the driver variable to locate your device. This approach works with all the frameworks above, only programming languages may differ. When you run your tests, the driver will connect to the SeeTest cloud and execute your test.
So, to sum the things up:
If your preference is a Java programming language, then JUnit is a great choice for you. The main advantage of JUnit is its popularity.
If you are looking for a solution to work with the .NET framework and C# programming language, then Nunit is an obvious choice.
TestNG, being very similar to JUnit provides some extra features to organize your tests. It is slightly less widespread than JUnit, though.
Cucumber supports a great number of programming languages and offers a kind of unique ability to have test cases be described in Gherkin language and steps defined on the programming language of your choice. However, it seems to be not very convenient to use it for testing anything on the lower level than business/functional
Currently, there are enough test automation frameworks available for use. Some of them are very different from the others, some are similar. The main benefit of adopting one or another framework is that it will allow you to keep your tests uniformed, standardized and accessible.scenarios.