1. Overview
In this article, we’ll introduce the TestNG testing framework.
We’ll focus on: framework setup, writing simple test case and configuration, test execution, test reports generation, and concurrent test execution.
2. Setup
Let’s start by adding the Maven dependency in our pom.xml file:
<dependency>
<groupId>org.testng</groupId>
<artifactId>testng</artifactId>
<version>7.1.0</version>
<scope>test</scope>
</dependency>
The latest version can be found in the Maven repository.
When using Eclipse, the TestNG plugin may be downloaded and installed from the Eclipse Marketplace.
3. Writing a Test Case
To write a test using TestNG, we just need to annotate the test method with org.testng.annotations.Test annotation:
@Test
public void givenNumber_whenEven_thenTrue() {
assertTrue(number % 2 == 0);
}
4. Test Configurations
While writing test cases, often we need to execute some configuration or initialization instructions before test executions, and also some cleanup after completion of tests. TestNG provides a number of initialization and clean-up features at method, class, group and suite levels:
@BeforeClass
public void setup() {
number = 12;
}
@AfterClass
public void tearDown() {
number = 0;
}
The setup() method annotated with @BeforeClass annotations will be invoked before execution of any methods of that test class, and tearDown() after execution all methods of the test class.
Similarly, we can use the @BeforeMethod, @AfterMethod, @Before/AfterGroup, @Before/AfterTest and @Before/AfterSuite annotations for any configuration at method, group, test and suite levels.
5. Test Execution
We can run the test cases with Maven’s “test” command, it will execute all the test cases annotated with @Test putting them to a default test suite. We can also run test cases from the TestNG test suite XML files, by using the maven-surefire-plugin:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.22.2</version>
<configuration>
<suiteXmlFiles>
<suiteXmlFile>
src\test\resources\test_suite.xml
</suiteXmlFile>
</suiteXmlFiles>
</configuration>
</plugin>
Note that if we have multiple XML files, covering all test cases, we can add all of them in the suiteXmlFiles tag:
<suiteXmlFiles>
<suiteXmlFile>
src/test/resources/parametrized_test.xml
</suiteXmlFile>
<suiteXmlFile>
src/test/resources/registration_test.xml
</suiteXmlFile>
</suiteXmlFiles>
In order to run the test standalone, we need to have the TestNG library in the classpath and the compiled test class along with the XML configuration file:
java org.testng.TestNG test_suite.xml
6. Grouping Tests
Tests can be run in groups, for example out of 50 test cases 15 can be grouped together and executed leaving others as is.
In TestNG grouping tests in suites are done using XML file:
<suite name="suite">
<test name="test suite">
<classes>
<class name="com.baeldung.RegistrationTest" />
<class name="com.baeldung.SignInTest" />
</classes>
</test>
</suite>
Notice that, both the test classes RegistrationTest, SignInTest now belongs to the same suite and once suite is executed, test cases in this class will get executed.
Apart from test suites, we can also create test groups in TestNG, where instead of test classes methods are grouped together. In order to do that, add the groups parameter in the @Test annotation:
@Test(groups = "regression")
public void givenNegativeNumber_sumLessthanZero_thenCorrect() {
int sum = numbers.stream().reduce(0, Integer::sum);
assertTrue(sum < 0);
}
Let’s use an XML to execute the groups:
<test name="test groups">
<groups>
<run>
<include name="regression" />
</run>
</groups>
<classes>
<class
name="com.baeldung.SummationServiceTest" />
</classes>
</test>
This will execute the test method tagged with group regression, in the SummationServiceTest class.
7. Parameterized Tests
Parameterized unit tests are used for testing the same code under several conditions. With the help of parameterized unit tests, we can set up a test method that obtains data from some data source. The main idea is to make the unit test method reusable and to test with a different set of inputs.
In TestNG, we can parametrize tests using @Parameter or @DataProvider annotation. While using the XML file annotate the test method with @Parameter:
@Test
@Parameters({"value", "isEven"})
public void
givenNumberFromXML_ifEvenCheckOK_thenCorrect(int value, boolean isEven) {
assertEquals(isEven, value % 2 == 0);
}
And provide the data using XML file:
<suite name="My test suite">
<test name="numbersXML">
<parameter name="value" value="1"/>
<parameter name="isEven" value="false"/>
<classes>
<class name="baeldung.com.ParametrizedTests"/>
</classes>
</test>
</suite>
Using data from XML file is useful, but we often need more complex data. @DataProvider annotation is used to handle these scenarios, which can be used to map complex parameter types for testing methods.@DataProvider for primitive data types:
@DataProvider(name = "numbers")
public static Object[][] evenNumbers() {
return new Object[][]{{1, false}, {2, true}, {4, true}};
}
@Test(dataProvider = "numbers")
public void
givenNumberFromDataProvider_ifEvenCheckOK_thenCorrect(Integer number, boolean expected) {
assertEquals(expected, number % 2 == 0);
}
@DataProviderfor objects:
@Test(dataProvider = "numbersObject")
public void
givenNumberObjectFromDataProvider_ifEvenCheckOK_thenCorrect(EvenNumber number) {
assertEquals(number.isEven(), number.getValue() % 2 == 0);
}
@DataProvider(name = "numbersObject")
public Object[][] parameterProvider() {
return new Object[][]{{new EvenNumber(1, false)},
{new EvenNumber(2, true)}, {new EvenNumber(4, true)}};
}
Using this, any object that has to be tested can be created and used in the test. This is mostly useful for integration test cases.
8. Ignoring Test Cases
We sometimes want not to execute a certain test case, temporarily during the development process. This can be done adding enabled=false, in the @Test annotation:
@Test(enabled=false)
public void givenNumbers_sumEquals_thenCorrect() {
int sum = numbers.stream.reduce(0, Integer::sum);
assertEquals(6, sum);
}
9. Dependent Tests
Let’s consider a scenario, where if the initial test case fails, all subsequent test cases should be executed, and rather be marked as skipped. TestNG provides this feature with the dependsOnMethods parameter of the @Test annotation:
@Test
public void givenEmail_ifValid_thenTrue() {
boolean valid = email.contains("@");
assertEquals(valid, true);
}
@Test(dependsOnMethods = {"givenEmail_ifValid_thenTrue"})
public void givenValidEmail_whenLoggedIn_thenTrue() {
LOGGER.info("Email {} valid >> logging in", email);
}
Notice that, the login test case depends on the email validation test case. Thus, if email validation fails the login test will be skipped.
10. Concurrent Test Execution
TestNG allows tests to run in parallel or in multi-threaded mode, thus providing a way to test these multi-threaded pieces of code.
You can configure, for methods, classes, and suites to run in their own threads reducing the total execution time.
10.1. Classes and Methods in Parallel
To run test classes in parallel, mention the parallel attribute in the suite tag in XML configuration file, with value classes:
<suite name="suite" parallel="classes" thread-count="2">
<test name="test suite">
<classes>
<class name="baeldung.com.RegistrationTest" />
<class name="baeldung.com.SignInTest" />
</classes>
</test>
</suite>
Note that, if we have multiple test tags in the XML file, these tests can also be run in parallel, by mentioning parallel =” tests”. Also to execute individual methods in parallel, mention parallel =” methods”.
10.2. Multi-Threaded Execution of Test Method
Let’s say we need to test the behavior of a code when running in multiple threads. TestNG allows to run a test method in multiple threads:
public class MultiThreadedTests {
@Test(threadPoolSize = 5, invocationCount = 10, timeOut = 1000)
public void givenMethod_whenRunInThreads_thenCorrect() {
int count = Thread.activeCount();
assertTrue(count > 1);
}
}
The threadPoolSize indicates that the method will run in n number of threads as mentioned. The invocationCount and timeOut indicate that the test will be executed multiple times and fail the test if it takes more time.
11. Functional Testing
TestNG comes with features which can be used for functional testing as well. In conjunction with Selenium, it can either be used to test functionalities of a web application or used for testing web services with HttpClient.
More details about functional testing with Selenium and TestNG is available here. Also some more pieces of stuff on integration testing in this article.
12. Conclusion
In this article, we had a quick look at how to setup TestNG and execute a simple test case, generate reports, concurrent execution of test cases and also a little about functional programming. For more features like dependent tests, ignoring test cases, test groups and suites you can refer our JUnit vs TestNG article here.
The implementation of all the code snippets can be found over on Github.
