Conditionally Ignore Tests in TestNG

1. Overview

In real-world automation, we often need to enable or disable tests based on runtime conditions, such as the environment (stage or production), feature flags, configuration, CI/CD parameters, and so on.

In this tutorial, we’ll walkthrough different approaches to conditionally skip tests in TestNG.

2. Understanding the Problem

In TestNG, we can disable a test by using @Test with a parameter “enabled” with a boolean value false:

@Test(enabled=false)
public void givenNumbers_sumEquals_thenCorrect() {
    int sum = numbers.stream().reduce(0, Integer::sum);
    Assert.assertEquals(6, sum);
}

The above code works because Java annotations are evaluated at compile time, not runtime. This means the code below is invalid in Java:

@Test(enabled = isEnabled())
public void givenNumbers_sumEquals_thenCorrect() {
}

TestNG can’t evaluate runtime conditions inside annotations. It’s the reason that method calls, expressions, and variables are not allowed. We can solve this by using a mechanism that evaluates conditions at runtime, during test discovery, or execution.

3. Throwing SkipException

TestNG provides a built-in SkipException that allows us to skip tests programmatically. When we throw it, TestNG marks the test as SKIPPED, not FAILED. We can use this to skip tests programmatically based on runtime conditions. 

We can throw SkipException from different places in our test code. The mechanism stays the same, only the location changes. Let’s explore both approaches:

3.1. Skipping Inside the Test Method

In this approach, TestNG discovers and invokes the test method as usual. Inside the test, we check the condition at runtime. If the condition is met, we throw SkipException, which tells TestNG to skip the rest of the test:

@Test
public void givenConditions_whenUsingSkipException_thenSkipTest() {
    if (shouldSkipTest()) {
        throw new SkipException("Skipping test for Demonstration!");
    }
    System.out.println("This line won't get printed");
}

public boolean shouldSkipTest() {
    return true;
}

Running the above test returns the following output:

Test ignored.

===============================================
Default Suite
Total tests run: 1, Passes: 0, Failures: 0, Skips: 1
===============================================

Here, we’re placing the skip logic inside the test method. This can hurt readability. If we need to skip multiple tests, we must duplicate the logic in each one. Rather than checking conditions in every test, we can centralize the skip logic in lifecycle hooks. 

3.2. Skipping Using @BeforeMethod

TestNG always executes configuration methods such as @BeforeMethod and @BeforeClass before running a test. When we throw SkipException from these methods, TestNG treats it as a deliberate skip. It aborts the test immediately and marks it as SKIPPED. The test method never runs:

public class SkipExceptionInSetupUnitTest {
    @BeforeMethod
    public void givenConditions_whenUsingSkipException_thenSkipTest() {
        if (shouldSkipTest()) {
            throw new SkipException("Skipping tests for UK region");
        }
    }

    public boolean shouldSkipTest() {
        return true;
    }

    @Test
    public void givenSkipConditionInSetup_whenUsingSkipException_thenSkipTest() {
        System.out.println("Dummy Test!");
    }

    @Test
    public void givenSkipConditionInSetup_whenUsingSkipException_thenSkipTest2() {
        System.out.println("Dummy Test 2!");
    }
}

Running the above tests returns the following output:

Test ignored.

Test ignored.

===============================================
Default Suite
Total tests run: 2, Passes: 0, Failures: 0, Skips: 2
Configuration Failures: 0, Skips: 2
===============================================

This approach skips all tests in the class. It gives us less granular control. We should use this when multiple tests share the same runtime condition. It helps us avoid duplicating skip logic in each test.

4. Implementing an IAnnotationTransformer

IAnnotationTransformer is a TestNG listener interface that allows us to modify test annotations at runtime. TestNG calls its transform() method before processing each @Test annotation. We can dynamically change attributes like enabled, priority, or groups without modifying the source code. This is useful for enabling or disabling tests based on runtime conditions:

public class SkipAnnotationTransformer implements IAnnotationTransformer {
    @Override
    public void transform(ITestAnnotation annotation, Class testClass, Constructor testConstructor, Method testMethod) {
        if (testMethod != null && isTargetTestClass(testMethod) && shouldSkipTest()) {
            annotation.setEnabled(false);
        }
    }

    private boolean isTargetTestClass(Method testMethod) {
        return testMethod.getDeclaringClass().equals(SkipAnnotationTransformerUnitTest.class);
    }

    public boolean shouldSkipTest() {
        return true;
    }
}

In the transform() method, we check if the test method belongs to SkipAnnotationTransformerUnitTest and if shouldSkipTest() returns true. When both conditions are met, we set annotation.setEnabled(false). It completely disables the test – TestNG never schedules the test, never runs configuration methods, and never includes it in test counts.

In the testng.xml file, we register our SkipAnnotationTransformer as a listener using the <listeners> tag. We define a test suite that scans all packages matching com.baeldung.testng.* to automatically discover and run test classes:

<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Test Suite">
    <listeners>
        <listener class-name="com.baeldung.testng.conditionalskip.SkipAnnotationTransformer"/>
    </listeners>
    <test name="All Tests">
        <packages>
            <package name="com.baeldung.testng.*"/>
        </packages>
    </test>
</suite>

We configure the Maven Surefire Plugin to run our TestNG tests. We add the surefire-testng dependency to force Surefire to use the TestNG provider instead of auto-detecting Junit. We point Surefire to our testng.xml file using  <suiteXmlFiles> so it uses our suite configuration:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>3.2.5</version>
    <dependencies>
        <dependency>
            <groupId>org.apache.maven.surefire</groupId>
            <artifactId>surefire-testng</artifactId>
            <version>3.2.5</version>
        </dependency>
    </dependencies>
    <configuration>
        <suiteXmlFiles>
            <suiteXmlFile>src/test/resources/testng.xml</suiteXmlFile>
        </suiteXmlFiles>
    </configuration>
</plugin>

We define our test cases in the SkipAnnotationTransformerUnitTest, which we’ll be running to verify the transformer behaviour:

public class SkipAnnotationTransformerUnitTest {
    @Test
    public void givenSkipAnnotation_whenUsingIAnnotationTransformer_thenSkipTest() {
        System.out.println("Dummy Test!");
    }

    @Test
    public void givenSkipAnnotation_whenUsingIAnnotationTransformer_thenSkipTest2() {
        System.out.println("Dummy Test 2!");
    }
}

For running the SkipAnnotationTransformerUnitTest only, we’ve created a demo testng-transformer-demo.xml file:

<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Transformer Demo Suite">
    <listeners>
        <listener class-name="com.baeldung.testng.conditionalskip.SkipAnnotationTransformer"/>
    </listeners>
    <test name="Transformer Demo">
        <classes>
            <class name="com.baeldung.testng.conditionalskip.SkipAnnotationTransformerUnitTest"/>
        </classes>
    </test>
</suite>

Now, we’ll run the test using the following command:

mvn test -Dsurefire.suiteXmlFiles=src/test/resource/testng-transformer-demo.xml

We don’t see the tests defined in SkipAnnotationTransformerUnitTest as either running or skipped. TestNG ignores them completely.

5. Conclusion

In this article, we discussed various approaches to ignore a test in TestNG conditionally.

We can throw SkipException for the simplest and most explicit approach. However, this applies to skip decisions during test execution, not during discovery. The primary advantage is visibility, as skipped tests are explicitly reported as “SKIPPED”.

Alternatively, we can use IAnnotationTransformer, which operates earlier during test discovery. We disable tests that don’t meet the condition before execution begins, so they never run and never appear as skipped. This results in clean counts and faster execution.

Choosing between these depends on whether we need report transparency or execution cleanliness. 

As always, the code is available over on GitHub.