Create and Run Apache JMeter Test Scripts via Java Program

1. Introduction

Apache JMeter is an open-source Java-based application designed to analyze and measure the performance of web applications. It allows a tester to simulate heavy loads on the servers, network, or objects to analyze overall performance under different loads. JMeter provides an easy-to-use GUI for defining, executing, and viewing reports for various load tests.

Although JMeter offers a user-friendly GUI for creating and executing test scripts, leveraging Java programming for automation can be beneficial in some situations, particularly within continuous integration and deployment pipelines.

In this tutorial, we’ll explore how to create and execute Apache JMeter test scripts programmatically using Java, accompanied by a practical example to illustrate the steps involved.

2. Setting Up the Environment

Before we start coding, let’s ensure we’ve got the required environment set up. To install JMeter, we can download it from the JMeter website. JMeter is compatible with Java 8 or higher. Alternatively, on macOS, we can install JMeter using Homebrew with the following command:

brew install jmeter

We also need to configure our Java project to include JMeter properties. For Maven projects, add the following dependencies to our pom.xml file:

<dependency>
    <groupId>org.apache.jmeter</groupId>
    <artifactId>ApacheJMeter_core</artifactId>
    <version>5.6.3</version>
</dependency>
<dependency>
    <groupId>org.apache.jmeter</groupId>
    <artifactId>ApacheJMeter_http</artifactId>
    <version>5.6.3</version>
</dependency>

These dependencies include Apache JMeter’s core functionality and HTTP Components. They provide essential classes and utilities for creating and running JMeter test plans, sending HTTP requests, and processing HTTP responses within JMeter test plans.

3. Creating the Test Script and Environment Variable File

We’ll generate a simple test script to emulate HTTP GET requests to a specified URL or application. The main purpose of this script is to perform load testing on the specified target application, which in this case is https://www.google.com.

It’ll simulate a genuine load on the provided URL by dispatching multiple concurrent requests, enabling us to assess the application’s performance across different load levels. In the below script, we first check the JMETER_HOME environment variable to locate the JMeter installation directory.

Once validated, we initialize the JMeter engine StandardJMeterEngine and create a test plan TestPlan. Below is the Java code to achieve this:

@Test
void givenJMeterScript_whenUsingCode_thenExecuteViaJavaProgram() throws IOException {
    String jmeterHome = System.getenv("JMETER_HOME");
    if (jmeterHome == null) {
        throw new RuntimeException("JMETER_HOME environment variable is not set.");
    }

    String file = Objects.requireNonNull(JMeterLiveTest.class.getClassLoader().getResource("jmeter.properties")).getFile();
    JMeterUtils.setJMeterHome(jmeterHome);
    JMeterUtils.loadJMeterProperties(file);
    JMeterUtils.initLocale();

    StandardJMeterEngine jmeter = new StandardJMeterEngine();

    HTTPSamplerProxy httpSampler = getHttpSamplerProxy();

    LoopController loopController = getLoopController();

    ThreadGroup threadGroup = getThreadGroup(loopController);

    TestPlan testPlan = getTestPlan(threadGroup);

    HashTree testPlanTree = new HashTree();
    HashTree threadGroupHashTree = testPlanTree.add(testPlan, threadGroup);
    threadGroupHashTree.add(httpSampler);

    SaveService.saveTree(testPlanTree, Files.newOutputStream(Paths.get("script.jmx")));
    Summariser summer = null;
    String summariserName = JMeterUtils.getPropDefault("summariser.name", "summary");
    if (summariserName.length() > 0) {
        summer = new Summariser(summariserName);
    }

    String logFile = "output-logs.jtl";
    ResultCollector logger = new ResultCollector(summer);
    logger.setFilename(logFile);
    testPlanTree.add(testPlanTree.getArray()[0], logger);

    jmeter.configure(testPlanTree);
    jmeter.run();

    System.out.println("Test completed. See output-logs.jtl file for results");
    System.out.println("JMeter .jmx script is available at script.jmx");
}

In the below getLoopController() method, a loop controller dictates the number of iterations or loops. In this scenario, we configure the loop controller to execute the test plan only once:

private static LoopController getLoopController() {
    LoopController loopController = new LoopController();
    loopController.setLoops(1);
    loopController.setFirst(true);
    loopController.setProperty(TestElement.TEST_CLASS, LoopController.class.getName());
    loopController.setProperty(TestElement.GUI_CLASS, LoopControlPanel.class.getName());
    loopController.initialize();
    return loopController;
}

The method getThreadGroup() defines a thread group called “Sample Thread Group“, specifying the number of virtual users/threads to simulate the ramp-up period. Each thread within this group represents a virtual user making requests to the target URL. The “Sample Thread Group” comprises 10 threads (virtual users) and ramps up over a 5-second interval:

private static ThreadGroup getThreadGroup(LoopController loopController) {
    ThreadGroup threadGroup = new ThreadGroup();
    threadGroup.setName("Sample Thread Group");
    threadGroup.setNumThreads(10);
    threadGroup.setRampUp(5);
    threadGroup.setSamplerController(loopController);
    threadGroup.setProperty(TestElement.TEST_CLASS, ThreadGroup.class.getName());
    threadGroup.setProperty(TestElement.GUI_CLASS, ThreadGroupGui.class.getName());
    return threadGroup;
}

Subsequently, in the getHttpSamplerProxy() method, we create an HTTP sampler HTTPSamplerProxy to dispatch HTTP GET requests to the target URL (https://www.google.com). We configure the sampler with the domain, path, and request method:

private static HTTPSamplerProxy getHttpSamplerProxy() {
    HTTPSamplerProxy httpSampler = new HTTPSamplerProxy();
    httpSampler.setDomain("www.google.com");
    httpSampler.setPort(80);
    httpSampler.setPath("/");
    httpSampler.setMethod("GET");
    httpSampler.setProperty(TestElement.TEST_CLASS, HTTPSamplerProxy.class.getName());
    httpSampler.setProperty(TestElement.GUI_CLASS, HttpTestSampleGui.class.getName());
    return httpSampler;
}

We construct the test plan by adding the thread group and HTTP sampler to a HashTree:

private static TestPlan getTestPlan(ThreadGroup threadGroup) {
    TestPlan testPlan = new TestPlan("Sample Test Plan");
    testPlan.setProperty(TestElement.TEST_CLASS, TestPlan.class.getName());
    testPlan.setProperty(TestElement.GUI_CLASS, TestPlanGui.class.getName());
    testPlan.addThreadGroup(threadGroup);
    return testPlan;
}

Finally, we configure the JMeter engine with the test plan tree, and we execute the test invoking the run method.

This script outlines a load profile that simulates 10 concurrent users gradually ramping up over 5 seconds. Each user initiates a single HTTP GET request to the target URL. The load remains consistent as there is no additional configuration for multiple iterations.

The jmeter.properties file serves as a configuration file used by Apache JMeter to define various settings and parameters for test execution. Within this file, the below property specifies the format for saving the test results upon completing the test execution:

jmeter.save.saveservice.output_format=xml

4. Understanding the Output File

We’ve saved the test plan as script.jmx in a .jmx file format that is compatible with loading and execution in the JMeter GUI. Additionally, we configured a Summarizer to gather and summarize test results. Furthermore, a result collector was established to store results in a .jtl file named output-logs.jtl.

4.1. Understanding the .jtl File

The content of the output-logs.jtl file are as follows:

<?xml version="1.0" encoding="UTF-8"?>
<testResults version="1.2">
<httpSample t="354" it="0" lt="340" ct="37" ts="1711874302012" s="true" lb="" rc="200" rm="OK"<br />  tn="Sample Thread Group 1-1" dt="text" by="22388" sby="111" ng="1" na="1">
    <java.net.URL>http://www.google.com/</java.net.URL>
</httpSample>
<httpSample t="351" it="0" lt="317" ct="21" ts="1711874302466" s="true" lb="" rc="200" rm="OK"<br />  tn="Sample Thread Group 1-2" dt="text" by="22343" sby="111" ng="1" na="1">
    <java.net.URL>http://www.google.com/</java.net.URL>
</httpSample>
<httpSample t="410" it="0" lt="366" ct="14" ts="1711874303024" s="true" lb="" rc="200" rm="OK"<br />  tn="Sample Thread Group 1-3" dt="text" by="22398" sby="111" ng="1" na="1">
    <java.net.URL>http://www.google.com/</java.net.URL>
</httpSample>
<httpSample t="363" it="0" lt="344" ct="19" ts="1711874303483" s="true" lb="" rc="200" rm="OK"<br />  tn="Sample Thread Group 1-4" dt="text" by="22367" sby="111" ng="1" na="1">
    <java.net.URL>http://www.google.com/</java.net.URL>
</httpSample>
</testResults>

The output contains <httpSample> element that represents an HTTP request made during the test, and it contains various attributes and metadata related to that request. Let’s break down these attributes:

• t – the total time the sample takes to execute in milliseconds
• it – idle time, i.e., the time to wait for a response
• lt – latency, i.e., the time taken for the request to reach the server and come back, excluding the idle time
• ct – connect time, i.e., the time taken to establish the connection to the server
• ts – timestamp when the sample was executed, represented in milliseconds since the epoch
• s – indicates whether the sample was successful (true) or not (false)
• lb – label associated with the sample, usually the sampler name
• rc – the HTTP response code returned by the server
• rm – response message associated with the response code
• tn – name of the thread that executed the sample
• dt – data type returned by the sample (e.g., text, binary)
• by – number of bytes received in the response body
• sby – number of bytes sent in the request
• ng/na – number of active threads in the thread group/ all thread groups

This data is crucial for analyzing the system performance under test, identifying bottlenecks, and optimizing the application’s performance.

4.2. Understanding the .jmx File

The .jmx file represents a JMeter test plan configuration in XML format. The content of the script.jmx file is as follows:

<?xml version="1.0" encoding="UTF-8"?>
<jmeterTestPlan version="1.2" properties="5.0" jmeter="5.4.1">
    <org.apache.jorphan.collections.HashTree>
        <TestPlan guiclass="TestPlanGui" testclass="TestPlan" testname="Sample Test Plan"/>
        <org.apache.jorphan.collections.HashTree>
            <ThreadGroup guiclass="ThreadGroupGui" testclass="ThreadGroup" testname="Sample Thread Group">
                <intProp name="ThreadGroup.num_threads">10</intProp>
                <intProp name="ThreadGroup.ramp_time">5</intProp>
                <elementProp name="ThreadGroup.main_controller" elementType="LoopController"<br />                  guiclass="LoopControlPanel" testclass="LoopController">
                    <boolProp name="LoopController.continue_forever">false</boolProp>
                    <intProp name="LoopController.loops">1</intProp>
                </elementProp>
            </ThreadGroup>
            <org.apache.jorphan.collections.HashTree>
                <HTTPSamplerProxy guiclass="HttpTestSampleGui" testclass="HTTPSamplerProxy">
                    <elementProp name="HTTPsampler.Arguments" elementType="Arguments">
                        <collectionProp name="Arguments.arguments"/>
                    </elementProp>
                    <stringProp name="HTTPSampler.domain">www.google.com</stringProp>
                    <intProp name="HTTPSampler.port">80</intProp>
                    <stringProp name="HTTPSampler.path">/</stringProp>
                    <stringProp name="HTTPSampler.method">GET</stringProp>
                </HTTPSamplerProxy>
                <org.apache.jorphan.collections.HashTree/>
            </org.apache.jorphan.collections.HashTree>
        </org.apache.jorphan.collections.HashTree>
    </org.apache.jorphan.collections.HashTree>
</jmeterTestPlan>

The test plan is defined within a <jmeterTestPlan> element. Within the test plan, a <ThreadGroup> element delineates the group of virtual users or threads that will execute the test scenarios. This includes details like the number of threads and the ramp-up time.

The thread group is orchestrated by a controller, exemplified by the <LoopController> element, which manages the execution flow, determining the number of loops and whether execution should continue indefinitely.

The actual HTTP requests to be sent are represented by <HTTPSampleProxy> elements, each detailing aspects like the domain, port, path, and HTTP method. Collectively, these elements form a comprehensive blueprint for orchestrating performance tests within the JMeter framework. It enables the simulation of user interactions with web servers and subsequent analysis of server responses.

5. Conclusion

In this article, we’ve demonstrated how to create and execute Apache JMeter test scripts programmatically using the JMeter Java API. Using this approach, developers can automate performance testing and integrate it seamlessly into their development workflows.

It enables efficient testing of web applications, helping identify and resolve performance bottlenecks early in the development cycle.

As always, the source code for this article can be found over on GitHub.