Intro to XPath with Java

1. Overview

The W3C recommends XPath as a standard syntax, providing a set of expressions to navigate XML documents.

In this tutorial, we’ll go over the basics of XPath with the support of the standard Java JDK.

We’ll take a simple XML document, process it, and learn how to extract the necessary information from it.

2. A Simple XPath Parser

import jakarta.xml.namespace.NamespaceContext;
import jakarta.xml.parsers.DocumentBuilder;
import jakarta.xml.parsers.DocumentBuilderFactory;
import jakarta.xml.parsers.ParserConfigurationException;
import jakarta.xml.xpath.XPath;
import jakarta.xml.xpath.XPathConstants;
import jakarta.xml.xpath.XPathExpressionException;
import jakarta.xml.xpath.XPathFactory;

import org.w3c.dom.Document;

public class DefaultParser {
    
    private File file;

    public DefaultParser(File file) {
        this.file = file;
    }
}

Now, let’s take a closer look at the elements we’ll find in the DefaultParser:

FileInputStream fileIS = new FileInputStream(this.getFile());
DocumentBuilderFactory builderFactory = newSecureDocumentBuilderFactory();
DocumentBuilder builder = builderFactory.newDocumentBuilder();
Document xmlDocument = builder.parse(fileIS);
XPath xPath = XPathFactory.newInstance().newXPath();
String expression = "/Tutorials/Tutorial";
NodeList nodeList = (NodeList) xPath.compile(expression).evaluate(xmlDocument, XPathConstants.NODESET);

Let’s break down the above code:

To produce a DOM object tree from our XML document, we’ll create a builderFactory instance using the newSecureDocumentBuilderFactory() method which internally creates and configures a DocumentBuilderFactory instance for secure XML parsing.

This method enhances security during XML parsing by disabling potentially dangerous features related to external entities and DTDs. This is a best practice when processing XML from untrusted sources to prevent XML-related vulnerabilities.

DocumentBuilderFactory builderFactory = newSecureDocumentBuilderFactory();

DocumentBuilder builder = builderFactory.newDocumentBuilder();

Having an instance of the DocumentBuilder class, we can parse XML documents from many different input sources like InputStream, File, URL, and SAX:

Document xmlDocument = builder.parse(fileIS);

A Document represents the entire XML document, is the root of the document tree, and provides our first access to data:

XPath xPath = XPathFactory.newInstance().newXPath();

From the XPath object, we’ll access the expressions and execute them over our document to extract what we need from it:

xPath.compile(expression).evaluate(xmlDocument, XPathConstants.NODESET);

We can compile an XPath expression passed as a string and define what kind of data we expect to receive, such as NODESET, NODE, or String.

3. Let’s Start

Now that we’ve reviewed the base components we’ll be using, let’s dive into some code with a simple XML example for testing purposes:

<?xml version="1.0"?>
<Tutorials>
    <Tutorial tutId="01" type="java">
        <title>Guava</title>
        <description>Introduction to Guava</description>
        <date>04/04/2016</date>
        <author>GuavaAuthor</author>
    </Tutorial>
    <Tutorial tutId="02" type="java">
        <title>XML</title>
        <description>Introduction to XPath</description>
        <date>04/05/2016</date>
        <author>XMLAuthor</author>
    </Tutorial>
</Tutorials>

3.1. Retrieve a Basic List of Elements

The first method is a simple use of an XPath expression to retrieve a list of nodes from the XML:

FileInputStream fileIS = new FileInputStream(this.getFile());
DocumentBuilderFactory builderFactory = DocumentBuilderFactory.newInstance();
DocumentBuilder builder = builderFactory.newDocumentBuilder();
Document xmlDocument = builder.parse(fileIS);
XPath xPath = XPathFactory.newInstance().newXPath();
String expression = "/Tutorials/Tutorial";
NodeList nodeList = (NodeList) xPath.compile(expression).evaluate(xmlDocument, XPathConstants.NODESET);

We can retrieve the tutorial list contained in the root node by using the expression above, or by using the expression “//Tutorial” but this one will retrieve all <Tutorial> nodes in the document from the current node no matter where they are located in the document, this means at whatever level of the tree starting from the current node.

The NodeList returns by specifying NODESET to the compile instruction. The return type is an ordered collection of nodes that can be accessed by passing an index as a parameter.

3.2. Retrieving a Specific Node by Its ID

We can look for an element based on any given ID just by filtering:

DocumentBuilderFactory builderFactory = DocumentBuilderFactory.newInstance();
DocumentBuilder builder = builderFactory.newDocumentBuilder();
Document xmlDocument = builder.parse(this.getFile());
XPath xPath = XPathFactory.newInstance().newXPath();
String expression = "/Tutorials/Tutorial[@tutId=" + "'" + id + "'" + "]";
node = (Node) xPath.compile(expression).evaluate(xmlDocument, XPathConstants.NODE);

Using this kind of expression, we can filter for any element we need by using the correct syntax. These kinds of expressions are called predicates and they are an easy way to locate specific data over a document, for example:

/Tutorials/Tutorial[1]

/Tutorials/Tutorial[first()]

/Tutorials/Tutorial[position()<4]

3.3. Retrieving Nodes by a Specific Tag Name

Now we’re going further by introducing axes, let’s see how this works by using it in an XPath expression:

Document xmlDocument = builder.parse(this.getFile());
this.clean(xmlDocument);
XPath xPath = XPathFactory.newInstance().newXPath();
String expression = "//Tutorial[descendant::title[text()=" + "'" + name + "'" + "]]";
NodeList nodeList = (NodeList) xPath.compile(expression).evaluate(xmlDocument, XPathConstants.NODESET);

With the expression used above, we’re looking for every <Tutorial> element that has a descendant <title> with the text passed as a parameter in the “name” variable.

Following the sample XML provided for this article, we could look for a <title> containing the text “Guava” or “XML” and we’ll retrieve the whole <Tutorial> element with all its data.

Axes provide a highly flexible way to navigate an XML document, and full documentation is available on the official site.

3.4. Manipulating Data in Expressions

XPath allows us to manipulate data too in the expressions if needed.

XPath xPath = XPathFactory.newInstance().newXPath();
String expression = "//Tutorial[number(translate(date, '/', '')) > " + date + "]";
nodeList = (NodeList) xPath.compile(expression).evaluate(xmlDocument, XPathConstants.NODESET);

In this expression we’re passing to our method a simple string as a date that looks like “ddmmyyyy” but the XML stores this data with the format “dd/mm/yyyy“, so to match a result we manipulate the string to convert it to the correct data format used by our document and we do it by using one of the functions provided by XPath

3.5. Retrieving Elements from a Document With Namespace Defined

If our XML document defines a namespace, as seen in the example_namespace.xml used here, the rules for retrieving the data will change because our XML begins like this:

<?xml version="1.0"?>
<Tutorials xmlns="/full_archive">
</Tutorials>

Now, when we use an expression similar to “//Tutorial”, we won’t get any result. That XPath expression is going to return all <Tutorial> elements that aren’t under any namespace, and in our new example_namespace.xml, all <Tutorial> elements are defined in the namespace /full_archive.

Let’s see how to handle namespaces.

First, we need to set the namespace context so that XPath can identify where we’re searching for our data:

xPath.setNamespaceContext(new NamespaceContext() {
    @Override
    public Iterator getPrefixes(String arg0) {
        return null;
    }
    @Override
    public String getPrefix(String arg0) {
        return null;
    }
    @Override
    public String getNamespaceURI(String arg0) {
        if ("bdn".equals(arg0)) {
            return "/full_archive";
        }
        return null;
    }
});

In the method above, we define “bdn” as the name for our namespace “/full_archive“. From now on, we must include “bdn” in the XPath expressions used to locate elements:

String expression = "/bdn:Tutorials/bdn:Tutorial";
NodeList nodeList = (NodeList) xPath.compile(expression).evaluate(xmlDocument, XPathConstants.NODESET);

Using the above expression, we can retrieve all <Tutorial> elements under “bdn” namespace.

3.6. Avoiding Empty Text Nodes Troubles

As we can see, in the code in the 3.3 section of this article a new function this.clean(xmlDocument) is called after parsing our XML to a Document object.

Sometimes, when we iterate through elements, childnodes, and so on, we may encounter unexpected behavior in our results if the document contains empty text nodes.

While iterating over all <Tutorial> elements to find the <title> information, we called the node.getFirstChild(), but instead of getting the desired result, we encountered an empty “#Text” node.

To fix the problem, we can navigate through our document and remove those empty nodes, like this:

NodeList childNodes = node.getChildNodes();
for (int n = childNodes.getLength() - 1; n >= 0; n--) {
    Node child = childNodes.item(n);
    short nodeType = child.getNodeType();
    if (nodeType == Node.ELEMENT_NODE) {
        clean(child);
    }
    else if (nodeType == Node.TEXT_NODE) {
        String trimmedNodeVal = child.getNodeValue().trim();
        if (trimmedNodeVal.length() == 0){
            node.removeChild(child);
        }
        else {
            child.setNodeValue(trimmedNodeVal);
        }
    } else if (nodeType == Node.COMMENT_NODE) {
        node.removeChild(child);
    }
}

By doing this, we can check each type of node we find and remove those we don’t need.

4. Conclusions

In this article, we explored the basics of XPath with the support of standard Java JDK. Java has excellent standard support available by default for parsing, reading, and processing XML/HTML document.

XPath expressions aren’t limited to Java; they can be used with XSLT to navigate XML documents. There are many popular libraries, such as JDOM, Saxon, XQuery, JAXP, Jaxen, and even Jackson. For  specific HTML parsing, libraries like JSoup are also available.