Mocking is an essential part of unit testing, and the Mockito library makes it easy to write clean and intuitive unit tests for your Java code.
Get started with mocking and improve your application tests using our Mockito guide:
Handling concurrency in an application can be a tricky process with many potential pitfalls. A solid grasp of the fundamentals will go a long way to help minimize these issues.
Get started with understanding multi-threaded applications with our Java Concurrency guide:
Spring 5 added support for reactive programming with the Spring WebFlux module, which has been improved upon ever since. Get started with the Reactor project basics and reactive programming in Spring Boot:
Since its introduction in Java 8, the Stream API has become a staple of Java development. The basic operations like iterating, filtering, mapping sequences of elements are deceptively simple to use.
But these can also be overused and fall into some common pitfalls.
To get a better understanding on how Streams work and how to combine them with other language features, check out our guide to Java Streams:
Explore Spring Boot 3 and Spring 6 in-depth through building a full REST API with the framework:
Yes, Spring Security can be complex, from the more advanced functionality within the Core to the deep OAuth support in the framework.
I built the security material as two full courses - Core and OAuth, to get practical with these more complex scenarios. We explore when and how to use each feature and code through it on the backing project.
You can explore the course here:
Spring Data JPA is a great way to handle the complexity of JPA with the powerful simplicity of Spring Boot.
Get started with Spring Data JPA through the guided reference course:
Refactor Java code safely — and automatically — with OpenRewrite.
Refactoring big codebases by hand is slow, risky, and easy to put off. That’s where OpenRewrite comes in. The open-source framework for large-scale, automated code transformations helps teams modernize safely and consistently.
Each month, the creators and maintainers of OpenRewrite at Moderne run live, hands-on training sessions — one for newcomers and one for experienced users. You’ll see how recipes work, how to apply them across projects, and how to modernize code with confidence.
Join the next session, bring your questions, and learn how to automate the kind of work that usually eats your sprint time.
Distributed systems often come with complex challenges such as service-to-service communication, state management, asynchronous messaging, security, and more.
Dapr (Distributed Application Runtime) provides a set of APIs and building blocks to address these challenges, abstracting away infrastructure so we can focus on business logic.
In this tutorial, we'll focus on Dapr's pub/sub API for message brokering. Using its Spring Boot integration, we'll simplify the creation of a loosely coupled, portable, and easily testable pub/sub messaging system:
1. Overview
Sometimes, we would like to remove all HTML tags and extract the text from an HTML document string.
The problem looks pretty straightforward. However, depending on the requirements, it can have different variants.
In this tutorial, we’ll discuss how to do that using Java.
2. Using Regex
Since we’ve already got the HTML as a String variable, we need to do a kind of text manipulation.
When facing text manipulation problems, regular expressions (Regex) could be the first idea coming up.
Removing HTML tags from a string won’t be a challenge for Regex since no matter the start or the end HTML elements, they follow the pattern “< … >”.
If we translate it into Regex, it would be “<[^>]*>” or “<.*?>”.
We should note that Regex does greedy matching by default. That is, the Regex “<.*>” won’t work for our problem since we want to match from ‘<‘ until the next ‘>‘ instead of the last ‘>‘ in a line.
Now, let’s test if it can remove tags from an HTML source.
2.1. Removing Tags From example1.html
Before we test removing HTML tags, first let’s create an HTML example, say example1.html:
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
"http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<title>This is the page title</title>
</head>
<body>
<p>
If the application X doesn't start, the possible causes could be:<br/>
1. <a href="maven.com">Maven</a> is not installed.<br/>
2. Not enough disk space.<br/>
3. Not enough memory.
</p>
</body>
</html>
Now, let’s write a test and use String.replaceAll() to remove HTML tags:
String html = ... // load example1.html
String result = html.replaceAll("<[^>]*>", "");
System.out.println(result);
If we run the test method, we see the result:
This is the page title
If the application X doesn't start, the possible causes could be:
1. Maven is not installed.
2. Not enough disk space.
3. Not enough memory.
The output looks pretty good. This is because all HTML tags have been removed.
It preserves whitespaces from the stripped HTML. But we can easily remove or skip those empty lines or whitespaces when we process the extracted text. So far, so good.
2.2. Removing Tags From example2.html
As we’ve just seen, using Regex to remove HTML tags is pretty straightforward. However, this approach may have problems since we cannot predict what HTML source we’ll get.
For example, an HTML document may have <script> or <style> tags, and we may not want to have their content in the result.
Further, the text in the <script>, <style>, or even the <body> tags could contain “<” or “>” characters. If this is the case, our Regex approach may fail.
Now, let’s see another HTML example, say example2.html:
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
"http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<title>This is the page title</title>
</head>
<script>
// some interesting script functions
</script>
<body>
<p>
If the application X doesn't start, the possible causes could be:<br/>
1. <a
id="link"
href="http://maven.apache.org/">
Maven
</a> is not installed.<br/>
2. Not enough (<1G) disk space.<br/>
3. Not enough (<64MB) memory.<br/>
</p>
</body>
</html>
This time, we have a <script> tag and “<” characters in the <body> tag.
If we use the same method on example2.html, we’ll get (empty lines have been removed):
This is the page title
// some interesting script functions
If the application X doesn't start, the possible causes could be:
1.
Maven
is not installed.
2. Not enough (
3. Not enough (Apparently, we’ve lost some text due to the “<” characters.
Therefore, using Regex to process XML or HTML is fragile. Instead, we can choose an HTML parser to do the job.
Next, we’ll address a few easy-to-use HTML libraries to extract text.
3. Using Jsoup
Jsoup is a popular HTML parser. To extract text from an HTML document, we can simply call Jsoup.parse(htmlString).text().
First, we need to add the Jsoup library to the classpath. For example, let’s say we’re using Maven to manage project dependencies:
<dependency>
<groupId>org.jsoup</groupId>
<artifactId>jsoup</artifactId>
<version>1.17.2</version>
</dependency>
Now, let’s test it with our example2.html:
String html = ... // load example2.html
System.out.println(Jsoup.parse(html).text());
If we give the method a run, it prints:
This is the page title If the application X doesn't start, the possible causes could be: 1. Maven is not installed. 2. Not enough (<1G) disk space. 3. Not enough (<64MB) memory.
As the output shows, Jsoup has successfully extracted texts from the HTML document. Also, the text in the <script> element has been ignored.
Additionally, by default, Jsoup will remove all text formatting and whitespaces, such as line breaks.
However, if it’s required, we can also ask Jsoup to preserve the line breaks.
4. Using HTMLCleaner
HTMLCleaner is another HTML parser. Its goal is to make “ill-formed and dirty” HTML from the Web suitable for further processing.
First, let’s add the HTMLCleaner dependency in our pom.xml:
<dependency>
<groupId>net.sourceforge.htmlcleaner</groupId>
<artifactId>htmlcleaner</artifactId>
<version>2.25</version>
</dependency>
We can set various options to control HTMLCleaner’s parsing behavior.
Here, as an example, let’s tell HTMLCleaner to skip the <script> element when parsing example2.html:
String html = ... // load example2.html
CleanerProperties props = new CleanerProperties();
props.setPruneTags("script");
String result = new HtmlCleaner(props).clean(html).getText().toString();
System.out.println(result);
HTMLCleaner will produce this output if we run the test:
This is the page title
If the application X doesn't start, the possible causes could be:
1.
Maven
is not installed.
2. Not enough (<1G) disk space.
3. Not enough (<64MB) memory.
As we can see, the content in the <script> element has been ignored.
Also, it converts <br/> tags into line breaks in the extracted text. This can be helpful if the format is significant.
On the other hand, HTMLCleaner preserves whitespace from the stripped HTML source. So, for example, the text “1. Maven is not installed” is broken into three lines.
5. Using Jericho
At last, we’ll see another HTML parser – Jericho. It has a nice feature: rendering HTML markup with simple text formatting. We’ll see it in action later.
As usual, let’s first add the Jericho dependency in the pom.xml:
<dependency>
<groupId>net.htmlparser.jericho</groupId>
<artifactId>jericho-html</artifactId>
<version>3.4</version>
</dependency>
In our example2.html, we have a hyperlink “Maven (http://maven.apache.org/)“. Now, let’s say we would like to have both the link URL and link text in the result.
To do that, we can create a Renderer object and use the includeHyperlinkURLs option:
String html = ... // load example2.html
Source htmlSource = new Source(html);
Segment segment = new Segment(htmlSource, 0, htmlSource.length());
Renderer htmlRender = new Renderer(segment).setIncludeHyperlinkURLs(true);
System.out.println(htmlRender);
Next, let’s execute the test and check the output:
If the application X doesn't start, the possible causes could be:
1. Maven <http://maven.apache.org/> is not installed.
2. Not enough (<1G) disk space.
3. Not enough (<64MB) memory.As we can see in the result above, the text has been pretty-formatted. Also, the text in the <title> element is ignored by default.
The link URL is included as well. Apart from rendering links (<a>), Jericho supports rendering other HTML tags, for example <hr/>, <br/>, bullet-list (<ul> and <li>), and so on.
6. Conclusion
In this article, we’ve addressed different ways to remove HTML tags and extract HTML text.
We should note that it’s not a good practice to use Regex to process XML/HTML.
