1. Overview
In this tutorial, we’ll be looking at how to check if a URL exists with an example in Java using the GET and HEAD HTTP methods.
2. URL Existence
There can be situations in programming when we have to know if a resource exists in the given URL before accessing it, or we may even need to check a URL to know the resource’s health.
We decide a resource’s existence at a URL by looking at its response code. Typically we look for a 200, which means “OK” and that the request has succeeded.
3. Using a GET Request
First of all, to make a GET request, we can create an instance of java.net.URL and pass the URL that we would like to access as a constructor argument. After that, we simply open the connection and get the response code:
URL url = new URL("https://httpbin.org/status/200");
HttpURLConnection huc = (HttpURLConnection) url.openConnection();
int responseCode = huc.getResponseCode();
Assert.assertEquals(HttpURLConnection.HTTP_OK, responseCode);
When the resource is not found at the URL, we get a 404 response code:
URL url = new URL("https://httpbin.org/status/404");
HttpURLConnection huc = (HttpURLConnection) url.openConnection();
int responseCode = huc.getResponseCode();
Assert.assertEquals(HttpURLConnection.HTTP_NOT_FOUND, responseCode);
As the default HTTP method in HttpURLConnection is GET, we’re not setting the request method in the examples in this section. We’ll see how to override the default method in the next section.
4. Using a HEAD Request
The HEAD is also an HTTP request method that is identical to GET except that it does not return the response body.
It acquires the response code along with the response headers that we’ll receive if the same resource is requested with a GET method.
To create a HEAD request, we can simply set the Request Method to HEAD before getting the response code:
URL url = new URL("https://httpbin.org/status/200");
HttpURLConnection huc = (HttpURLConnection) url.openConnection();
huc.setRequestMethod("HEAD");
int responseCode = huc.getResponseCode();
Assert.assertEquals(HttpURLConnection.HTTP_OK, responseCode);
Similarly, when the resource is not found at the URL:
URL url = new URL("https://httpbin.org/status/404");
HttpURLConnection huc = (HttpURLConnection) url.openConnection();
huc.setRequestMethod("HEAD");
int responseCode = huc.getResponseCode();
Assert.assertEquals(HttpURLConnection.HTTP_NOT_FOUND, responseCode);
By using the HEAD method and thereby not downloading the response body, we reduce the response time and bandwidth, and we improve performance.
Although most modern servers support the HEAD method, some home-grown or legacy servers might reject the HEAD method with an invalid method type error. So, we should use the HEAD method with caution.
5. Following Redirects
Finally, when looking for URL existence, it might be a good idea not to follow redirects. But this can also depend on the reason we’re looking for the URL.
When a URL is moved, the server can redirect the request to a new URL with 3xx response codes. The default is to follow a redirect. We can choose to follow or ignore the redirect based on our need.
To do this, we can either override the default value of followRedirects for all the HttpURLConnections:
URL url = new URL("https://httpbin.org/status/200");
HttpURLConnection.setFollowRedirects(false);
HttpURLConnection huc = (HttpURLConnection) url.openConnection();
int responseCode = huc.getResponseCode();
Assert.assertEquals(HttpURLConnection.HTTP_OK, responseCode);
Or, we can disable following redirects for a single connection by using the setInstanceFollowRedirects() method:
URL url = new URL("https://httpbin.org/status/200");
HttpURLConnection huc = (HttpURLConnection) url.openConnection();
huc.setInstanceFollowRedirects(false);
int responseCode = huc.getResponseCode();
Assert.assertEquals(HttpURLConnection.HTTP_OK, responseCode);
6. Conclusion
In this article, we looked at checking the response code to find the availability of a URL. Also, we looked at how it might be a good idea to use the HEAD method to save bandwidth and get a quicker response.
The code example used in this tutorial is available in our GitHub project.
