HttpClient with SSL

1. Overview

This article will show how to configure the Apache HttpClient 4 with “Accept All” SSL support. The goal is simple – consume HTTPS URLs which do not have valid certificates.

If you want to dig deeper and learn other cool things you can do with the HttpClient – head on over to the main HttpClient guide.

2. The SSLPeerUnverifiedException

Without configuring SSL with the HttpClient, the following test – consuming an HTTPS URL – will fail:

public class HttpLiveTest {

    @Test(expected = SSLPeerUnverifiedException.class)
    public void whenHttpsUrlIsConsumed_thenException() 
      throws ClientProtocolException, IOException {
        DefaultHttpClient httpClient = new DefaultHttpClient();
        String urlOverHttps = "https://localhost:8080/spring-security-rest-basic-auth";
        HttpGet getMethod = new HttpGet(urlOverHttps);
        HttpResponse response = httpClient.execute(getMethod);
        assertThat(response.getStatusLine().getStatusCode(), equalTo(200));
    }
}

The exact failure is:

javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated
    at sun.security.ssl.SSLSessionImpl.getPeerCertificates(SSLSessionImpl.java:397)
    at org.apache.http.conn.ssl.AbstractVerifier.verify(AbstractVerifier.java:126)
    ...

The javax.net.ssl.SSLPeerUnverifiedException exception occurs whenever a valid chain of trust couldn’t be established for the URL.

3. Configure SSL – Accept All (HttClient < 4.3)

Let’s now configure the http client to trust all certificate chains regardless of their validity:

@Test
public void givenAcceptingAllCertificates_whenHttpsUrlIsConsumed_thenException() 
  throws IOException, GeneralSecurityException {
    TrustStrategy acceptingTrustStrategy = new TrustStrategy() {
        @Override
        public boolean isTrusted(X509Certificate[] certificate, String authType) {
            return true;
        }
    };
    SSLSocketFactory sf = new SSLSocketFactory(acceptingTrustStrategy, 
      SSLSocketFactory.ALLOW_ALL_HOSTNAME_VERIFIER);
    SchemeRegistry registry = new SchemeRegistry();
    registry.register(new Scheme("https", 8443, sf));
    ClientConnectionManager ccm = new PoolingClientConnectionManager(registry);

    DefaultHttpClient httpClient = new DefaultHttpClient(ccm);

    String urlOverHttps = "https://localhost:8443/spring-security-rest-basic-auth/api/bars/1";
    HttpGet getMethod = new HttpGet(urlOverHttps);
    HttpResponse response = httpClient.execute(getMethod);
    assertThat(response.getStatusLine().getStatusCode(), equalTo(200));
}

With the new TrustStrategy now overriding the standard certificate verification process (which should consult a configured trust manager) – the test now passes and the client is able to consume the HTTPS URL.

4. The Spring RestTemplate with SSL (HttpClient < 4.3)

Now that we have seen how to configure a raw HttpClient with SSL support, let’s take a look at a more high level client – the Spring RestTemplate.

With no SSL configured, the following test fails as expected:

@Test(expected = ResourceAccessException.class)
public void whenHttpsUrlIsConsumed_thenException() {
    String urlOverHttps = "https://localhost:8443/spring-security-rest-basic-auth/api/bars/1";
    ResponseEntity<String> response = 
      new RestTemplate().exchange(urlOverHttps, HttpMethod.GET, null, String.class);
    assertThat(response.getStatusCode().value(), equalTo(200));
}

So let’s configure SSL:

import static org.apache.http.conn.ssl.SSLSocketFactory.ALLOW_ALL_HOSTNAME_VERIFIER;
import java.security.GeneralSecurityException;
import java.security.cert.X509Certificate;
import org.apache.http.auth.AuthScope;
import org.apache.http.auth.UsernamePasswordCredentials;
import org.apache.http.conn.scheme.Scheme;
import org.apache.http.conn.ssl.SSLSocketFactory;
import org.apache.http.conn.ssl.TrustStrategy;
import org.apache.http.impl.client.DefaultHttpClient;
import org.springframework.http.HttpMethod;
import org.springframework.http.ResponseEntity;
import org.springframework.http.client.HttpComponentsClientHttpRequestFactory;
import org.springframework.web.client.ResourceAccessException;
import org.springframework.web.client.RestTemplate;

...
@Test
public void givenAcceptingAllCertificates_whenHttpsUrlIsConsumed_thenException() 
  throws GeneralSecurityException {
    HttpComponentsClientHttpRequestFactory requestFactory = 
      new HttpComponentsClientHttpRequestFactory();
    DefaultHttpClient httpClient = (DefaultHttpClient) requestFactory.getHttpClient();
    TrustStrategy acceptingTrustStrategy = new TrustStrategy() {
    @Override
        public boolean isTrusted(X509Certificate[] certificate, String authType) {
            return true;
        }
    };
    SSLSocketFactory sf = new SSLSocketFactory(acceptingTrustStrategy, ALLOW_ALL_HOSTNAME_VERIFIER);
    httpClient.getConnectionManager().getSchemeRegistry().register(new Scheme("https", 8443, sf));

    String urlOverHttps = "https://localhost:8443/spring-security-rest-basic-auth/api/bars/1";
    ResponseEntity<String> response = new RestTemplate(requestFactory).
      exchange(urlOverHttps, HttpMethod.GET, null, String.class);
    assertThat(response.getStatusCode().value(), equalTo(200));
}

As you can see, this is very similar to the way we configured SSL for the raw HttpClient – we configure the request factory with SSL support and then we instantiate the template passing this preconfigured factory.

5. Conclusion

This tutorial discussed how to configure SSL for an Apache HttpClient so that it is able to consume any HTTPS URL, regardless of the certificate. The same configuration for the Spring RestTemplate is also illustrated.

An important thing to understand however is that this strategy entirely ignores certificate checking – which makes it insecure and only to be used where that makes sense.

The implementation of these examples can be found in the github project – this is an Eclipse based project, so it should be easy to import and run as it is.

I usually post about REST APIs and HTTP on Google+ - you can follow me there:

GET THE 3 EBOOKS >>
Download the 3 eBooks - Build Your App "The Right Way"
×
Build Your Web App with Spring (and quickly prototype it to 90%)

, ,

  • I. N. Secure

    You should at least mention that your TrustStrategy implementation does not perform any verification of the used certificate at all, which renders the use of ssl pretty much useless, as a man in the middle can present any certificate matching the name, no matter who it was issued or signed by.

    • baeldung

      The article does mention repeatedly – in the very beginning (The goal is simple – consume HTTPS URLs which do not have valid certificates.) then in the Accept All chapter (Let’s now configure the http client to trust all certificate chains regardless of their validity…) and once more in the Conclusion, so hopefully it should be clear that the solution is simply ignoring certificates. The goal here is to be tolerant of invalid certificate chains when analyzing large datasets which may contain such URLs.
      Thanks.
      Eugen.

    • znogger

      True, but a SSL connection provides two benefits: encryption and verification of the endpoint. If you trust any certificate then you loose the latter benefit. The connection is still encrypted and at least a man in the middle would have KNOW that the connector accepts any certificate to do something nasty. The barrier for an attacker has been lowered significantly but there are still advantages left. It still has a security level higher than a non-SSL connection.

      • baeldung

        Agreed – that particular connection has a lot more entry-points for a potential attacker. However, these connections are throw away – attacking such a connection would lead to nothing. It’s only when that’s the case that one should ignore certificates. One such usecase I had – in fact, the usecase that prompted me to look into this solution – was processing URLs to extract some data from the page (title, description).

  • Rich Freedman

    This is a totally insecure, and therefore really bad, idea.
    Typically, the reason that folks look to this approach is that they’re using self-signed certificates.
    The proper way to handle this situation is to create a custom trust store that trusts the self-signed cert and it’s CA.
    For an example of how to do this the right way, see http://blog.chariotsolutions.com/2013/01/https-with-client-certificates-on.html

    • baeldung

      It is definitely not secure as it simply configures the HttpClient to ignore certificate checking (which the article does state at the very beginning). That being said, I found there are valid usecases for such a strategy – one being heavy data analysis of large number of URLs – some of which unfortunately do use HTTPS without a valid certificate chain. I will try to make it even more clear in the conclusion of the article – thanks for pointing it out.
      Eugen.

  • David Karr

    Just for clarity’s sake, your example in step 2 isn’t a https url. You also misspelled “RestTempalte” in step 4.

    • baeldung

      Fixed the issues – thank for spotting these.

  • znogger

    Hi Baeldung,

    You use Apache HttpClient rather than what comes by default with the JDK. That choice is yours.
    But I think you should at least begin your article by telling WHY you’ve made this choice. I’ve actually met programmers who thought that they HAD to bring in Apache’s HttpClient in order to do any kind of http client stuff.

    There is nothing in what you do here that cannot be done with standard JDK classes. This is not to say that Apache HttpClient did not have its time and (some will argue) still very much has its time. The deficiencies (and even sometimes severe bugs) that gave the standard JDK classes a bad rep (I’m talking about HttpURLConnection and HttpsURLConnection) have – in my opinion – been solved in Java 1.5, Java 6 and Java 7. Having said that there are still many that will argue that Apache HttpClient is still easier to use. I don’t doubt that is the case. !

    For my own part I can only say that I haven’t come across something I cannot do in the http/https client space with the standard JDK classes. Apache HttpClient may be easier to use but this hasn’t been enough of a reason for me (yet) to bring in an external library.

    Just my two cents.

    • baeldung

      Thanks for the interesting feedback. This is definitely just one solution to the problem – there are many other valid ones. My own personal preference is for the Apache client simply because I find it has a nicer API than the JDK classes. Another advantage is that it doesn’t have a multi-year release schedule – if you need something changed in the JDK classes, there isn’t much chance of that happening any time soon (or at all), whereas if you need a change in the Apache HTTP client, you can either suggest it or contribute it yourself. Granted, that may or may not happen, but I like knowing that I have the option to do so. Finally, it plays well with Spring and the RestTemplate – which is not something that I can say about the core classes. And finally, chances are that in a project of significant size, you’ll have it on the classpath anyway – there are so many other libraries that define the httpclient as a dependency. For me personally, all of this justifies the cost of adding it – but I can understand going for the core classes as well.

      About making it clear why I choose HttpClient – my thinking is that since the article is named “HttpClient with SSL”, it’s transparent that it simply illustrates how to set up SSL with the HttpClient (and nothing more) – not necessarily that the HttpClient is the best option out there. I may write that article at some point (why I prefer HttpClient over the core classes).

      Thanks again for the comment.

      Eugen.

  • carma.robot

    This is so stupid. Why don’t you just show people how to create a self-signed certificate with the correct common name, then show them how to import that into the right trust store? Or how to load a custom trust store? What’s the point of using TLS/SSL if you’re disabling all the checks that matter? Why don’t just write a post that says: “Here’s how to solve your SSL/TLS issues – just don’t use it”. By writing this blog you are helping to propagate security stupidy.

    • baeldung

      To answer your question – there are situations when this is genuinely useful and the right thing to do – one of these is – in my own experience – doing some quick validations during a data analysis pipeline. The need was to verify the link and get some minimal information about the page (title for example) – I didn’t care that the page was over HTTPS or not, and I didn’t want to deal with the certificate handling. This is a quick and easy way to ignore that – if it’s required. Like anything else in life – you need to apply your own good judgement if you should use it or not.

  • Alexey

    You should probably update this article for the new HttpClient 4.3 – many of the mentioned classes have been deprecated.

    • http://www.baeldung.com/ Eugen Paraschiv

      Hey Alexey – you’re right, the examples are for HttpClient 4.x but pre 4.3 (a lot of APIs have changed in 4.3). I have updated the github project with new examples and will update the article as well.
      Thanks for reaching out. Cheers,
      Eugen.