1. Overview
Applications and devices use SSL certificates to secure connections. They help secure communication between the server and the client. These certificates have an expiry time and need to be replaced.
Doing this replacement manually is time-consuming and can lead to downtime or a bad user experience. In this article, we’re going to learn how to hot reload them. We’ll do this in a Spring Boot application using HashiCorp Vault.
2. Key Concepts
In this section, we’ll learn about some basic concepts that are helpful in later sections.
2.1. SSL Certificate
SSL stands for Secure Sockets Layer, a security protocol that creates an encrypted link between a web server and a web browser.
An SSL certificate is a digital certificate that authenticates the identity of a website or server and creates an encrypted connection. This certificate includes information such as the issuer authority name, the recipient, the public key, the expiry date, and other details to validate the authenticity of a website or server.
Throughout this article, when we use the word “certificate,” we are referring to an SSL certificate.
2.2. X.509 Certificate
X.509 is one of the standards for defining digital certificates. It’s the most commonly used one on the internet. It contains the identity of a website or server and a public key and is either signed by a certificate authority or is self-signed.
2.3. Root CA
In the domain of SSL, there are a handful of authorities that are widely trusted and issue certificates. These are called Root CAs (Certificate Authorities), e.g., GeoTrust, DigiCert, etc. The certificates issued by them are called Root Certificates. Operating systems and browsers recognize these trusted Root CAs, allowing them to validate other certificates. This system ensures that users can securely connect to websites and verify their identity.
Directly exposing the Root CA certificate may pose certain security issues. To mitigate this, we create an Intermediate CA (Certificate Authority) that acts as a link between the Root CA and the end-user certificates. We can create as many intermediate CAs as needed. Each Intermediate CA issues certificates on behalf of its parent CA, creating a chain of trust without directly exposing the Root CA’s private key.
This setup enhances security by keeping the Root CA secure while allowing the Intermediate CA to validate and issue certificates. Browsers and devices rely on this chain of trust to ensure secure connections to websites and services.
2.5. HashiCorp Vault
Vault is a tool to securely store and access sensitive information like tokens, passwords, keys, digital certificates, etc. HashiCorp is a company that has many products and one such product is Vault. It stores secrets, rotates secrets, encrypts data, and also issues certificates.
2.6. PKI Secret Engine
In HashiCorp Vault, we use a PKI secret engine to generate dynamic X.509 certificates. With this secrets engine, services can get certificates without going through the usual manual process of generating a private key and CSR, submitting to a CA, and waiting for a verification and signing process to be completed.
2.7. Vault Agent
It’s a client daemon that communicates with the Vault server and requests the issuance of certificates. We can configure it to generate certificates at regular intervals in a specific directory. With the help of the Vault Agent, we’ll achieve the hot reloading of certificates.
So far, we have understood some basic concepts related to the SSL world and HashiCorp Vault. Now, let’s understand how these components will work together to enable our Spring Boot application to hot reload the certificates issued by HashiCorp Vault.
In a nutshell, we’ll enable our application to reload certificates from a configured directory upon their expiration. The Vault Agent, which is a separate process, requests the Vault server to issue certificates and then writes them to a directory at regular intervals.
3. Configuring Vault Server
We’ve seen that it’s better to use an Intermediate CA than a Root CA. In this section, we’ll setup our Vault server, configure a Root CA, and then an Intermediate CA.
First, follow this guide to install the Vault and verify its version by running the vault -v command.
Now run these commands one by one to setup the Root CA:
vault server -dev -dev-root-token-id=root
export VAULT_ADDR='http://127.0.0.1:8200'
export VAULT_TOKEN=root
vault secrets enable pki
vault secrets tune -max-lease-ttl=24h pki
vault write -field=certificate pki/root/generate/internal common_name="localhost" \
issuer_name="root-2024" ttl=24h
vault write pki/config/urls issuing_certificates="${VAULT_ADDR}/v1/pki/ca" \
crl_distribution_points="${VAULT_ADDR}/v1/pki/crl”
vault write pki/roles/localhost-12 allow_any_name=true max_ttl=12h
Here, we’re starting the Vault server in dev mode. By default, it runs on localhost:8200. We’ve set root as the token. We can also access the GUI of Vault Server on localhost:8200 from the browser.
We’re exporting address and token and it will be used in subsequent commands. We’re enabling the PKI secret engine at path /pki to issue a certificate with an expiry time of a maximum of 24h.
We’re also assigning CRL location and the location of the issuing certificate. Finally, create a role named localhost-12 that can issue a root certificate with a maximum expiry time of 12h.
Just like the above section, we’ll run some commands to setup an intermediate CA:
vault secrets enable -path=pki-int pki
vault secrets tune -max-lease-ttl=12h pki-int
vault write -format=json pki-int/intermediate/generate/internal common_name="localhost \
Intermediate Authority" issuer_name="localhost-intermediate" \
| jq -r '.data.csr' > pki-intermediate.csr
vault write -format=json pki/root/sign-intermediate issuer_ref="root-2024" \
[email protected] format=pem_bundle ttl="12h" \
| jq -r '.data.certificate' > intermediate.cert.pem
vault write pki-int/intermediate/set-signed [email protected]
vault write pki-int/roles/localhost-3 allow_any_name=true max_ttl=3h
Here, we’ve enabled another PKI script engine at path /pki-int with a maximum expiry time of 12h.
Then, we generate our intermediate certificate signing request and saving in a file pki-intermediate.csr. With this signing request, signing it using our Root CA. After this, we’re setting the intermediate certificate authorities signing certificate to the root-signed certificate.
Lastly, we create a role localhost-3 to issue certificates that will expire in 3h.
Now, our intermediate certificate authority is configured. We can test it by using the command below:
vault write pki-int/issue/localhost-3 common_name="localhost" ttl=1h
This command should generate certificates on the terminal with an expiry time of 1h.
This is the manual way to generate certificates. To automate the generation of certificates from an Intermediate CA, we’ll use Vault Agent.
To use the intermediate CA, we’ll tie a policy to the role localhost-3 so that it can use the PKI secret engine:
path "pki-int/issue/localhost-3" {
capabilities = ["update"]
}
Save the above snippet in a file called policy.hcl and run the command below to create this policy:
vault policy write vault-agent-policy path/to/policy.hcl
Now, we’ll enable an authentication method like AppRole:
vault auth enable approle
vault write auth/approle/role/localhost-3 \
token_policies="vault-agent-policy" \
secret_id_ttl="24h" \
token_ttl="1h" \
token_max_ttl="4h"
The above command will tie the role localhost-3 to vault-agent-policy.
Write role-id and secret-id to files.
vault read auth/approle/role/localhost/role-id
echo “role-id-from-above-command" > role-id
vault write -f auth/approle/role/localhost/secret-id
echo “secret-id-from-above-command" > secret-id
Now we’ll configure a template and save it in a file template.tpl:
{{- with pkiCert "pki-int/issue/localhost-3" "common_name=localhost" "alt_names=localhost" "ttl=5m" -}}
{{ .Cert }}{{ .CA }}{{ .Key }}
{{ .Key | trimSpace | writeToFile "/path/to/private_key.key" "" "" "0400" }}
{{ .CA | trimSpace | writeToFile "/path/to/ca.pem" "" "" "0644" }}
{{ .Cert | trimSpace | writeToFile "/path/to/certificate.crt" "" "" "0644" }}
{{- end -}}
In this template, a role localhost-3 that we created earlier will generate certificates with an expiry time of 5mins and save it to files. Every time before the certificate expires, it’ll ask the Vault server to issue a certificate and write them to these files.
Lastly, we need a configuration file for the Vault Agent:
pid_file = "/paht/to/pidfile.pid"
vault {
address = "http://localhost:8200"
}
auto_auth {
method {
type = "approle"
config = {
role_id_file_path = "/path/to/role-id"
secret_id_file_path = "/path/to/secret-id"
remove_secret_id_file_after_reading = false
}
}
}
template {
source = "/path/to/template.tpl"
destination = "/path/to/all-certs"
}
Save this configuration in a file called vault_agent.hcl. In this file, we’re mentioning about Vault server address, auth method, location of the template.tpl file and a destination file all-certs where the Vault Agent writes all certificates and keys. Then, create a file pidfile.pid that Vault Agent uses to write its process id.
Now run the below command to start the Vault Agent:
sudo vault agent -config=/path/to/vault_agent.hcl
Our Vault Agent is started. We can verify the generation of certificates in files private_key.key, ca.pem, certificate.crt, and all-certs files. We’re running with the sudo command to avoid any permission issues.
So, we’ve configured the Vault server and Vault Agent. Let’s configure our Spring Boot app.
First, we create a Spring Boot application with spring-boot-starter-web and spring-cloud-starter-vault-config:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
<version>3.4.0</version>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-vault-config</artifactId>
<version>4.2.0</version>
</dependency>
Second, we add the following Spring Boot properties in the application.properties file to read and hot reload the certificates:
server.ssl.bundle=demo
spring.ssl.bundle.pem.demo.reload-on-update=true
spring.ssl.bundle.pem.demo.keystore.certificate=file:/path/to/certificate.crt
spring.ssl.bundle.pem.demo.keystore.private-key=file:/path/to/private_key.key
The above properties refer to the location of private_key.key and certificate.crt file mentioned in the template.tpl file. Spring Boot automatically reads and reloads the certificate from these locations when content changes.
The application will start successfully only when the certificates are valid. We can verify also by looking at logs. If we monitor logs for more than five minutes, then we’ll see the logs of this type:
Connector [https-jsse-nio-8080], TLS virtual host [_default_], certificate type [UNDEFINED] configured from keystore [/Users/$HOME/.keystore] using alias [tomcat] with trust store [null]
The above logs continue to appear when certificates are reloaded.
6. Conclusion
In this article, we’ve done a lot of things. We started by learning the basic concepts related to SSL and HashiCorp Vault. We learned about Root CAs and Intermediate CAs, and also understood the importance of Intermediate CA certificates.
Next, we made a couple of configurations. We learned how to run a Vault server, configure a Root CA, and set up an Intermediate CA. We then explored Vault Agent, its importance, and how to configure and run it to issue certificates.
Lastly, we created a Spring Boot app and configured it to hot reload the certificates.
