1. Introduction
In this tutorial, we’ll use MongoDB’s Client-Side Field Level Encryption, or CSFLE, to encrypt selected fields in our documents. We’ll cover explicit/automatic encryption and explicit/automatic decryption, highlighting the differences between encryption algorithms.
Ultimately, we’ll have a simple application that can insert and retrieve documents with a mix of encrypted and unencrypted fields.
2. Scenario and Setup
Both MongoDB Atlas and MongoDB Enterprise support Automatic Encryption. MongoDB Atlas has a free forever cluster that we can use to test all features.
Also, it’s worth noting that Field Level Encryption is distinct from storage at rest, which encrypts an entire database or disk. By selectively encrypting specific fields, we can better protect sensitive data while allowing efficient querying and indexing. So, we’ll start with a simple Spring Boot application to insert and retrieve data with Spring Data MongoDB.
First, we’ll create a document class containing a mix of unencrypted and encrypted fields. We’ll start with manual encryption and then see how the same can be achieved with automatic encryption. For the manual encryption, we’ll need an intermediary object to represent our encrypted POJO, and we’ll create methods to encrypt/decrypt each field.
2.1. Spring Boot Starters and Encryption Dependencies
Firstly, to connect to MongoDB, we’ll need spring-boot-starter-data-mongodb:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-mongodb</artifactId>
</dependency>Then, let’s add mongodb-crypt to our project to enable encryption features:
<dependency>
<groupId>org.mongodb</groupId>
<artifactId>mongodb-crypt</artifactId>
<version>1.7.3</version>
</dependency>
Since we’re using Spring Boot, this is the only dependency we’ll need now.
2.2. Creating Our Master Key
The master key is used to encrypt and decrypt data uniquely. Anyone that has it can read our data. Therefore, it’s essential to keep it safe.
MongoDB recommends using a remote key management service. But, to keep things simple, let’s create a local key manager:
public class LocalKmsUtils {
public static byte[] createMasterKey(String path) {
byte[] masterKey = new byte[96];
new SecureRandom().nextBytes(masterKey);
try (FileOutputStream stream = new FileOutputStream(path)) {
stream.write(masterKey);
}
return masterKey;
}
// ...
}The only requirement for our local key is that it’s 96 bytes long. We’ll create a local key store that’s used just for the sake of demonstration – we’re filling it with random bytes.
This key needs to be generated only once, so let’s create a method to retrieve it if it’s already created:
public static byte[] readMasterKey(String path) {
byte[] masterKey = new byte[96];
try (FileInputStream stream = new FileInputStream(path)) {
stream.read(masterKey, 0, 96);
}
return masterKey;
}Finally, we’ll create a method to return a map containing our master key in the format required by ClientEncryptionSettings we’ll create later:
public static Map<String, Map<String, Object>> providersMap(String masterKeyPath) {
File masterKeyFile = new File(masterKeyPath);
byte[] masterKey = masterKeyFile.isFile()
? readMasterKey(masterKeyPath)
: createMasterKey(masterKeyPath);
Map<String, Object> masterKeyMap = new HashMap<>();
masterKeyMap.put("key", masterKey);
Map<String, Map<String, Object>> providersMap = new HashMap<>();
providersMap.put("local", masterKeyMap);
return providersMap;
}It supports using many keys, but we use only one for this tutorial.
2.3. Customizing the Configuration
To ease configuration, let’s create a few custom properties. Then, we’ll use a configuration class to hold these and a couple of objects we’ll need for encryption.
Let’s start with a configuration that points to our local master key:
@Configuration
public class EncryptionConfig {
@Value("${com.baeldung.csfle.master-key-path}")
private String masterKeyPath;
// ...
}Then, let’s include configurations for our key vault:
@Value("${com.baeldung.csfle.key-vault.namespace}")
private String keyVaultNamespace;
@Value("${com.baeldung.csfle.key-vault.alias}")
private String keyVaultAlias;
// gettersThe key vault is a collection of encryption keys. So, the namespace combines a database and a collection name. And the alias is a simple name to retrieve our key vault later.
Finally, let’s create a property to hold our encryption key ID:
private BsonBinary dataKeyId;
// getters and setters
It’ll be populated later when we make our MongoDB client configuration.
3. Creating the MongoClient and Encryption Objects
To create the objects and settings needed for encryption, let’s create a custom MongoDB client so we have more control over its configuration.
Let’s start by extending AbstractMongoClientConfiguration, adding the usual parameters to acquire a connection, and injecting our encryptionConfig:
@Configuration
public class MongoClientConfig extends AbstractMongoClientConfiguration {
@Value("${spring.data.mongodb.uri}")
private String uri;
@Value("${spring.data.mongodb.database}")
private String db;
@Autowired
private EncryptionConfig encryptionConfig;
@Override
protected String getDatabaseName() {
return db;
}
// ...
}Next, let’s create a method to return the MongoClientSettings object needed to create our client and ClientEncryption objects. We’ll use our connection uri variable:
private MongoClientSettings clientSettings() {
return MongoClientSettings.builder()
.applyConnectionString(new ConnectionString(uri))
.build();
}Then, we’ll create our ClientEncryption bean, which is responsible for making data keys and encryption operations. It’s constructed from a ClientEncryptionSettings object, which receives our clientSettings(), the key vault namespace from our EncryptionConfig, and a map from our providersMap() method:
@Bean
public ClientEncryption clientEncryption() {
ClientEncryptionSettings encryptionSettings = ClientEncryptionSettings.builder()
.keyVaultMongoClientSettings(clientSettings())
.keyVaultNamespace(encryptionConfig.getKeyVaultNamespace())
.kmsProviders(LocalKmsUtils.providersMap(encryptionConfig.getMasterKeyPath()))
.build();
return ClientEncryptions.create(encryptionSettings);
}
Ultimately, we return it for later use in constructing our data key.
3.1. Creating Our Data Key
A final step before creating our MongoDB client is a method to generate the data key if it doesn’t exist. Next, we’ll receive our ClientEncryption object to get a reference for our key vault document by alias:
private BsonBinary createOrRetrieveDataKey(ClientEncryption encryption) {
BsonDocument key = encryption.getKeyByAltName(encryptionConfig.getKeyVaultAlias());
if (key == null) {
createKeyUniqueIndex();
DataKeyOptions options = new DataKeyOptions();
options.keyAltNames(Arrays.asList(encryptionConfig.getKeyVaultAlias()));
return encryption.createDataKey("local", options);
} else {
return (BsonBinary) key.get("_id");
}
}When no result is returned, we generate the key with createDataKey(), passing our alias configuration. Otherwise, we get its “_id” field. Also, even though we’re using a single key, we create a unique index for the keyAltNames field so we don’t risk creating duplicate aliases. We do that with createIndex() and a partial filter expression because this field isn’t required:
private void createKeyUniqueIndex() {
try (MongoClient client = MongoClients.create(clientSettings()) {
MongoNamespace namespace = new MongoNamespace(encryptionConfig.getKeyVaultNamespace());
MongoCollection<Document> keyVault = client.getDatabase(namespace.getDatabaseName())
.getCollection(namespace.getCollectionName());
keyVault.createIndex(Indexes.ascending("keyAltNames"), new IndexOptions().unique(true)
.partialFilterExpression(Filters.exists("keyAltNames")));
}
}It’s worth noting that MongoClient needs to be closed when no longer in use. Since we need it here only once to create the index, we use it in a try-with-resources block, so it’s closed right after use.
3.2. Putting It Together to Create Our Client
Finally, let’s override mongoClient() to create our client and encryption objects, then store our data key ID in our encryptionConfig.
@Bean
@Override
public MongoClient mongoClient() {
ClientEncryption encryption = clientEncryption();
encryptionConfig.setDataKeyId(createOrRetrieveDataKey(encryption));
return MongoClients.create(clientSettings());
}And with all that setup, we’re ready to encrypt some fields.
4. Service for Encrypting Fields
Let’s create a service class to save and retrieve documents with encrypted fields. But before we start encrypting, we need documents.
4.1. Document Classes
Let’s start with a class to hold a few basic properties:
@Document("citizens")
public class Citizen {
private String name;
private String email;
private Integer birthYear;
// getters and setters
}Then, we need a version of the same type but with binary properties. Since we’re doing explicit encryption, we’ll use this class to hold the encrypted data:
@Document("citizens")
public class EncryptedCitizen {
private String name;
private Binary email;
private Binary birthYear;
// getters and setters
}4.2. Initializing the Service
Our service class holds all the configurations necessary for encryption. The algorithm types, the ClientEncryption bean, and the EncryptionConfig. Also, it has a reference to MongoTemplate so that we can save and fetch documents:
@Service
public class CitizenService {
public static final String DETERMINISTIC_ALGORITHM = "AEAD_AES_256_CBC_HMAC_SHA_512-Deterministic";
public static final String RANDOM_ALGORITHM = "AEAD_AES_256_CBC_HMAC_SHA_512-Random";
private final MongoTemplate mongo;
private final EncryptionConfig encryptionConfig;
private final ClientEncryption clientEncryption;
public CitizenService(
MongoTemplate mongo, EncryptionConfig encryptionConfig, ClientEncryption clientEncryption) {
this.mongo = mongo;
this.encryptionConfig = encryptionConfig;
this.clientEncryption = clientEncryption;
}
// ...
}MongoDB allows two types of encryption algorithms: deterministic and random. A deterministic algorithm will always result in the same encrypted value, whereas a random one won’t. This makes random algorithms more secure but implies that fields encrypted with it cannot be easily queried for. That’s because we have to encrypt values before querying. On the other hand, when decrypting, the chosen algorithm doesn’t matter.
And now, let’s add a method for encrypting values:
public Binary encrypt(BsonValue bsonValue, String algorithm) {
Objects.requireNonNull(bsonValue);
Objects.requireNonNull(algorithm);
EncryptOptions options = new EncryptOptions(algorithm);
options.keyId(encryptionConfig.getDataKeyId());
BsonBinary encryptedValue = clientEncryption.encrypt(bsonValue, options);
return new Binary(encryptedValue.getType(), encryptedValue.getData());
}This method uses the passed algorithm and the data key from our configuration and returns a type compatible with our EncryptedCitizen. Also, let’s add a couple of helpers for the types we need:
Binary encrypt(String value, String algorithm) {
Objects.requireNonNull(value);
Objects.requireNonNull(algorithm);
return encrypt(new BsonString(value), algorithm);
}
Binary encrypt(Integer value, String algorithm) {
Objects.requireNonNull(value);
Objects.requireNonNull(algorithm);
return encrypt(new BsonInt32(value), algorithm);
}Note that null values aren’t encrypted. If we have null field values in our object, they won’t be present in our document.
4.3. Saving Documents
Finally, let’s add a method in our service class to save documents. Again, we’ll use a deterministic algorithm for the email and a random one for the birth year:
public void save(Citizen citizen) {
EncryptedCitizen encryptedCitizen = new EncryptedCitizen();
encryptedCitizen.setName(citizen.getName());
if (citizen.getEmail() != null) {
encryptedCitizen.setEmail(encrypt(citizen.getEmail(), DETERMINISTIC_ALGORITHM));
} else {
encryptedCitizen.setEmail(null);
}
if (citizen.getBirthYear() != null) {
encryptedCitizen.setBirthYear(encrypt(citizen.getBirthYear(), RANDOM_ALGORITHM));
} else {
encryptedCitizen.setBirthYear(null);
}
mongo.save(encryptedCitizen);
}We can now fetch documents using mongo.findAll(EncryptedCitizen.class). But the encrypted fields won’t be readable.
5. Decrypting Fields
To decrypt fields, we need to call ClientEncryption.decrypt() for every field we want to decrypt. This method receives an encrypted BsonBinary and returns a decrypted BsonValue.
Let’s start with a method for decrypting Binary values, converting it to a BsonBinary before passing it to ClientEncryption.decrypt(). It’s essential to use the BsonBinary constructor that receives the binary subtype; otherwise, we might get a MongoCryptException:
public BsonValue decryptProperty(Binary value) {
Objects.requireNonNull(value);
return clientEncryption.decrypt(
new BsonBinary(value.getType(), value.getData()));
}Then, we’ll use it in a method for decrypting an instance of EncryptedCitizen:
private Citizen decrypt(EncryptedCitizen encrypted) {
Objects.requireNonNull(encrypted);
Citizen citizen = new Citizen();
citizen.setName(encrypted.getName());
BsonValue decryptedBirthYear = encrypted.getBirthYear() != null
? decryptProperty(encrypted.getBirthYear())
: null;
if (decryptedBirthYear != null) {
citizen.setBirthYear(decryptedBirthYear.asInt32()
.intValue());
}
BsonValue decryptedEmail = encrypted.getEmail() != null
? decryptProperty(encrypted.getEmail())
: null;
if (decryptedEmail != null) {
citizen.setEmail(decryptedEmail.asString()
.getValue());
}
return citizen;
}Finally, let’s put it all together and create a findAll() implementation that decrypts the data received from our database:
public List<Citizen> findAll() {
List<EncryptedCitizen> allEncrypted = mongo.findAll(EncryptedCitizen.class);
return allEncrypted.stream()
.map(this::decrypt)
.collect(Collectors.toList());
}5.1. Configuring Automatic Decryption
In addition, the MongoDB client allows us to configure automatic decryption. We have to configure our client’s auto-encryption settings to enable this feature, which receives our master key configuration. So, let’s go back to EncryptionConfig to create a new configuration property:
@Value("${com.baeldung.csfle.auto-decryption:false}")
private boolean autoDecryption;
// default getterWe’re setting the default value to false, so the property isn’t required. Then, in MongoClientConfig, we’ll refactor clientSettings() to check if auto decryption is enabled and build AutoEncryptionSettings:
MongoClientSettings clientSettings() {
Builder settings = MongoClientSettings.builder()
.applyConnectionString(new ConnectionString(uri));
if (encryptionConfig.isAutoDecryption()) {
settings.autoEncryptionSettings(
AutoEncryptionSettings.builder()
.keyVaultNamespace(encryptionConfig.getKeyVaultNamespace())
.kmsProviders(LocalKmsUtils.providersMap(encryptionConfig.getMasterKeyPath()))
.bypassAutoEncryption(true)
.build());
}
return settings.build();
}Most importantly, we’re setting bypassAutoEncryption(true). This is required, as only automatic decryption has been configured so far. And this is all the configuration we need. With this feature enabled, the decryption is done by the MongoDB client.
6. Querying Encrypted Fields
To filter by encrypted fields when querying, if we have only the unencrypted value, we have to encrypt the value we want before executing the query. For example, let’s add a method in CitizenService to query by email:
Citizen findByEmail(String email) {
Query byEmail = new Query(Criteria.where("email")
.is(encrypt(email, DETERMINISTIC_ALGORITHM)));
return mongo.findOne(byEmail, Citizen.class);
}As long as the field is saved with a deterministic algorithm, it returns the expected document.
7. Automatic Encryption
Configuring automatic encryption is possible by specifying the cryptSharedLibPath in our MongoClient. Let’s start by including a couple of configurations in EncryptionConfig. The autoEncryptionLib will only be required if we specify autoEncryption as true, so we use null as the default value:
@Value("${com.baeldung.csfle.auto-encryption:false}")
private boolean autoEncryption;
@Value("${com.baeldung.csfle.auto-encryption-lib:#{null}}")
private File autoEncryptionLib;
// default getters
Also, let’s add a helper method to retrieve our data key as a UUID String. We’ll need it later on to configure our client:
public String dataKeyIdUuid() {
if (dataKeyId == null)
throw new IllegalStateException("data key not initialized");
return dataKeyId.asUuid()
.toString();
}7.1. Updating Driver Dependencies
To use the cryptSharedLibPath driver option, we’ll also have to make sure we’re using the latest versions of mongodb-driver-sync, mongodb-driver-core, and bson:
<dependency>
<groupId>org.mongodb</groupId>
<artifactId>mongodb-driver-sync</artifactId>
<version>4.9.1</version>
</dependency>
<dependency>
<groupId>org.mongodb</groupId>
<artifactId>mongodb-driver-core</artifactId>
<version>4.9.1</version>
</dependency>
<dependency>
<groupId>org.mongodb</groupId>
<artifactId>bson</artifactId>
<version>4.9.1</version>
</dependency>7.2. Refactoring MongoClientConfig
The autoEncryptionLib points to the crypt_shared library file, which must be downloaded before using this feature. Let’s refactor clientSettings() in MongoClientConfig to check if this option is enabled, we already have our data key, and the auto encryption lib is an actual file:
if (encryptionConfig.isAutoDecryption()) {
AutoEncryptionSettings.Builder builder = AutoEncryptionSettings.builder()
.keyVaultNamespace(encryptionConfig.getKeyVaultNamespace())
.kmsProviders(LocalKmsUtils.providersMap(encryptionConfig.getMasterKeyPath()));
if (encryptionConfig.isAutoEncryption() && encryptionConfig.getDataKeyId() != null) {
File autoEncryptionLib = encryptionConfig.getAutoEncryptionLib();
if (!autoEncryptionLib.isFile()) {
throw new IllegalArgumentException("encryption lib must be an existing file");
}
// ...
} else {
builder.bypassAutoEncryption(true);
}
settings.autoEncryptionSettings(builder.build());
}Now we only set bypassAutoEncryption to true if auto encryption isn’t enabled. Next, we need to define our extra options and schema map:
Map<String, Object> map = new HashMap<>();
map.put("cryptSharedLibRequired", true);
map.put("cryptSharedLibPath", autoEncryptionLib.toString());
builder.extraOptions(map);The cryptSharedLibRequired option will force crypt_shared to be correctly configured instead of trying to spawn mongocryptd if it isn’t. Using crypt_shared is preferred as we don’t need additional services running on our machine.
7.3. Encryption Schema
For automatic encryption to work, we must provide an encryption schema map for every collection we want to encrypt. So, our next step is to use our “citizens” collection and define the fields we want to encrypt. For this, we’ll define some key objects: encryptMetadata and properties:
String keyUuid = encryptionConfig.dataKeyIdUuid();
HashMap<String, BsonDocument> schemaMap = new HashMap<>();
schemaMap.put(getDatabaseName() + ".citizens", BsonDocument.parse("{"
+ " bsonType: \"object\","
+ " encryptMetadata: {"
+ " keyId: [UUID(\"" + keyUuid + "\")]"
+ " },"
+ " properties: {"
+ " email: {"
+ " encrypt: {"
+ " bsonType: \"string\","
+ " algorithm: \"AEAD_AES_256_CBC_HMAC_SHA_512-Deterministic\""
+ " }"
+ " },"
+ " birthYear: {"
+ " encrypt: {"
+ " bsonType: \"int\","
+ " algorithm: \"AEAD_AES_256_CBC_HMAC_SHA_512-Random\""
+ " }"
+ " }"
+ " }"
+ "}"));
builder.schemaMap(schemaMap);We set encryptMetadata with our key ID for all our encrypted properties, so we don’t need to define it for every property definition. For properties, we defined email and birthYear and specified their bsonType and encryption algorithm.
7.4. Simplifying Work on CitizenService
Now that we have enabled automatic encryption, we no longer need explicit encryption. Let’s refactor CitizenService to take our configuration into account, starting with the save() method:
public void save(Citizen citizen) {
if (encryptionConfig.isAutoEncryption()) {
mongo.save(citizen);
} else {
// same as before
}
}Note that we only have a fallback for manual encryption for demonstration purposes. A production application would not need such a fallback.
Then, for findByEmail(), if we have auto encryption on, we don’t need to manually encrypt the value of email anymore:
public Citizen findByEmail(String email) {
Criteria emailCriteria = Criteria.where("email");
if (encryptionConfig.isAutoEncryption()) {
emailCriteria.is(email);
} else {
emailCriteria
.is(encrypt(email, DETERMINISTIC_ALGORITHM));
}
Query byEmail = new Query(emailCriteria);
if (encryptionConfig.isAutoDecryption()) {
return mongo.findOne(byEmail, Citizen.class);
} else {
EncryptedCitizen encryptedCitizen = mongo.findOne(byEmail, EncryptedCitizen.class);
return decrypt(encryptedCitizen);
}
}8. Conclusion
In this article, we learned how MongoDB’s CSFLE feature works, how it’s configured, and the classes involved during encryption and decryption.
Moreover, we saw the differences between random and deterministic encryption algorithms. Finally, we configured our client to encrypt and decrypt fields automatically.
And as always, the source code is available over on GitHub.
