Using Amazon Textract in Spring Boot to Extract Text From Images

1. Overview

Businesses often need to extract meaningful data from various types of images, such as processing invoices or receipts for expense tracking, identity documents for KYC (Know Your Customer) processes, or automating data entry from forms. However, manually extracting text from images is a time-consuming and expensive process.

Amazon Textract offers an automated solution, extracting printed texts and handwritten data from documents using machine learning.

In this tutorial, we’ll explore how to use Amazon Textract within a Spring Boot application to extract text from images. We’ll walk through the necessary configuration and implement the functionality to extract text from both local image files and images stored in Amazon S3.

2. Setting up the Project

Before we can start extracting text from images, we’ll need to include an SDK dependency and configure our application correctly.

2.1. Dependencies

Let’s start by adding the Amazon Textract dependency to our project’s pom.xml file:

<dependency>
    <groupId>software.amazon.awssdk</groupId>
    <artifactId>textract</artifactId>
    <version>2.27.5</version>
</dependency>

This dependency provides us with the TextractClient and other related classes, which we’ll use to interact with the Textract service.

2.2. Defining AWS Configuration Properties

Now, to interact with the Textract service and extract text from images, we need to configure our AWS credentials for authentication and AWS region where we want to use the service.

We’ll store these properties in our project’s application.yaml file and use @ConfigurationProperties to map the values to a POJO, which our service layer references when interacting with Textract:

@Validated
@ConfigurationProperties(prefix = "com.baeldung.aws")
class AwsConfigurationProperties {
    @NotBlank
    private String region;

    @NotBlank
    private String accessKey;

    @NotBlank
    private String secretKey;

    // standard setters and getters
}

We’ve also added validation annotations to ensure all the required properties are configured correctly. If any of the defined validations fail, the Spring ApplicationContext will fail to start up. This allows us to conform to the fail-fast principle.

Below is a snippet of our application.yaml file, which defines the required properties that’ll be mapped to our AwsConfigurationProperties class automatically:

com:
  baeldung:
    aws:
      region: ${AWS_REGION}
      access-key: ${AWS_ACCESS_KEY}
      secret-key: ${AWS_SECRET_KEY}

We use the ${} property placeholder to load the values of our properties from environment variables.

Accordingly, this setup allows us to externalize the AWS properties and easily access them in our application.

2.3. Declaring TextractClient Bean

Now that we’ve configured our properties, let’s reference them to define our TextractClient bean:

@Bean
public TextractClient textractClient() {
    String region = awsConfigurationProperties.getRegion();
    String accessKey = awsConfigurationProperties.getAccessKey();
    String secretKey = awsConfigurationProperties.getSecretKey();
    AwsBasicCredentials awsCredentials = AwsBasicCredentials.create(accessKey, secretKey);

    return TextractClient.builder()
      .region(Region.of(region))
      .credentialsProvider(StaticCredentialsProvider.create(awsCredentials))
      .build();
}

The TextractClient class is the main entry point for interacting with the Textract service. We’ll autowire it in our service layer and send requests to extract texts from image files.

3. Extracting Text From Images

Now that we’ve defined our TextractClient bean, let’s create a TextExtractor class and reference it to implement our intended functionality:

public String extract(@ValidFileType MultipartFile image) {
    byte[] imageBytes = image.getBytes();
    DetectDocumentTextResponse response = textractClient.detectDocumentText(request -> request
      .document(document -> document
        .bytes(SdkBytes.fromByteArray(imageBytes))
        .build())
      .build());
    
    return transformTextDetectionResponse(response);
}

private String transformTextDetectionResponse(DetectDocumentTextResponse response) {
    return response.blocks()
      .stream()
      .filter(block -> block.blockType().equals(BlockType.LINE))
      .map(Block::text)
      .collect(Collectors.joining(" "));
}

In our extract() method, we convert the MultipartFile to a byte array and pass it as a Document to the detectDocumentText() method.

Amazon Textract currently only supports PNG, JPEG, TIFF, and PDF file formats. We create a custom validation annotation @ValidFileType to ensure the uploaded file is in one of these supported formats.

For our demonstration, in our helper method transformTextDetectionResponse(), we transform the DetectDocumentTextResponse into a simple String by joining the text content of each block. However, the transformation logic can be customized based on business requirements.

In addition to passing the image from our application, we can also extract text from images stored in our S3 buckets:

public String extract(String bucketName, String objectKey) {
    textractClient.detectDocumentText(request -> request
      .document(document -> document
        .s3Object(s3Object -> s3Object
          .bucket(bucketName)
          .name(objectKey)
          .build())
        .build())
      .build());
    
    return transformTextDetectionResponse(response);
}

In our overloaded extract() method, we take the S3 bucket name and object key as parameters, allowing us to specify the location of our image in S3.

It’s important to note that we invoke our TextractClient bean’s detectDocumentText() method, which is a synchronous operation used for processing single-page documents. However, for processing multi-page documents, Amazon Textract offers asynchronous operations instead.

4. IAM Permissions

Finally, for our application to function, we’ll need to configure some permissions for the IAM user we’ve configured in our application:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowTextractDocumentDetection",
            "Effect": "Allow",
            "Action": "textract:DetectDocumentText",
            "Resource": "*"
        },
        {
            "Sid": "AllowS3ReadAccessToSourceBucket",
            "Effect": "Allow",
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::bucket-name/*"
        }
    ]
}

In our IAM policy, the AllowTextractDocumentDetection statement allows us to invoke the DetectDocumentText API to extract text from images.

If we’re extracting texts from images stored in S3, we’ll also need to include the AllowS3ReadAccessToSourceBucket statement to allow read access to our S3 bucket.

Our IAM policy conforms to the least privilege principle, granting only the necessary permissions required by our application to function correctly.

5. Conclusion

In this article, we’ve explored using Amazon Textract with Spring Boot to extract text from images.

We discussed how to extract text from local image files as well as from images stored in Amazon S3.

Amazon Textract is a powerful service that’s heavily used in fintech and healthtech industries, helping automate tasks such as processing invoices or extracting patient data from medical forms.