Upload and Delete files with Amazon S3 and Spring Boot

Introduction

As a developer, i am pretty sure that you may have come across with scenarios where you need to store images (either user uploaded or application itself) of your application.  There are several possibilities to store file(s) as follows.

  • Store the file(s) somewhere in the hosting server where the application is deployed (if it is a web application).
  • Store the file(s) in the database as binary files.
  • Store the file using cloud storage services.

Here we are going to evaluate the third option (given above) which is “Store the file using cloud storage services“.

Amazon Simple Storage Service (S3) is an AWS object storage platform which helps you to store the files in form of objects, and, store and retrieve any amount of data from anywhere.  Each file stored in Amazon S3 (as an object) is represented using a key.

 

 

Spring Boot Application and Amazon S3 Cloud  

 

Untitled Diagram (12).png

AWS Java SDK supports various APIs related to Amazon S3 service for working with files stored in S3 bucket.

 

 

Amazon S3 Account Configuration

Please follow the instructions given in the Amazon S3 official documentation for creating and configuring the S3 account and bucket.

Click here to visit the official documentation. 

I will list down the steps in below for your convenience.

 

Sign up with Amazon

First you need to go to https://aws.amazon.com/s3/ for sign up with Amazon S3 service.  You have to follow up the instructions given in the sign up document and need to complete the registration.

AWS will notify you by email when your account is active and available for you to use. Once your account is active and ready to use, you can

 

 

Creating S3 bucket

login to https://console.aws.amazon.com/s3 and click on “Create bucket” from below screen.

Screen Shot 2018-08-01 at 1.34.18 PM.png

 

 

Then provide the required details (e.g;- bucket name , region)  and  click on “Create“.

Screen Shot 2018-08-01 at 1.33.54 PM.png

 

 

Then it will list down the newly created bucket as follows.

Screen Shot 2018-08-01 at 1.35.03 PM.png

 

 

Create an IAM User

Go to https://console.aws.amazon.com/iam/In the navigation pane, choose Users and then choose Add user.

 

Input User name, choose Programmatic access for Access type:

Screen Shot 2018-08-01 at 1.38.48 PM.png

 

Press Next: Permissions button -> go to Set permissions for peopleshrm-app-user screen.

Now, choose Attach existing policies directly -> filter policy type s3, then check AmazonS3FullAccess. Then click on “Next Review“.

Screen Shot 2018-08-01 at 1.40.12 PM.png

Once you have reviewed the details, press “Create user“. Now you have successfully created the user.

Screen Shot 2018-08-01 at 1.41.15 PM.png

Click on Download .csv for downloading the credentials.  {Access key IDSecret access key}.

 

 

Spring Boot Amazon S3 client application.

 

Final Project Structure

Your final project structure should looks like below.

Screen Shot 2018-08-02 at 9.12.53 PM.png

 

 

Add Amazon SDK for the project

First we need to create a simple spring boot project and add the following AWS SDK dependency for the project.

 <dependency>
    <groupId>com.amazonaws</groupId>
    <artifactId>aws-java-sdk</artifactId>
    <version>1.11.106</version>
</dependency>

 

AWS Java SDK supports various APIs related to Amazon S3 service for working with files stored in S3 bucket.

 

application.properties

Add the AWS S3 bucket related configurations in the application.properties as follows. You will find those configurations details in the CSV  file (Download.csv) that you have already downloaded.  Change the properties

aws.access.key.id = your-access-key
aws.access.key.secret = your-private-key
aws.region = your-aws-service-region
aws.s3.audio.bucket = your-bucket-name

 

 

AmazonS3Config.java

 

AmazonS3 is a class from amazon dependency. All other fields are just a representation of variables from our application.properties file. The @Value annotation will bind application properties directly to class fields during application initialization.

 

AmazonS3ClientService.java

 

AmazonS3ClientServiceImpl.java

The two methods responsible for uploading and deleting files are annotated with @Async to make them asynchronous.  That means those methods will be executed in some other background thread except the main thread (the thread responsible for serving the request)

When uploading files, the below code segment is responsible for adding PublicRead permissions to the file being uploaded. It means that anyone who has the file url can access the file.

putObjectRequest.withCannedAcl(CannedAccessControlList.PublicRead);

 

 

FileHandlerController.java

 

 

Testing Application

 

Lets upload a file.

Screen Shot 2018-08-02 at 9.29.48 PM.png

Let’s test our application by making requests using Postman. We need to choose POST method, in the Body we should select ‘form-data’. As a key we should enter ‘file’ and choose value type ‘File’. Then choose any file from your local drive as a value.

As you can see that, we are trying to send a POST request to /files endpoint with file called “zuul-api-gateway.jpg“.  Once the request is submitted, the file will be uploaded to the S3 bucket in the background thread (@Async method).

Once the file is uploaded (after few seconds), it should be listed under your S3 bucket as follows.

Screen Shot 2018-08-02 at 10.46.27 PM.png

 

 

Lets remove a file 

In order to delete the file, the following HTTP DELETE request to the following endpoint should be sent along with name of the file to be deleted.

DELETE   http://localhost:8080/files?file_name=zuul-api-gateway.jpg

 

Screen Shot 2018-08-02 at 10.47.42 PM.png

 

Then again go back to the Amazon S3 bucket and check whether the file exists there. You can see that file is successfully removed and no longer listed there.

In this article, we have discussed how to upload files to the Amazon S3 bucket and how to remove the stored files.  If you have any query or clarification, please feel free to drop a comment or message.

 

The Source Code

The complete source code of this article can be found at GitHubClick here to download.

 

 

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s