Upload and Delete files with Amazon S3 and Spring Boot

Introduction

As a developer, i am pretty sure that you may have come across with scenarios where you need to store images (either user uploaded or application itself) of your application. There are several possibilities to store file(s) as follows.

Store the file(s) somewhere in the hosting server where the application is deployed (if it is a web application).
Store the file(s) in the database as binary files.
Store the file using cloud storage services.

Here we are going to evaluate the third option (given above) which is “Store the file using cloud storage services“.

Amazon Simple Storage Service (S3) is an AWS object storage platform which helps you to store the files in form of objects, and, store and retrieve any amount of data from anywhere. Each file stored in Amazon S3 (as an object) is represented using a key.

Spring Boot Application and Amazon S3 Cloud

Untitled Diagram (12).png

AWS Java SDK supports various APIs related to Amazon S3 service for working with files stored in S3 bucket.

Amazon S3 Account Configuration

Please follow the instructions given in the Amazon S3 official documentation for creating and configuring the S3 account and bucket.

Click here to visit the official documentation.

I will list down the steps in below for your convenience.

Sign up with Amazon

First you need to go to https://aws.amazon.com/s3/ for sign up with Amazon S3 service. You have to follow up the instructions given in the sign up document and need to complete the registration.

AWS will notify you by email when your account is active and available for you to use. Once your account is active and ready to use, you can

Creating S3 bucket

Screen Shot 2018-08-01 at 1.34.18 PM.png

Then provide the required details (e.g;- bucket name , region) and click on “Create“.

Screen Shot 2018-08-01 at 1.33.54 PM.png

Then it will list down the newly created bucket as follows.

Screen Shot 2018-08-01 at 1.35.03 PM.png

Create an IAM User

Go to https://console.aws.amazon.com/iam/ . In the navigation pane, choose Users and then choose Add user.

Input User name, choose Programmatic access for Access type:

Press Next: Permissions button -> go to Set permissions for peopleshrm-app-user screen.

Now, choose Attach existing policies directly -> filter policy type s3, then check AmazonS3FullAccess. Then click on “Next Review“.

Screen Shot 2018-08-01 at 1.40.12 PM.png

Once you have reviewed the details, press “Create user“. Now you have successfully created the user.

Screen Shot 2018-08-01 at 1.41.15 PM.png

Click on Download .csv for downloading the credentials. {Access key ID, Secret access key}.

Spring Boot Amazon S3 client application.

Final Project Structure

Your final project structure should looks like below.

Screen Shot 2018-08-02 at 9.12.53 PM.png

Add Amazon SDK for the project

First we need to create a simple spring boot project and add the following AWS SDK dependency for the project.

 <dependency>
    <groupId>com.amazonaws</groupId>
    <artifactId>aws-java-sdk</artifactId>
    <version>1.11.106</version>
</dependency>

AWS Java SDK supports various APIs related to Amazon S3 service for working with files stored in S3 bucket.

application.properties

Add the AWS S3 bucket related configurations in the application.properties as follows. You will find those configurations details in the CSV file (Download.csv) that you have already downloaded. Change the properties

aws.access.key.id = your-access-key
aws.access.key.secret = your-private-key
aws.region = your-aws-service-region
aws.s3.audio.bucket = your-bucket-name

AmazonS3Config.java

import org.springframework.beans.factory.annotation.Value;

import org.springframework.context.annotation.Bean;

import org.springframework.context.annotation.Configuration;

import com.amazonaws.auth.AWSCredentialsProvider;

import com.amazonaws.auth.AWSStaticCredentialsProvider;

import com.amazonaws.auth.BasicAWSCredentials;

import com.amazonaws.regions.Region;

import com.amazonaws.regions.Regions;

@Configuration

public class AmazonS3Config 

{

    @Value("${aws.access.key.id}")

    private String awsKeyId;

    @Value("${aws.access.key.secret}")

    private String awsKeySecret;

    @Value("${aws.region}")

    private String awsRegion;

    @Value("${aws.s3.audio.bucket}")

    private String awsS3AudioBucket;

    @Bean(name = "awsKeyId")

    public String getAWSKeyId() {

        return awsKeyId;

    }

    @Bean(name = "awsKeySecret")

    public String getAWSKeySecret() {

        return awsKeySecret;

    }

    @Bean(name = "awsRegion")

    public Region getAWSPollyRegion() {

        return Region.getRegion(Regions.fromName(awsRegion));

    }

    @Bean(name = "awsCredentialsProvider")

    public AWSCredentialsProvider getAWSCredentials() {

        BasicAWSCredentials awsCredentials = new BasicAWSCredentials(this.awsKeyId, this.awsKeySecret);

        return new AWSStaticCredentialsProvider(awsCredentials);

    }

    @Bean(name = "awsS3AudioBucket")

    public String getAWSS3AudioBucket() {

        return awsS3AudioBucket;

    }

}

view raw
AmazonS3Config.java
hosted with ❤ by GitHub

AmazonS3 is a class from amazon dependency. All other fields are just a representation of variables from our application.properties file. The @Value annotation will bind application properties directly to class fields during application initialization.

AmazonS3ClientService.java

public interface AmazonS3ClientService

{

    void uploadFileToS3Bucket(MultipartFile multipartFile, boolean enablePublicReadAccess);

    void deleteFileFromS3Bucket(String fileName);

}

view raw
AmazonS3ClientService.java
hosted with ❤ by GitHub

AmazonS3ClientServiceImpl.java

import com.amazonaws.services.s3.model.CannedAccessControlList;

import com.amazonaws.services.s3.model.DeleteObjectRequest;

import com.amazonaws.services.s3.model.PutObjectRequest;

import com.springbootdev.amazon.s3.example.aws.service.AmazonS3ClientService;

import org.slf4j.Logger;

import org.slf4j.LoggerFactory;

import org.springframework.beans.factory.annotation.Autowired;

import org.springframework.scheduling.annotation.Async;

import org.springframework.stereotype.Component;

import com.amazonaws.AmazonServiceException;

import com.amazonaws.auth.AWSCredentialsProvider;

import com.amazonaws.regions.Region;

import com.amazonaws.services.s3.AmazonS3;

import com.amazonaws.services.s3.AmazonS3ClientBuilder;

import org.springframework.web.multipart.MultipartFile;

import java.io.File;

import java.io.FileOutputStream;

import java.io.IOException;

@Component

public class AmazonS3ClientServiceImpl implements AmazonS3ClientService 

{

    private String awsS3AudioBucket;

    private AmazonS3 amazonS3;

    private static final Logger logger = LoggerFactory.getLogger(AmazonS3ClientServiceImpl.class);

    @Autowired

    public AmazonS3ClientServiceImpl(Region awsRegion, AWSCredentialsProvider awsCredentialsProvider, String awsS3AudioBucket) 

    {

        this.amazonS3 = AmazonS3ClientBuilder.standard()

                .withCredentials(awsCredentialsProvider)

                .withRegion(awsRegion.getName()).build();

        this.awsS3AudioBucket = awsS3AudioBucket;

    }

    @Async

    public void uploadFileToS3Bucket(MultipartFile multipartFile, boolean enablePublicReadAccess) 

    {

        String fileName = multipartFile.getOriginalFilename();

        try {

            //creating the file in the server (temporarily)

            File file = new File(fileName);

            FileOutputStream fos = new FileOutputStream(file);

            fos.write(multipartFile.getBytes());

            fos.close();

            PutObjectRequest putObjectRequest = new PutObjectRequest(this.awsS3AudioBucket, fileName, file);

            if (enablePublicReadAccess) {

                putObjectRequest.withCannedAcl(CannedAccessControlList.PublicRead);

            }

            this.amazonS3.putObject(putObjectRequest);

            //removing the file created in the server

            file.delete();

        } catch (IOException | AmazonServiceException ex) {

            logger.error("error [" + ex.getMessage() + "] occurred while uploading [" + fileName + "] ");

        }

    }

    @Async

    public void deleteFileFromS3Bucket(String fileName) 

    {

        try {

            amazonS3.deleteObject(new DeleteObjectRequest(awsS3AudioBucket, fileName));

        } catch (AmazonServiceException ex) {

            logger.error("error [" + ex.getMessage() + "] occurred while removing [" + fileName + "] ");

        }

    }

}

view raw
AmazonS3ClientServiceImpl.java
hosted with ❤ by GitHub

The two methods responsible for uploading and deleting files are annotated with @Async to make them asynchronous. That means those methods will be executed in some other background thread except the main thread (the thread responsible for serving the request)

When uploading files, the below code segment is responsible for adding PublicRead permissions to the file being uploaded. It means that anyone who has the file url can access the file.

putObjectRequest.withCannedAcl(CannedAccessControlList.PublicRead);

FileHandlerController.java

import com.springbootdev.amazon.s3.example.aws.service.AmazonS3ClientService;

import org.springframework.beans.factory.annotation.Autowired;

import org.springframework.web.bind.annotation.*;

import org.springframework.web.multipart.MultipartFile;

import java.util.HashMap;

import java.util.Map;

@RestController

@RequestMapping("/files")

public class FileHandlerController {

    @Autowired

    private AmazonS3ClientService amazonS3ClientService;

    @PostMapping

    public Map<String, String> uploadFile(@RequestPart(value = "file") MultipartFile file)

    {

        this.amazonS3ClientService.uploadFileToS3Bucket(file, true);

        Map<String, String> response = new HashMap<>();

        response.put("message", "file [" + file.getOriginalFilename() + "] uploading request submitted successfully.");

        return response;

    }

    @DeleteMapping

    public Map<String, String> deleteFile(@RequestParam("file_name") String fileName)

    {

        this.amazonS3ClientService.deleteFileFromS3Bucket(fileName);

        Map<String, String> response = new HashMap<>();

        response.put("message", "file [" + fileName + "] removing request submitted successfully.");

        return response;

    }

}

view raw
FileHandlerController.java
hosted with ❤ by GitHub

Testing Application

Lets upload a file.

Screen Shot 2018-08-02 at 9.29.48 PM.png

Let’s test our application by making requests using Postman. We need to choose POST method, in the Body we should select ‘form-data’. As a key we should enter ‘file’ and choose value type ‘File’. Then choose any file from your local drive as a value.

As you can see that, we are trying to send a POST request to /files endpoint with file called “zuul-api-gateway.jpg“. Once the request is submitted, the file will be uploaded to the S3 bucket in the background thread (@Async method).

Once the file is uploaded (after few seconds), it should be listed under your S3 bucket as follows.

Screen Shot 2018-08-02 at 10.46.27 PM.png

Lets remove a file

In order to delete the file, the following HTTP DELETE request to the following endpoint should be sent along with name of the file to be deleted.

DELETE http://localhost:8080/files?file_name=zuul-api-gateway.jpg

Screen Shot 2018-08-02 at 10.47.42 PM.png

Then again go back to the Amazon S3 bucket and check whether the file exists there. You can see that file is successfully removed and no longer listed there.

In this article, we have discussed how to upload files to the Amazon S3 bucket and how to remove the stored files. If you have any query or clarification, please feel free to drop a comment or message.

The Source Code

The complete source code of this article can be found at GitHub. Click here to download.

www.SpringBootDev.com

Upload and Delete files with Amazon S3 and Spring Boot

Introduction

Spring Boot Application and Amazon S3 Cloud

Amazon S3 Account Configuration

Sign up with Amazon

Creating S3 bucket

Create an IAM User

Spring Boot Amazon S3 client application.

Final Project Structure

Add Amazon SDK for the project

application.properties

AmazonS3Config.java

AmazonS3ClientService.java

AmazonS3ClientServiceImpl.java

FileHandlerController.java

Testing Application

Lets upload a file.

Lets remove a file

The Source Code

Leave a Reply Cancel reply

	import org.springframework.beans.factory.annotation.Value;
	import org.springframework.context.annotation.Bean;
	import org.springframework.context.annotation.Configuration;

	import com.amazonaws.auth.AWSCredentialsProvider;
	import com.amazonaws.auth.AWSStaticCredentialsProvider;
	import com.amazonaws.auth.BasicAWSCredentials;
	import com.amazonaws.regions.Region;
	import com.amazonaws.regions.Regions;

	@Configuration
	public class AmazonS3Config
	{
	@Value("${aws.access.key.id}")
	private String awsKeyId;

	@Value("${aws.access.key.secret}")
	private String awsKeySecret;

	@Value("${aws.region}")
	private String awsRegion;

	@Value("${aws.s3.audio.bucket}")
	private String awsS3AudioBucket;

	@Bean(name = "awsKeyId")
	public String getAWSKeyId() {
	return awsKeyId;
	}

	@Bean(name = "awsKeySecret")
	public String getAWSKeySecret() {
	return awsKeySecret;
	}

	@Bean(name = "awsRegion")
	public Region getAWSPollyRegion() {
	return Region.getRegion(Regions.fromName(awsRegion));
	}

	@Bean(name = "awsCredentialsProvider")
	public AWSCredentialsProvider getAWSCredentials() {
	BasicAWSCredentials awsCredentials = new BasicAWSCredentials(this.awsKeyId, this.awsKeySecret);
	return new AWSStaticCredentialsProvider(awsCredentials);
	}

	@Bean(name = "awsS3AudioBucket")
	public String getAWSS3AudioBucket() {
	return awsS3AudioBucket;
	}
	}

	public interface AmazonS3ClientService
	{
	void uploadFileToS3Bucket(MultipartFile multipartFile, boolean enablePublicReadAccess);

	void deleteFileFromS3Bucket(String fileName);
	}

	import com.amazonaws.services.s3.model.CannedAccessControlList;
	import com.amazonaws.services.s3.model.DeleteObjectRequest;
	import com.amazonaws.services.s3.model.PutObjectRequest;
	import com.springbootdev.amazon.s3.example.aws.service.AmazonS3ClientService;
	import org.slf4j.Logger;
	import org.slf4j.LoggerFactory;
	import org.springframework.beans.factory.annotation.Autowired;
	import org.springframework.scheduling.annotation.Async;
	import org.springframework.stereotype.Component;

	import com.amazonaws.AmazonServiceException;
	import com.amazonaws.auth.AWSCredentialsProvider;
	import com.amazonaws.regions.Region;
	import com.amazonaws.services.s3.AmazonS3;
	import com.amazonaws.services.s3.AmazonS3ClientBuilder;
	import org.springframework.web.multipart.MultipartFile;

	import java.io.File;
	import java.io.FileOutputStream;
	import java.io.IOException;

	@Component
	public class AmazonS3ClientServiceImpl implements AmazonS3ClientService
	{
	private String awsS3AudioBucket;
	private AmazonS3 amazonS3;
	private static final Logger logger = LoggerFactory.getLogger(AmazonS3ClientServiceImpl.class);

	@Autowired
	public AmazonS3ClientServiceImpl(Region awsRegion, AWSCredentialsProvider awsCredentialsProvider, String awsS3AudioBucket)
	{
	this.amazonS3 = AmazonS3ClientBuilder.standard()
	.withCredentials(awsCredentialsProvider)
	.withRegion(awsRegion.getName()).build();
	this.awsS3AudioBucket = awsS3AudioBucket;
	}

	@Async
	public void uploadFileToS3Bucket(MultipartFile multipartFile, boolean enablePublicReadAccess)
	{
	String fileName = multipartFile.getOriginalFilename();

	try {
	//creating the file in the server (temporarily)
	File file = new File(fileName);
	FileOutputStream fos = new FileOutputStream(file);
	fos.write(multipartFile.getBytes());
	fos.close();

	PutObjectRequest putObjectRequest = new PutObjectRequest(this.awsS3AudioBucket, fileName, file);

	if (enablePublicReadAccess) {
	putObjectRequest.withCannedAcl(CannedAccessControlList.PublicRead);
	}
	this.amazonS3.putObject(putObjectRequest);
	//removing the file created in the server
	file.delete();
	} catch (IOException \| AmazonServiceException ex) {
	logger.error("error [" + ex.getMessage() + "] occurred while uploading [" + fileName + "] ");
	}
	}

	@Async
	public void deleteFileFromS3Bucket(String fileName)
	{
	try {
	amazonS3.deleteObject(new DeleteObjectRequest(awsS3AudioBucket, fileName));
	} catch (AmazonServiceException ex) {
	logger.error("error [" + ex.getMessage() + "] occurred while removing [" + fileName + "] ");
	}
	}
	}

	import com.springbootdev.amazon.s3.example.aws.service.AmazonS3ClientService;
	import org.springframework.beans.factory.annotation.Autowired;
	import org.springframework.web.bind.annotation.*;
	import org.springframework.web.multipart.MultipartFile;

	import java.util.HashMap;
	import java.util.Map;

	@RestController
	@RequestMapping("/files")
	public class FileHandlerController {

	@Autowired
	private AmazonS3ClientService amazonS3ClientService;

	@PostMapping
	public Map<String, String> uploadFile(@RequestPart(value = "file") MultipartFile file)
	{
	this.amazonS3ClientService.uploadFileToS3Bucket(file, true);

	Map<String, String> response = new HashMap<>();
	response.put("message", "file [" + file.getOriginalFilename() + "] uploading request submitted successfully.");

	return response;
	}

	@DeleteMapping
	public Map<String, String> deleteFile(@RequestParam("file_name") String fileName)
	{
	this.amazonS3ClientService.deleteFileFromS3Bucket(fileName);

	Map<String, String> response = new HashMap<>();
	response.put("message", "file [" + fileName + "] removing request submitted successfully.");

	return response;
	}
	}

Introduction

Spring Boot Application and Amazon S3 Cloud

Amazon S3 Account Configuration

Sign up with Amazon

Creating S3 bucket

Create an IAM User

Spring Boot Amazon S3 client application.

Final Project Structure

Add Amazon SDK for the project

application.properties

AmazonS3Config.java

AmazonS3ClientService.java

AmazonS3ClientServiceImpl.java

FileHandlerController.java

Testing Application

Lets upload a file.

Lets remove a file

The Source Code

Share this:

Like this:

Leave a Reply Cancel reply