Introduction
As a developer, i am pretty sure that you may have come across with scenarios where you need to store images (either user uploaded or application itself) of your application. There are several possibilities to store file(s) as follows.
- Store the file(s) somewhere in the hosting server where the application is deployed (if it is a web application).
- Store the file(s) in the database as binary files.
- Store the file using cloud storage services.
Here we are going to evaluate the third option (given above) which is “Store the file using cloud storage services“.
Amazon Simple Storage Service (S3) is an AWS object storage platform which helps you to store the files in form of objects, and, store and retrieve any amount of data from anywhere. Each file stored in Amazon S3 (as an object) is represented using a key.
Spring Boot Application and Amazon S3 Cloud
AWS Java SDK supports various APIs related to Amazon S3 service for working with files stored in S3 bucket.
Amazon S3 Account Configuration
Please follow the instructions given in the Amazon S3 official documentation for creating and configuring the S3 account and bucket.
Click here to visit the official documentation.
I will list down the steps in below for your convenience.
Sign up with Amazon
First you need to go to https://aws.amazon.com/s3/ for sign up with Amazon S3 service. You have to follow up the instructions given in the sign up document and need to complete the registration.
AWS will notify you by email when your account is active and available for you to use. Once your account is active and ready to use, you can
Creating S3 bucket
login to https://console.aws.amazon.com/s3 and click on “Create bucket” from below screen.
Then provide the required details (e.g;- bucket name , region) and click on “Create“.
Then it will list down the newly created bucket as follows.
Create an IAM User
Go to https://console.aws.amazon.com/iam/ . In the navigation pane, choose Users and then choose Add user.
Input User name, choose Programmatic access for Access type:
Press Next: Permissions button -> go to Set permissions for peopleshrm-app-user screen.
Now, choose Attach existing policies directly -> filter policy type s3, then check AmazonS3FullAccess. Then click on “Next Review“.
Once you have reviewed the details, press “Create user“. Now you have successfully created the user.
Click on Download .csv for downloading the credentials. {Access key ID, Secret access key}.
Spring Boot Amazon S3 client application.
Final Project Structure
Your final project structure should looks like below.
Add Amazon SDK for the project
First we need to create a simple spring boot project and add the following AWS SDK dependency for the project.
<dependency> <groupId>com.amazonaws</groupId> <artifactId>aws-java-sdk</artifactId> <version>1.11.106</version> </dependency>
AWS Java SDK supports various APIs related to Amazon S3 service for working with files stored in S3 bucket.
application.properties
Add the AWS S3 bucket related configurations in the application.properties as follows. You will find those configurations details in the CSV file (Download.csv) that you have already downloaded. Change the properties
aws.access.key.id = your-access-key aws.access.key.secret = your-private-key aws.region = your-aws-service-region aws.s3.audio.bucket = your-bucket-name
AmazonS3Config.java
import org.springframework.beans.factory.annotation.Value; | |
import org.springframework.context.annotation.Bean; | |
import org.springframework.context.annotation.Configuration; | |
import com.amazonaws.auth.AWSCredentialsProvider; | |
import com.amazonaws.auth.AWSStaticCredentialsProvider; | |
import com.amazonaws.auth.BasicAWSCredentials; | |
import com.amazonaws.regions.Region; | |
import com.amazonaws.regions.Regions; | |
@Configuration | |
public class AmazonS3Config | |
{ | |
@Value("${aws.access.key.id}") | |
private String awsKeyId; | |
@Value("${aws.access.key.secret}") | |
private String awsKeySecret; | |
@Value("${aws.region}") | |
private String awsRegion; | |
@Value("${aws.s3.audio.bucket}") | |
private String awsS3AudioBucket; | |
@Bean(name = "awsKeyId") | |
public String getAWSKeyId() { | |
return awsKeyId; | |
} | |
@Bean(name = "awsKeySecret") | |
public String getAWSKeySecret() { | |
return awsKeySecret; | |
} | |
@Bean(name = "awsRegion") | |
public Region getAWSPollyRegion() { | |
return Region.getRegion(Regions.fromName(awsRegion)); | |
} | |
@Bean(name = "awsCredentialsProvider") | |
public AWSCredentialsProvider getAWSCredentials() { | |
BasicAWSCredentials awsCredentials = new BasicAWSCredentials(this.awsKeyId, this.awsKeySecret); | |
return new AWSStaticCredentialsProvider(awsCredentials); | |
} | |
@Bean(name = "awsS3AudioBucket") | |
public String getAWSS3AudioBucket() { | |
return awsS3AudioBucket; | |
} | |
} |
AmazonS3 is a class from amazon dependency. All other fields are just a representation of variables from our application.properties file. The @Value annotation will bind application properties directly to class fields during application initialization.
AmazonS3ClientService.java
public interface AmazonS3ClientService | |
{ | |
void uploadFileToS3Bucket(MultipartFile multipartFile, boolean enablePublicReadAccess); | |
void deleteFileFromS3Bucket(String fileName); | |
} |
AmazonS3ClientServiceImpl.java
import com.amazonaws.services.s3.model.CannedAccessControlList; | |
import com.amazonaws.services.s3.model.DeleteObjectRequest; | |
import com.amazonaws.services.s3.model.PutObjectRequest; | |
import com.springbootdev.amazon.s3.example.aws.service.AmazonS3ClientService; | |
import org.slf4j.Logger; | |
import org.slf4j.LoggerFactory; | |
import org.springframework.beans.factory.annotation.Autowired; | |
import org.springframework.scheduling.annotation.Async; | |
import org.springframework.stereotype.Component; | |
import com.amazonaws.AmazonServiceException; | |
import com.amazonaws.auth.AWSCredentialsProvider; | |
import com.amazonaws.regions.Region; | |
import com.amazonaws.services.s3.AmazonS3; | |
import com.amazonaws.services.s3.AmazonS3ClientBuilder; | |
import org.springframework.web.multipart.MultipartFile; | |
import java.io.File; | |
import java.io.FileOutputStream; | |
import java.io.IOException; | |
@Component | |
public class AmazonS3ClientServiceImpl implements AmazonS3ClientService | |
{ | |
private String awsS3AudioBucket; | |
private AmazonS3 amazonS3; | |
private static final Logger logger = LoggerFactory.getLogger(AmazonS3ClientServiceImpl.class); | |
@Autowired | |
public AmazonS3ClientServiceImpl(Region awsRegion, AWSCredentialsProvider awsCredentialsProvider, String awsS3AudioBucket) | |
{ | |
this.amazonS3 = AmazonS3ClientBuilder.standard() | |
.withCredentials(awsCredentialsProvider) | |
.withRegion(awsRegion.getName()).build(); | |
this.awsS3AudioBucket = awsS3AudioBucket; | |
} | |
@Async | |
public void uploadFileToS3Bucket(MultipartFile multipartFile, boolean enablePublicReadAccess) | |
{ | |
String fileName = multipartFile.getOriginalFilename(); | |
try { | |
//creating the file in the server (temporarily) | |
File file = new File(fileName); | |
FileOutputStream fos = new FileOutputStream(file); | |
fos.write(multipartFile.getBytes()); | |
fos.close(); | |
PutObjectRequest putObjectRequest = new PutObjectRequest(this.awsS3AudioBucket, fileName, file); | |
if (enablePublicReadAccess) { | |
putObjectRequest.withCannedAcl(CannedAccessControlList.PublicRead); | |
} | |
this.amazonS3.putObject(putObjectRequest); | |
//removing the file created in the server | |
file.delete(); | |
} catch (IOException | AmazonServiceException ex) { | |
logger.error("error [" + ex.getMessage() + "] occurred while uploading [" + fileName + "] "); | |
} | |
} | |
@Async | |
public void deleteFileFromS3Bucket(String fileName) | |
{ | |
try { | |
amazonS3.deleteObject(new DeleteObjectRequest(awsS3AudioBucket, fileName)); | |
} catch (AmazonServiceException ex) { | |
logger.error("error [" + ex.getMessage() + "] occurred while removing [" + fileName + "] "); | |
} | |
} | |
} |
The two methods responsible for uploading and deleting files are annotated with @Async to make them asynchronous. That means those methods will be executed in some other background thread except the main thread (the thread responsible for serving the request)
When uploading files, the below code segment is responsible for adding PublicRead permissions to the file being uploaded. It means that anyone who has the file url can access the file.
putObjectRequest.withCannedAcl(CannedAccessControlList.PublicRead);
FileHandlerController.java
import com.springbootdev.amazon.s3.example.aws.service.AmazonS3ClientService; | |
import org.springframework.beans.factory.annotation.Autowired; | |
import org.springframework.web.bind.annotation.*; | |
import org.springframework.web.multipart.MultipartFile; | |
import java.util.HashMap; | |
import java.util.Map; | |
@RestController | |
@RequestMapping("/files") | |
public class FileHandlerController { | |
@Autowired | |
private AmazonS3ClientService amazonS3ClientService; | |
@PostMapping | |
public Map<String, String> uploadFile(@RequestPart(value = "file") MultipartFile file) | |
{ | |
this.amazonS3ClientService.uploadFileToS3Bucket(file, true); | |
Map<String, String> response = new HashMap<>(); | |
response.put("message", "file [" + file.getOriginalFilename() + "] uploading request submitted successfully."); | |
return response; | |
} | |
@DeleteMapping | |
public Map<String, String> deleteFile(@RequestParam("file_name") String fileName) | |
{ | |
this.amazonS3ClientService.deleteFileFromS3Bucket(fileName); | |
Map<String, String> response = new HashMap<>(); | |
response.put("message", "file [" + fileName + "] removing request submitted successfully."); | |
return response; | |
} | |
} |
Testing Application
Lets upload a file.
Let’s test our application by making requests using Postman. We need to choose POST method, in the Body we should select ‘form-data’. As a key we should enter ‘file’ and choose value type ‘File’. Then choose any file from your local drive as a value.
As you can see that, we are trying to send a POST request to /files endpoint with file called “zuul-api-gateway.jpg“. Once the request is submitted, the file will be uploaded to the S3 bucket in the background thread (@Async method).
Once the file is uploaded (after few seconds), it should be listed under your S3 bucket as follows.
Lets remove a file
In order to delete the file, the following HTTP DELETE request to the following endpoint should be sent along with name of the file to be deleted.
DELETE http://localhost:8080/files?file_name=zuul-api-gateway.jpg
Then again go back to the Amazon S3 bucket and check whether the file exists there. You can see that file is successfully removed and no longer listed there.
In this article, we have discussed how to upload files to the Amazon S3 bucket and how to remove the stored files. If you have any query or clarification, please feel free to drop a comment or message.
The Source Code
The complete source code of this article can be found at GitHub. Click here to download.