Category Archives: AWS S3

Build AWS S3 Client

This post shows how to create a Java console application to interact with Amazon S3 by using AWS Java SDK. For more information, see Using the AWS SDK for Java. A simple UI allows the user to perform tasks such as list the buckets in the account, list objects in a bucket, create a bucket, create an object and so on.

This is a command line client that eliminates unnecessary clutter and shows the basic logic to interact with Amazon S3. Hopefully, this will help you to understand the syntax (and semantic ) of the API.

A separate project handles the creation of an S3 authenticated client which is allowed to access the EC2 service REST API.

You can download the code here: aws-s3-client. Please, refer to the README file for the latest example code information. You must also download the companion project at aws-auth-client and include it in the client app project.

Prerequisites

📝 You must have Maven installed. The dependencies are satisfied by building the Maven package.
– 🚨 Also, assure to download the [aws-client-auth](https://github.com/milexm/aws-client-auth) project and include it in this client app project.
– 📝 If you use Eclipse to build the application (why not?) follow the steps describe at: GCP Cloud Service Client Apps – Common Tasks.

Application Internals

Application Class Diagram

The following is the application class diagram.

Application Workflow

The following figure shows the application event trace.

aws s3 client event trace

The first time the user starts the application, the Main class performs the following actions:

  • Creates an authorized S3 client
  • Initializes the operations classes
  • Initializes the SimpleUI class
  • Starts the loop to process user inputs

The SimpleUI class loops to process the user’s commands until the loop is exited. At that point, the application terminates.

Modify the pom.xml File

A key step in creating the application project is to configure the pom.xml file correctly to define the dependencies required to implement the client application. You can find the file at: pom.xml.

Application Components

This section describes the components of the application and delegates the details to the actual code implementation.

  1. Main.  Gets authorization to access the S3 service, initializes the command classes. Delegates to the SimpleUI class the display of the selection menu and the processing of the user’s input.
  2. SimpleUI. Displays the menu of choices for the user. It processes the user’s input and calls the proper function based on the user’s selection. Each function calls the related AWS S3 Java library method (which in turn calls the related REST API).
  3. AwsClientAuthentication. Creates Amazon S3 authenticated client.
  4. BucketOperations. Contains methods to perform S3 Bucket operations. The following code shows how to create a bucket, for example.
    public static void CreateBucket(String bucketName) throws IOException {        
    
      try {
            System.out.println("Creating bucket " + bucketName + "\n");
            // Create the bucket.
              s3Client.createBucket(bucketName);
          }
          catch (AmazonServiceException ase) {
            StringBuffer err = new StringBuffer();
            err.append(("Caught an AmazonServiceException, which means your request made it "
                         + "to Amazon S3, but was rejected with an error response for some reason."));
            err.append(String.format("%n Error Message:  %s %n", ase.getMessage()));
            err.append(String.format(" HTTP Status Code: %s %n", ase.getStatusCode()));
            err.append(String.format(" AWS Error Code: %s %n", ase.getErrorCode()));
            err.append(String.format(" Error Type: %s %n", ase.getErrorType()));
            err.append(String.format(" Request ID: %s %n", ase.getRequestId()));
         }
         catch (AmazonClientException ace) {
                System.out.println("Caught an AmazonClientException, which means the client encountered "
                  + "a serious internal problem while trying to communicate with S3, "
                  + "such as not being able to access the network.");
                System.out.println("Error Message: " + ace.getMessage());
         }
    }
    
  5. ObjectOperations . Contains methods to perform S3 Object operations. The following code shows how to list objects in a bucket, for example.
       public static void listObject(String bucketName) throws IOException {          
    
            try {
                    System.out.println("Listing objects");
    
                    ListObjectsRequest listObjectsRequest = new ListObjectsRequest()
                        .withBucketName(bucketName)
                        .withPrefix("m");
                    ObjectListing objectListing;
                    do {
                        objectListing = s3Client.listObjects(listObjectsRequest);
                        for (S3ObjectSummary objectSummary :
                            objectListing.getObjectSummaries()) {
                            System.out.println(" - " + objectSummary.getKey() + "  " +
                                    "(size = " + objectSummary.getSize() +
                                    ")");
                        }
                        listObjectsRequest.setMarker(objectListing.getNextMarker());
                    } while (objectListing.isTruncated());
            }
            catch (AmazonServiceException ase) {
                StringBuffer err = new StringBuffer();
    
                err.append(("Caught an AmazonServiceException, which means your request made it "
                      + "to Amazon S3, but was rejected with an error response for some reason."));
                err.append(String.format("%n Error Message:  %s %n", ase.getMessage()));
                err.append(String.format(" HTTP Status Code: %s %n", ase.getStatusCode()));
                err.append(String.format(" AWS Error Code: %s %n", ase.getErrorCode()));
                err.append(String.format(" Error Type: %s %n", ase.getErrorType()));
                err.append(String.format(" Request ID: %s %n", ase.getRequestId()));
    
            }
            catch (AmazonClientException ace) {
                System.out.println("Caught an AmazonClientException, which means the client encountered "
                    + "a serious internal problem while trying to communicate with S3, "
                    + "such as not being able to access the network.");
                System.out.println("Error Message: " + ace.getMessage());
            }
        }
       

Security Access Credentials

🚨 You need to set up your AWS security credentials before the sample code is able to connect to AWS. You can do this by creating a file named “credentials” in the  ~/.aws/ directory on Mac (C:\Users\USER_NAME.aws\ on Windows) and saving the following lines in the file:

[default]
    aws_access_key_id = <your access key id>;
    aws_secret_access_key = <your secret key>;

For information on how to create security credentials, see Create Access Credentials. See also Providing AWS Credentials in the AWS SDK for Java.

References

Getting Started with the AWS SDK for Java

Providing AWS Credentials in the AWS SDK for Java

Amazon S3 Documentation

Working with Amazon S3 Buckets

Working with Amazon S3 Objects

AWS Toolkit for Eclipse

Java Development Blog