Home Assignment 1


For this assignment, you will build a basic application using Amazon’s cloud services. You will become familiar with how to launch and access virtual machines using EC2, and learn how to execute code on these machines. You will learn how to use Amazon’s Simple Queue Service (SQS), as well as Amazon’s Simple Storage Service (S3).

There are several ways of doing this assignment, including a variety of languages you can use, and a variety of ways of interacting with AWS. For example, in order to interact with AWS, you can use the AWS management console, command-line tools, SOAP APIs, REST APIs, as well as libraries in other languages. However, I will only provide nice detailed instructions on how to do this assignment in one particular way (you are welcome to do it another way, but I will be unable to help you along the way if you run into issues). Specifically, I will give guidance on how to use the AWS management console to create AWS resources, and how to use the AWS SDK for Java to interact with these resources (both from within AWS, and outside of AWS).

In order to do the assignment in the way I will outline, which uses the AWS management console for various purposes, you will have to set up an AWS account. To set up an account, you will have to provide a credit card number, but this credit card will not be charged because everything we will do in this assignment falls under the “Free Usage Tier” of AWS . However, if you wish to do some of the tutorials provided by AWS, or test some of their example code, you might end up being charged.

For a great overview of some best practices for using the Amazon Web Services I reccomend this white paper.

Note! Do not make any of your AWS keys, credentials, or even keys to your S3-buckets publicly available (e.g. GitHub), or you might end up with a juicy bill.

Deadline and Deliverables

The deadline for the homework is until the next session (24/2). Before then you should hand in a zip-file containing your workspace, and the log/text-files from your S3-bucket. You hand it in by mailing the zip-file to the TA (Victor Millnert, victor@control.lth.se).

Overview of the Application Structure

The recommended structure during the development phase of this assignment is shown below. Note how the code that is later to be run on your EC2Instance is run on you local computer during this development phase (saves a lot of time). Once everything is working, this will be moved to it’s own EC2Instance.

Once all the code is working as intended the EC2Instance Code can be moved to the Amazon Cloud, and the final structure will look like the following:

For a more thorough description of the application you can read this nice article by Amazon.

A Natural Working Order

  1. Read the entire homework manual
  2. Set up an AWS Account (instructions are given below)
  3. Set up Eclipse (instructions are given below)
  4. Study some “AWS java sample code”
  5. Create your own S3-bucket
  6. Create your own SQS-queue
  7. Start developing the Client App
  8. Start developing the EC2Instance-code
  9. Export the working EC2Instance-code to a EC2Instance
    1. Export your working EC2Instance-code as a runnable JAR-file
    2. Go online to the AWS Management Console
    3. Initiate a new Linux micro instance
    4. copy your runnable JAR-file up to your newly created EC2Instance
    5. ssh into your new EC2Instance
    6. create a /.aws/ directory to store your credentials in
      • sudo mkdir /root/.aws/
    7. copy your credentials to /home/ec2-user/ (these credentials were created during step 3.)
    8. copy your credentials from /home/ec2-user/ to /root/.aws/
    9. delete the credentials in /home/ec2-user/
    10. Run the runnable JAR-file, and see if it works
      • sudo java -jar RunnableJAR-file.jar
  10. Make the script run on start-up
    1. Log into your modified EC2Instance (created in the previous step, with the runnable JAR-file)
    2. Open /etc/rc.local using an editor of your choice (perhaps vi)
    3. After the line touch /var/lock/subsys/local add the following line:
      • java -jar /home/ec2-user/RunnableJAR-file.jar &
    4. Now the JAR-file should execute on startup of the instance
  11. Create an AMI-image from your modified EC2Instance
    1. Go online to your AWS Management Console
    2. Go to the EC2 Dashboard
    3. Select your modified EC2Instance
    4. Choose Actions -> Image -> Create
    5. After a while your new AMI-image should be visible under the AMIs-menu
  12. Launch a new Instance from your newly created AMI-image
  13. Start your Client Application and do a full test of your cloud-app
  14. Zip your workspace (including the EC2Instance-code) and the log/text-files (10 is enough) from your S3-bucket
  15. Mail it to the TA (victor@control.lth.se)
  16. Terminate all the EC2instances
  17. Delete your SQS-queues
  18. Delete your S3-bucket
  19. Destroy your AMI-image
  20. Destroy your credentials

Setting up an AWS Account

  1. To sign up for a Free-tier AWS Account just follow the instructions on https://aws.amazon.com/free/.
  2. Become acustomed with the AWS Management console. They provide a useful ‘getting started tutorial’.
  3. Play around and create a EC2Instance, an SQS-queue, and an S3-bucket.

Setting up Eclipse

If you don’t like Eclipse I’m sure you can find another way..

The Different Parts of the Application

The final structure of the application is shown again, for reference when describing the requirements of the different parts in some more depth.

Client App

Runs locally on your machine. Should do the following:

The available processes should be:

Note: You need some clever way to make sure that the result in the SQS-outbox is indeed the result to your request and not the request made by a different client app (meaning it should be possible to have several client apps).

SQS Queues

You will need one “inbox” where the requests will be put and one “outbox” for the responses.

Some notes on the SQS-queue:

// Queue Attributes
Map queueAttributes = new HashMap();
queueAttributes.put("ReceiveMessageWaitTimeSeconds", "20");
queueAttributes.put("VisibilityTimeout", "10");

* Create an Inbox and an Outbox Queue
*   - Enable SQS Long Polling. Forces the component that tries to receive a massage to wait.
*   - Set the Visibility Timeout. Ensures that only the intended can receive the message during 
*   the timeout. 
System.out.println("Creating a SQS-inbox.\n");
CreateQueueRequest createInboxQueueRequest = new CreateQueueRequest("Inbox");
String inboxUrl = sqs.createQueue(createInboxQueueRequest.withAttributes(queueAttributes)).getQueueUrl();

2+ EC2 Instances

Should be run on your local computer during the development phase, and then (once it works) moved to the Amazon Cloud. These instances read from the “inbox” queue, process the requests, put the processed object in S3 bucket, send a response message to the “outbox”-queue, and delete the message from the “inbox”-queue once they have finished processing it.

To create an EC2 Instance from a certain AMI you can do as below:

AWSCredentials credentials = null;
try {
    credentials = new ProfileCredentialsProvider("YourCredentialsName").getCredentials();
} catch (Exception e) {
    throw new AmazonClientException(
            "Cannot load the credentials from the credential profiles file. " +
            "Please make sure that your credentials file is at the correct " +
            "location (/Users/Username/.aws/credentials), and is in valid format.",

// Create an EC2Client 
AmazonEC2 ec2 = new AmazonEC2Client(credentials);
ec2.setEndpoint("ec2.eu-central-1.amazonaws.com"); // It is better to use 'setEndpoint()' than 'setRegion'  

RunInstancesRequest runInstancesRequest = new RunInstancesRequest();
// AMI for normal Linux EC2Instance (setRegion Frankfurt): ami-b43503a9
               .withInstanceType("t2.micro") // Select t2.micro to be eligible for free tier 
                   .withKeyName("YourKeyName") // Which you will use to ssh into your instance

RunInstancesResult runInstancesResult = ec2.runInstances(runInstancesRequest);

S3 Bucket

The S3 bucket will contain all of the requests/responses which the system has processed. This information should at least include: MessageID, timestamp, the three numbers, process, result (if it’s a result)

Some good info on the S3-bucket can be found here.

Note that all S3-buckets need unique names!

To create a S3-bucket the following code can be used:

String bucketName = "your-special-bucket-" + UUID.randomUUID();

 * Create a new S3 bucket - Amazon S3 bucket names are globally unique,
 * so once a bucket name has been taken by any user, you can't create
 * another bucket with that same name.
 * You can optionally specify a location for your bucket if you want to
 * keep your data closer to your applications or users.
System.out.println("Creating bucket " + bucketName + "\n");