An interesting question came to my mind when I was playing around with AWS Autoscaling group setup. What is actually a better, most elegant way to deploy apps in your EC2 instances, especially in the AWS autoscaling group? To be honest, I don’t know the right question, but the alternative way of doing that to just pulling it from S3 in user data I know for sure now.

EC2 on fire

I am going to use the CI tools provided by AWS: CodeBuild and CodeDeploy. Let’s see how it works and what advantages it gives.

The code of the application, which is being deployed, you can find in github repo.

CodeBuild

Basically, what I am going to achieve in the end is that in every commit of my app in the GitHub repository, the build in the AWS CodeBuild happens and the resulting artifact is being stored in the S3 bucket.

Change application for building with AWS CodeBuild

The building process of the application could be describe by means of the special file stored in the root of the project. Let’s create one for our project.

buildspec.yml:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

version: 0.2

phases:
  install:
    runtime-versions:
      java: corretto11
  build:
    commands:
      - echo Build started on $(date)
      - gradle jar
  post_build:
    commands:
      - echo Build completed on $(date)
artifacts:
  files:
    - build/libs/simple-cookie-http-1.0-SNAPSHOT.jar
  discard-paths: true

If you ever configured the GitLab CI, you would probably be familiar with the syntax. We are saying to AWS CodeBuild to use the gradle jar to build the application. Besides the build and post_build commands, there is the whole list of other build stages, which could be used for more sophisticated scenarios. As an environment one of the CodeBuild’s provided environments is used, we are using Amazon Linux 2 - standard 3.0 particularly, which has Gradle already installed into it.

Well, the only thing, which is left for the app is to add the jar task in our gradle build:

build.gradle:

1
2
3
4
5
6
7

jar {
    manifest {
        attributes 'Main-Class': 'ServerKt'
    }

    from { configurations.compile.collect { it.isDirectory() ? it : zipTree(it) } }
}

Configure AWS CodeBuild

I want to keep this text as brief as possible, so I would skip the detailed step of configuring the AWS CodeBuild from scratch, they are quite straightforward via AWS CLI or by means of Console. I would write down, in my opinion, important items of this process and specific to my application.

The most important thing is the source of the code, which is going to be built. As the code of test project is hosted in GitHub, we are going to use it as a source.

GitHub as a code source

AWS provides good integration with gitHub, it’s very easy to configure the webhook, to have notifications from GitHub when different events happen. The simple one is the push to the master branch, exactly on this event we are going to build our project.

GitHub webpush

All the other settings are quite intuitive and easy-to-find in tutorials. The only thing, which is worth mentioning is that the resulting artifact is stored in S3 bucket simple-app-deployment-codebuild:

target_bucket

That’s it let’s push the changes in the application to the repository, and check how CodeBuild works:

first-build-history

And in the S3 bucket, we can find the jar file with our app.

CodeDeploy

Well, that was the first part of the puzzle, let’s check how we can deliver the jar-file to the instances in our AWS autoscaling group.

The AWS has a few services helping with CI duties and one of them is AWS CodeDeploy. This service could be used for different scenarios, thus a few predefined integration tutorials could be found right in the documentation, the autoscaling group deployment also. I would focus again on the crucial parts, which were important for me during practice.

Configure AWS CodeDeploy

The CodeDeploy has a few terms, which could be a bit confusing, when you see them at first time.

Application - just a general representation of your application. This thing is a root entity, all the other, more specific configs reference to it. I have created it with the type EC2/on-premise. You can see other options on the image: codedeploy-app

Deployment Group - this is the entity, which has the main power in this service, it manages where and how the deployment will be done. You have different pre-configured options for deployment strategies: all at once, half at a time, one at a time, or even create your own. Besides you can select different approaches in general like either that is going to be Blue/Green deployment or at a place. For the case of practice with the service and deploying to my existing autoscaling group, I have selected simplest all at once.

Revisions and Deployments - the last terms you should know. The revision says what should be deployed or from where. The deployment is almost the same, except that it’s actually the fact of deployment either pending or already deployed, or failed if you were not lucky. So you can register revision and then create a deployment with this revision or create deployment right away providing the location of the artifact, the revision is being created implicitly in this case.

deployement

AWS Launch template and application spec

OK, the most mysterious part is left. How the heck actually the deployment happens, how the Deployment knows what to do with the package, how it knows when the action should be done? Because if autoscaling group adds one instance to the group, the instance would get the application deployed and being run, automatically. Besides, if you register a new revision of the application, that revision will be deployed to the existing group, also automatically.

The answer is in the deployment agent. It uses the config files, supplied with the deployment pack. The file provides a description of how actually the application should be extracted and started, or terminated. There are more application events that could be configured, but we are interested in a simple application startup for the sake of simplicity here.

Let’s take a look at this file, appspec.yml:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

version: 0.0
os: linux
files:
  - source: /simple-cookie-http-1.0-SNAPSHOT.jar
    destination: /home/ec2-user/app
hooks:
  ApplicationStart:
    - location: scripts/run_app.sh
      timeiout: 300
      runas: ec2-user

It says to extract from the deployment package our jar file to the destination folder and on the event of application start, run the run_app.sh script.

run_app.sh:

1
2
3

#!/bin/bash
nohup java -jar ~/app/simple-cookie-http-1.0-SNAPSHOT.jar >~/app/app.log 2>&1 &
echo "Simple app has been started"

This ApplicationStart happens when new EC2 instance starts.

The last thing, which we need to configure before actual deployment is user data in the EC2 launch template:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

#!/bin/bash
yum update -y
yum install -y java-1.8.0
yum remove -y java-1.7.0-openjdk
yum install -y ruby

cd /home/ec2-user
mkdir app
chown -R ec2-user app
curl -O https://aws-codedeploy-eu-central-1.s3.amazonaws.com/latest/install
chmod +x ./install
./install auto

The most important is the last three rows, they are downloading, installing, and running the code deploy agent on the instance. After that the whole chain is ready to work.

I mentioned many times the deployment package, but didn’t explain what’s that. That is just a zip-archive with the appspec.yml and application (our jar-file, plus scripts). To make my life easier, I have added just one more task in the build.gradle:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

task zip (type: Zip) {
    dependsOn 'jar'

    archiveFileName = "simple-cookie-http.zip"
    destinationDirectory = file("$buildDir/dist")

    from "$buildDir/libs", "appspec.yml"
    from ("$projectDir") {
        include "scripts/*.sh"
    }
}
1

gradle zip

Now, after uploading the zip file to the S3 bucket we are ready for the first try.

Deployment test

Let’s start the fun part:

1
2
3
4
5

aws deploy create-deployment --application-name SimpleCookieApp \
--deployment-config-name CodeDeployDefault.OneAtATime \
--deployment-group-name SimpleCookieAppDG \
--description "Deployment from CLI" \
--s3-location bucket=simple-app-deployment-codebuild,bundleType=zip,key=simple-app-cookie/simple-cookie-http.zip

After a while, we should see in the deployment details this picture: first-deploy

That’s nice, let’s increase the autoscaling group( we can just increase the desired number of instance in it). A moment later a new deployment appears, pay attention to the event, which caused the deployment:

first-deploy

Well, this is what I was trying to achieve. CodeDeploy governs the version of the app being deployed in every instance and updates the instances with new versions.

Conclusion

While playing with the CI services from AWS, I caught myself thinking, I like to work with them. Yes, maybe the CodeBuild and CodeDeploy are not so nice and polish as GitLab or other CI tools, but they doing their job, doing well. Besides they have a lot of integrating tasks with AWS out from the box, which makes it really easy to configure setup in a few minutes (ok, hours).

In the case of CodeDeploy, it is even better, it knows about the AWS infrastructure much more than any CI, which should be configured manually. Don’t you think that it’s so cool to have managed the deployment process, then just pulling s3 bucket in user data of the EC2? The only thing, which I should mention, that I wouldn’t use them if actually don’t need integration with AWS Services or this integration is a small part of the application which you have. In this case, as I said, there are some a bit more mature and convenient tools to use.