From the Docks to the Gate

This is a story about a simple web-service that answers only one question: if a given number is prime. The core problem is first solved with a Java class, which is then wrapped into a WebServlet and tested within a web server environment. The web server, however, does not get directly installed, but a docker image is created, containing all the mentioned components. Eventually, once sufficiently tested, the docker container is pushed into a container registry, from which it is deployed and run. The simple service is finally made public and available to the world, scalable, all without having to manage servers or clusters.

1. Into the Elastic Container Registry we go ..

My focus is not so much on Web-Services nor Docker, still, as a starting point, we want to create a simple Web-Service, implemented in Java and made available via Tomcat, (a pure Java HTTP web server environment in which Java code can run.) This web server will then be put into a docker container and stored at the Amazon Elastic Container Registry (ECR), a private, but fully-managed container registry that makes it easy for developers to store, manage, and deploy Docker container images.

Steps to create a docker container image and store it in ECR include:

  1. Install Docker and create a Docker Account
  2. Install the AWS and ECS Command Line Clients
  3. Create an AWS account
  4. Write the Web-Service Java-Code
  5. Create a Docker Image
  6. Push the Image into ECR

1.1. Installing Docker

There are many good introductions into Docker available. All explaining the install process and how to get started:

Before moving on, make sure you have the needed tools installed, like so:

You also need a docker account and being logged-in ..

1.2. Installing the AWS and ECS Command Line Client

Installing the AWS Command Line Client allows for quick and repeatable execution of tasks that you may otherwise do in AWS Web-UI. How to install the aws-cli depends on the operating system that is installed on your computer, but is explained in much detail over here. If you are using Mac OS and have HomeBrew, the free and open-source software package management system installed, then there is an even easier way: brew install awscli

As the result of this step, entering aws –version in terminal / command shell, should show something like this:

We will also need the ECS Command Line Client. Again, how to install it depends on your  operating system, but is explained in much detail over here. If you are using Mac OS and have HomeBrew, then there is an easier way: brew install amazon-ecs-cli

Entering ecs-cli –version in terminal, should show something like this:

1.3. Creating an AWS account / or login if you already have one

To use Amazon Web Services, you will need an AWS account, which can be created here. If you already have an account, login using the “AWS Management Console” link in the My Account menu or use your company’s single sign-on solution.

Create an IAM User

Create an IAM User in your AWS Account / or skip, if you already have any IAM User you want to use. Follow the steps here: Creating IAM Users (Console)

E.g.:

  • I used ‘edu_usr‘ for the username
  • selected ONLY Programmatic access
  • created a group ‘edu_grp
  • selected ‘PowerUserAccess‘, and also ‘IAMFullAccess‘ for a policy
  • clicked ‘Create group’
  • did not add any tags
  • clicked on “Create user’

It’s important to download the csv file, which contains the access id and key, which the awscli will need going forward.

The aws configure command collects the id, key, region, and profile name from you and then creates the hidden ~/.aws directory and in there text-files with the names credentials and config

1.4. Writing the Web-Service Code

Here is project that contains a simple Java class (Prime.java), which provides a static boolean method, returning true if a given number is prime.

The projects contains a second class (PrimeService.java), which wraps the aforementioned function into a web-service:

1.5. Creating a Docker Image

Dockerfile:  The container will be based on Tomcat 9.0.24 running on Java 11 openSDK. All default WEB apps are removed and a new ROOT.war is placed into the web container’s web-apps directory. Replacing the server.xml file, allows us to reconfigure Tomcat, to host the content on port 80 instead of the default port 8080. Finally, port 80 gets exposed and the web container is launched.

After building the project, which creates the ROOT.war file in ./build/libs, the docker images can be built with this command.

The output may look something like this:

Seeing it running

Issuing the docker-compose up -d command will launch the container and with the cool HTTPie tool installed, or with a web browser, we can see it working:

1.6. Pushing the Image into ECR

Before pushing an image into AWS’ Container Registry, a Repository needs to be created. This needs to be done only once. After that, an updated image gets pushed into the same repository.

Creating a ECR Repository

Here is the aws cli command that will create a repo with the name prime_ws

The response from this aws command should look something like this:

Tagging the Docker Image

The image now needs to be tagged with the repositoryUri, found in the previous response.

Pushing the tagged image

Finally the image can be pushed into ECR. As explained here, to allow the docker command to access AWS ECR, it needs to be authenticated.

Which results in something like this:

Subsequent changes to the code base can now be build and pushed more easily like so:

2. Heading out the Fargate

Next, we will implicitly create an Amazon Virtual Private Cloud (Amazon VPC) and use Amazon Elastic Container Service (Amazon ECS) to pull the container we created and pushed into the registry and make the service publicly available.

2.1. Creating the Task-execution Role and an empty cluster

To allow ECS to do things like logging, a role needs to be created (TASK_EXEC_ROLE) and the AmazonECSTaskExecutionRolePolicy attached to this role. We also want to create an Amazon ECS empty cluster, which can also implicitly create a VPC (Amazon Virtual Private Cloud) configured with two subnets. Moreover, to eventually allow the container to be accessed on its exposed port 80, a security group with the proper authorization needs to be created.
Executing ./setup.sh will:

  • create the Task-execution Role
  • attach the task execution role policy
  • create an Amazon ECS empty cluster, implicitly also a VPC configured with two public subnets
  • create a security Group, authorizing network access

2.2. Create/Update Docker-Compose and ECS-Params files

Just like when running the docker container locally, we need to have a compose file, but this time referring to the container resource inside ECR.

Docker-Compose File

ECS-Parameter File

The ecs-params.yml file refers to the subnets that were created when executing ./setup.sh. Update the ecs-params.yml file with the subnets that were created with the VPC. Also update the security_groups value, with the one we created above.

2.3. Launch

The Cluster in the implicitly created VPC is all set up and ready to go. We created the fargateTaskExecutionRole to allow the container to do stuff and a security role, allowing the container to be accessed via HTTP Port 80. We updated the docker-compose file, to now point to the container’s location in the ECR and put an ecs-params.yml in place, for those parameters that aren’t native to the Docker Compose files. Now it’s time to compose and bring up the service. Details about the ecs-cli compose service command can be found here.

The output is telling us the IP address, i.e., how to reach the service – and just like we did locally, we can now access the service via HTTPie or with a Web browser:

3. Preexisting Conditions

Instead of having the Amazon Elastic Container Service (Amazon ECS) implicitly create an Amazon Virtual Private Cloud (Amazon VPC), lets first create a VPC and then use Amazon ECS to pull the container from ECR and make the service publicly available.

This may sound like the less exciting, but I think there might be a good chance that you are not always in control of creating your very own VPC, but must use one that has been created for you.

3.1. Creating a VPN (or skip, if you already have one)

Running the ./vpc.sh bash script does many things. So here we go, navigating into the ./explicit directory and executing the ./vpc.sh script will:

  • create a VPC
  • add a name tag to VPC
  • create public Subnets and add name tags
  • create private Subnets and add name tags
  • create Internet gateway and attach it to the VPC
  • create Route Table
  • create route to Internet Gateway
  • associate public Subnets with Route Table
  • enable Auto-assign Public IP on Public Subnets
  • allocate Elastic IP Address for NAT Gateway
  • create NAT Gateway
  • create route to NAT Gateway
  • create a security Group
  • authorize traffic from same security group
  • create an application load balancer (alb)

3.2. Creating a Cluster

Before running the ./launch.sh script, the following variable values need to updated, to match those of the VPC, created or re-used:

While the docker-compose file remains the same, the ecs-params.yml needs to be updated, again to match the VPC. Please note that assign_public_ip needs the be set to DISABLED.

  • task_execution_role
  • subnets: (… private subnets)
  • security_groups:
  • assign_public_ip: DISABLED

With the the launch script and the parameter file updated, we can create and compose the cluster:

The launch script:

  • creates the Task-execution Role
  • attaches the task execution role policy:AmazonECSTaskExecutionRolePolicy
  • creates the cluster using the private subnets
  • creates a  Target Group (which needs a health check path)
  • creates a Listener, which forward traffic from the Load Balancer to the Target Group
  • creates a task (using the docker-compose file) and runs one instance on your cluster
  • discovers the Load Balancer’s DNS NAME and performs an HTTP GET request using the DNS NAME and health check path

Again, the last line of the script outputs how to reach the service. E.g.: http prime-alb-199922134.us-west-2.elb.amazonaws.com/isPrime/14
HTTP/1.1 200
Connection: keep-alive
Content-Length: 26
Date: Thu, 12 Sep 2019 17:32:29 GMT
14 is not a prime number.

The sky is the limit

We have arrived. Starting with a Web Service (implemented in Java and configured to become a docker-container), we ran the docker-container locally, before pushing it into the private AWS Elastic Container Registry (ECR). We then used Amazon’s Elastic Container Service (ECS) to implicitly create a Virtual Private Cloud (Amazon VPC), when it created an empty cluster. A slightly modified docker-compose file, which now pointed to the docker image in ECR, was used to create a task, which ran as one instance on the cluster.

We also created (or used and existing) Virtual Private Cloud as a starring point, which required a Target-Group to receive traffic from a load balancer and a slightly modified ecs-params.yml file, containing task parameters that aren’t native to docker compose files.

In summary, the sky is the limit is not entirely true in the case, the Amazon ECS Service has its limits and so does Fargate. No being able to attach to (opening a shell on) the running container is a limitation, but running a docker container scalable, without having to manage servers or clusters is very appealing.

Source Code

https://github.com/wolfpaulus/dock2gate

 

Share this post:

Leave a Reply