Easy Elixir auto scaling deployment with AWS and Travis CI

I’m one of those developers that finds excuses for not setting up my own production environments, advocating services such as Heroku where I don’t need to know the details about what’s actually running my code. I blame this in some degree to Rails’ myriad of hosting technologies. When I started fcgi was the thing, then some new app server would appear every year with new requirements, benefits and drawbacks. On top of that you had to set up a proxy web server, and of course have the correct version of Ruby installed.

Lately at Varvet we’ve started building API backends in Elixir as an alternative to Ruby. It was Elixir’s functional elegance and its strong concurrency capabilities that made us pick it up. The fact that Elixir applications also are super convenient to host came as a bonus. Phoenix uses cowboy to handle what Ruby require two separate moving parts to do (app and web server). Not only that, it allows it to be bundled with your application into one single binary with no run-time dependencies!

Build local release

We assume that you’ve generated a Phoenix app and got it running with mix phoenix.server which in short involves.

We then need to add the mix package exrm by adding it to mix.exs:

def deps do
  […, {:exrm, "~> 1.0.6"}]
end

You should now be able to release your app with:

MIX_ENV=prod mix do compile, release
rel/my_app/bin/my_app console # to see that it works

Build compatible release

You should now have a binary that contains everything you need to run the application. The only catch is that it’s compiled for your OS which probably isn’t the same as your server’s. But what about your CI server? Travis CI runs on Linux, let’s use that!

S3

Sign up for AWS and create an S3 bucket, we’ll name it “my-app-builds”. Then create an IAM user with write-only access to “my-app-builds”, and add its secret access key to Travis’s repository settings in order to use their artifact feature to upload our builds.

Configure .travis.yml

language: elixir
elixir:
  - 1.2.6
otp_release:
  - 18.1
addons:
  s3_region: "us-east-1"
  artifacts:
    key: iam_write_key
    bucket: my-app-builds
    region: "us-east-1"
    paths:
      - ./my_app_prod.tar.gz
    target_paths: "$TRAVIS_COMMIT"
script:
  - mix test
after_success:
  - MIX_ENV=prod mix do compile, release
  - cp rel/my_app/releases/*/my_app.tar.gz my_app_prod.tar.gz

We’ll be building the app with our prod config, but you can easily add e.g. a staging build and have Travis build them both for you. Make sure you’ve told Travis to build your repo, commit and push and you should see the build in your S3 bucket after a few minutes.

Boot a server

The first step is to create an IAM EC2 role, we’ll call it MyAppBuildDownloader. When it’s created, edit it and attach a read-only policy. Now launch an AWS EC2 instance and give it the role you just created to host our application. Choose an Ubuntu AMI (e.g. ami-fce3c696).

AWS setup

We need something to start and restart our application if something happens. On Ubuntu this is Upstart.

sudo vi /etc/init/my_app.conf


description "my_app"

setuid ubuntu
setgid ubuntu

start on startup
stop on shutdown

respawn

env MIX_ENV=prod
env PORT=8080
env HOME=/var/apps/my_app

console log

export MIX_ENV
export PORT
export HOME

exec /bin/sh /var/apps/my_app/bin/my_app foreground


sudo apt-get update && sudo apt-get install -y awscli
mkdir ~/builds
sudo mkdir -p /var/apps/my_app
sudo chown ubuntu:ubuntu -R /var/apps/my_app/

To control which build to download we’re simply going to put a file on S3 at “my-app-builds/sha” containing the git commit sha that we want to deploy. Now we can download the build from S3 and start the app:

aws s3 cp s3://my-app-builds/sha /home/ubuntu/sha --region=us-east-1
aws s3 cp s3://my-app-builds/`cat /home/ubuntu/sha` /home/ubuntu/builds --recursive --include "*my_app_prod.tar.gz" --region=us-east-1
tar -xzf /home/ubuntu/builds/my_app_prod.tar.gz -C /var/apps/my_app
sudo initctl start my_app

Run curl http://localhost:8080/api/health to verify that the app started successfully.

Auto scaling!

First we need to add a load balancer, it should forward HTTP (port 80) and HTTPS (port 443) to port 8080 and be internet-facing. Add your only instance to it and check that you can reach your app on the load balancer’s DNS e.g: curl http://my-app-prod-lb-<some id>.eu-west-1.elb.amazonaws.com/api/health

Now that we know we have one healthy instance we can duplicate it. Select your instance in the AWS console and select “Actions”“Image”“Create Image”. Allow the instance to reboot and wait for the image to appear under AMIs.

Next use “Create launch configuration” under “Auto scaling” and choose your new AMI. Under “Configure details”“Advanced details”“User data” add:

#!/bin/bash
aws s3 cp s3://my-app-builds/sha /home/ubuntu/sha --region=us-east-1
aws s3 cp s3://my-app-builds/`cat /home/ubuntu/sha` /home/ubuntu/builds --recursive --include "*my_app_prod.tar.gz" --region=us-east-1
tar -xzf /home/ubuntu/builds/my_app_prod.tar.gz -C /var/apps/my_app
sudo initctl start my_app

Click “Create Auto Scaling group” and choose your launch configuration. You can set up scaling policies if you like but we’ll start by setting the number of desired instances to 1. Now remove the your old instance from the load balancer and wait for the auto scaling group member to boot try curl http://my-app-prod-lb--<some id>.eu-west-1.elb.amazonaws.com/health again.

Scaling is now a matter of changing a number (or setting up a scaling policy if you want to completely checkout for vacation). Your newly booted servers will always run your specified build and be added to your load balancer, ready to receive traffic in a matter of seconds!

Resources

I’ve created an example Phoenix app to reference. There shouldn’t be any special config required for this setup but if you get stuck please compare your Phoenix config with my repo and let me know if you’re still having difficulties.

It is possible to script pretty much everything with AWS but that’s not always required. The downside of using the GUI web console is that it can be tricky do document all the steps. I’ve gathered a number of print screens of my setup here to reference.

This post doesn’t mention much about deployment. I’ve been using Capistrano to instruct the servers to download new builds and restart their Erlang BEAM process. You can use the AWS SDK to download the IPs/domains for all your current servers with something like this script. There are however proper Elixir alternatives that I have yet to test such as edeliver and dicon.

Versions etc

Erlang/OTP 18.1 \ Elixir 1.2.6 \ AWS region: us-east-1 \ Ruby 2.2.3 \ aws-cli 1.2.9 \ Python 3.4.3 \ Ubuntu 14.04.3 \ upstart 1.12.1 \ cloud-init 0.7.5

Epilogue

If you have any feedback on how this can be improved, or if you spot any errors, please let me know by posting a comment below!