Fast AWS Amplify Console rollbacks with blue/green deploys

I’m a big fan of AWS Amplify Console for hosting static applications on AWS without having to manage your own pipelines, S3 buckets, CloudFront distributions, and so on.

One feature that’s missing is the ability to quickly rollback to a previous build. If you deploy a broken application, the only way to restore a previous version is to go through your full deploy process which may take 5–10 minutes — and will feel a lot longer if production is down. We’ve had a lot of communication with AWS about this issue, and are hopeful that they’ll release native rollback support sometime soon (amplify-console#307).

Through this process we worked with our AWS TAM Sumeet Mankoo and SA Arundeep Nagaraj to come up with a workaround to enable fast rollbacks in exchange for some extra build costs and complexity in our Git workflows. By connecting multiple branches to Amplify we can have multiple versions of our application always available — similar to a blue/green deployment approach. Switching applications between branches is a fast operation, so we can go between our current HEAD deploy, and swap to HEAD-1 for a rollback.

Our application requires linear history in Github for our production branch which makes this pretty simple. I haven’t figured out if there’s a good way to manage these rollback branches for branches which include merge commits or force pushes to mess with their history.

As part of our deploy job in Jenkins, I added the following to push the commit for rollback:

echo "Pushing previous commit to main-minus-one branch for instant rollback: \$(git rev-parse HEAD^1)"
git push https://\${githubToken}:x-oauth-basic@github.com/org/repo \$(git rev-parse HEAD^1):main-minus-one --force

Once you’ve connected multiple branches in Amplify Console, and automated the git operations to push the correct commits, all that remains is to update the branch/domain association to rollback. This can be done in a few ways, but here’s an example using the update-domain-association AWS CLI command:

aws amplify update-domain-association --app-id abcde12345 --domain-name example.com --sub-domain-settings prefix=www,branchName=main-minus-one

Then when you’ve pushed a fix up to main you can swap back:

aws amplify update-domain-association --app-id abcde12345 --domain-name example.com --sub-domain-settings prefix=www,branchName=main

Swapping branches happens in under 10 seconds, which is a huge improvement on our previous 5–10 minute process. Here’s a GIF of the branch swap process (using the trusty border: 1px solid red test):

<insert gif here, medium upload failing right now…>

If you need to support rollbacks for multiple releases, you should be able to add any number of branches — just keep in mind that you’ll be paying $0.01 / build-minute for each of them on every deploy.