AWS Re:invent Day 3 Highlights

Other Re:invent 2021 content:

• Full list of announcements with links to docs and blog posts
• Day 1 highlights
• Day 2 highlights
• Day 3 highlights
• Day 4 highlights

Here are my most interesting announcements from the third day of AWS Re:invent.

DevOps Guru for RDS (blog post)

Managing relational databases can be a pain, especially once you’re running a large fleet of them. DevOps Guru has gained some new functionality to monitor and alert on performance and operational issues on those databases. These are based on the metrics out of Performance Insights which provides helpful metrics on database load, query details, locks, etc.

Unfortunately, this feature is only available for Aurora databases at this time — I’m hoping for plain old RDS support soon.

DynamoDB Infrequent Access Tables (blog post)

DynamoDB added a new storage class for tables which reduces storage costs by 60% in exchange for 25% higher read and write costs:

• Storage: $0.25 per GB-month to $0.10 per GB-month
• Reads: $0.25 per million read request units -> $0.31 per million read request units
• Writes: $1.25 per million write request units -> $1.56 per million write request units

It looks like this setting is all or nothing for a whole table — would’ve been nice if you could transition rows over after some amount of time. I’ve seen some ideas for patterns where it can still be helpful:

• Time Series Tables where you have a table per time period and can transition older tables
• Using DynamoDB TTLs to clean up old data and use DynamoDB streams to replicate to an archive table using Infrequent Access

VPC Network Access Analyzer (blog post)

I suspect this one will take a while to wrap our heads around, but it seems like it’s the flip side of the Network Reachability tool AWS released last year — instead of taking two resources and determining if network connections between them can happen, it can analyze your entire account / VPC against criteria you specify (or use a pre-populated scope out of the box) and tell you all of the paths and resources between them.

AWS has also added Automated Reasoning to its research areas along with an article introducing the technology powering this new tool as well as IAM Access Analyzer, S3 Block Public Access, and VPC Reachability Analyzer.

SQS DLQ Redrive Tasks (blog post)

SQS has the concept of a Dead Letter Queue (DLQ) which is a separate queue where messages which fail to be processed successfully get redirected so that they don’t just hang around blocking up your queue all day long. You can use the DLQ to investigate what went wrong — like figuring out if there was a poison pill message with invalid data or structure.

Once you’ve figured out the issue and coded around it, you’d generally want to retry (or “redrive) those messages through your system. Until now this had to be done manually — but AWS has now built it into the console and manages the operations for you. Some #awswishlist items:

• Support for redrive to other services like EventBridge which use SQS as a DLQ
• API support for starting, stopping, and monitoring redrive tasks

Customer Carbon Footprint Tool

Not a lot of information yet on this, but it’s interesting to hear that AWS will be translating customer resource usage to carbon generation. I’m hopeful that can help to drive some change and help people do things more efficiently.

VPC IP Address Manager (IPAM)

This slipped by at first, sounding like some tool to manage large amounts of EIPs and BYOIPs (although apparently it can do that), but it can also help you manage your CIDRs and private IP pools for your VPCs across your entire org.

For those of you with existing VPCs, after you create IPAM it will start monitoring, without any action on your part, to create an inventory of all your VPCs and EIPs. Once you create pools, IPAM will then backfill your VPCs into the pool. This means you can create VPCs today, using your existing workflow, and use IPAM for monitoring and audit only. Later on, you can switch your workflow to IPAM-based automated VPC assignment.

SageMaker Updates

Yesterday was the ML keynote, so as expected there were a bunch of SageMaker updates announced (in order of interestingness):

• SageMaker Serverless Inference
• SageMaker Universal Notebook
• SageMaker Studio Lab
• SageMaker Inference Recommender

I’ll check in again with my highlights tomorrow — hoping for some Lambda/DynamoDB news from Werner Vogel’s keynote today! Looks like something else leaked a little early!