Using AWS Lambda to copy RDS snapshots between regions

At work we needed to make MySQL database on RDS backups between regions without having a running instance in the destination region, I mean, no read replicas wanted. Someone suggested to use a cron to copy the backups between regions. I thought this had to been done so I decided to do a research and I ran into this excellent post that explains how to make the copy using Lambda functions with Python: Copying RDS snapshot to another region for cross-region recovery

This function get the last snapshots for all RDS databases in the source region and copies them to the destination region. Then it deletes old snapshots in the destination region to save space. The function can be triggered using CloudWatch or RDS events, for example when the database backup is finished.

Paulina Budzon, the post author, commented that the function can be improved so I made some changes:

• Added database list to be backup-ed, instead of all databases in RDS
• Changed variable naming to avoid reference to the destination region
• Removed source region example reference in SourceDBSnapshotIdentifier string
• Added variables for source and destination regions

I shared the code here but it can be got from my fork https://github.com/lgallard/aws-maintenance, or you can get it from Paulina’s https://github.com/pbudzon/aws-maintenance, because she merged my  pull request.

I hope it helps somebody else:

import boto3
 import operator

aws_account = 'XXXX'
 source = 'us-east-1'
 destination = 'sa-east-1'
 databases = ['mysqldb01', 'pgdb01']

def copy_latest_snapshot():
 client = boto3.client('rds', source)
 foreign_client = boto3.client('rds', destination)

response = client.describe_db_snapshots(
 SnapshotType='automated',
 IncludeShared=False,
 IncludePublic=False
 )

if len(response['DBSnapshots']) == 0:
 raise Exception("No automated snapshots found")

snapshots_per_project = {}

for snapshot in response['DBSnapshots']:
 if snapshot['DBInstanceIdentifier'] not in databases or snapshot['Status'] != 'available' :
 continue

if snapshot['DBInstanceIdentifier'] not in snapshots_per_project.keys():
 snapshots_per_project[snapshot['DBInstanceIdentifier']] = {}

snapshots_per_project[snapshot['DBInstanceIdentifier']][snapshot['DBSnapshotIdentifier']] = snapshot[
 'SnapshotCreateTime']

for project in snapshots_per_project:
 sorted_list = sorted(snapshots_per_project[project].items(), key=operator.itemgetter(1), reverse=True)

copy_name = project + "-" + sorted_list[0][1].strftime("%Y-%m-%d")

print("Checking if " + copy_name + " is copied")

try:
 foreign_client.describe_db_snapshots(
 DBSnapshotIdentifier=copy_name
 )
 except:
 response = foreign_client.copy_db_snapshot(
 SourceDBSnapshotIdentifier='arn:aws:rds:' + source + ':' + aws_account + ':snapshot:' + sorted_list[0][0],
 TargetDBSnapshotIdentifier=copy_name,
 CopyTags=True
 )

if response['DBSnapshot']['Status'] != "pending" and response['DBSnapshot']['Status'] != "available":
 raise Exception("Copy operation for " + copy_name + " failed!")
 print("Copied " + copy_name)

continue

print("Already copied")

def remove_old_snapshots():
 client = boto3.client('rds', source)
 foreign_client = boto3.client('rds', destination)

response = foreign_client.describe_db_snapshots(
 SnapshotType='manual'
 )

if len(response['DBSnapshots']) == 0:
 raise Exception("No manual snapshots in "+ destination + " found")

snapshots_per_project = {}
 for snapshot in response['DBSnapshots']:
 if snapshot['DBInstanceIdentifier'] not in databases or snapshot['Status'] != 'available' :
 continue

if snapshot['DBInstanceIdentifier'] not in snapshots_per_project.keys():
 snapshots_per_project[snapshot['DBInstanceIdentifier']] = {}

snapshots_per_project[snapshot['DBInstanceIdentifier']][snapshot['DBSnapshotIdentifier']] = snapshot[
 'SnapshotCreateTime']

for project in snapshots_per_project:
 if len(snapshots_per_project[project]) > 1:
 sorted_list = sorted(snapshots_per_project[project].items(), key=operator.itemgetter(1), reverse=True)
 to_remove = [i[0] for i in sorted_list[1:]]

for snapshot in to_remove:
 print("Removing " + snapshot)
 foreign_client.delete_db_snapshot(
 DBSnapshotIdentifier=snapshot
 )

def lambda_handler(event, context):
 copy_latest_snapshot()
 remove_old_snapshots()

if __name__ == '__main__':
 lambda_handler(None, None)</pre>

Reference: Copying RDS snapshot to another region for cross-region recovery