SQS Dead Letter Requeue

Python Lambda function for handling SQS Dead Letter Queues.

Heavily inspired by @Jimdo's Golang implementation of SQS Dead Letter Handling

Requirements

• Python 2.7

Building it

Just build the Lambda zip

make build

Create an S3 bucket to store the Lambda function

make create_deploy_bucket

Shipping the packaged Lambda function

make ship

Deploy Lambda function plus scheduled event

• Use the CloudFormation template provided (cloudformation/requeue_lambda.yaml)
• Or alternatively use the Terraform module (terraform/modules/requeue_lambda)

Running it locally

Local setup/configuration

Make sure you have AWS credentials configured and a default region specified.

This can be with environment variables:

export AWS_ACCESS_KEY_ID=<my-access-key>
export AWS_SECRET_ACCESS_KEY=<my-secret-key>
export AWS_DEFAULT_REGION=<my-default-region>

or setting them in either an AWS credentials file (/.aws/credentials) or AWS config file (/.aws/config):

[default]
aws_access_key_id = <my-access-key>
aws_secret_access_key = <my-secret-key>
region = <my-default-region>

You will also need to set the name of the main queue to be requeued:

export QUEUE_NAME=<my-queue-name>

Virtualenv/dependencies

virtualenv env
. env/bin/activate
pip install -r requirements.txt

Run it

python requeue.py

Generating test data

There is an included script to generate 100 messages on to a chosen queue. Run it:

python seed_queue.py ${QUEUE_NAME}_dead_letter

TODO

• CloudFormation template that includes both Queues
• Create CloudFormation "Launch Stack" buttons