From 83d066c8f2293661f31005e7ea2b2fd260cff126 Mon Sep 17 00:00:00 2001 From: James Hutchby Date: Tue, 25 Oct 2022 23:56:36 +0200 Subject: [PATCH] update documentation --- README.md | 137 +++++++++++++++++++++++++++++------------------------- 1 file changed, 74 insertions(+), 63 deletions(-) diff --git a/README.md b/README.md index ca36610..73f7a5a 100644 --- a/README.md +++ b/README.md @@ -10,33 +10,43 @@ It can be configured to run periodically using CloudWatch events. ## Quick start 1. Create an AWS lambda function: - - Author from scratch - - Runtime: Node.js 14.x + - Author from scratch + - Runtime: Node.js 16.x + - Architecture: x86_64 2. tab "Code" -> "Upload from" -> ".zip file": - - Upload ([pgdump-aws-lambda.zip](https://github.com/jameshy/pgdump-aws-lambda/releases/latest)) - - tab "Configuration" -> "General Configuration" -> "Edit" - - Timeout: 15 minutes - - Edit the role and attach the policy "AmazonS3FullAccess" - - Save -3. Test - - Create new test event, e.g.: - ```json - { - "PGDATABASE": "dbname", - "PGUSER": "postgres", - "PGPASSWORD": "password", - "PGHOST": "host", - "S3_BUCKET" : "db-backups", - "ROOT": "hourly-backups" - } - ``` - - *Test* and check the output - -4. Create a CloudWatch rule: - - Event Source: Schedule -> Fixed rate of 1 hour - - Targets: Lambda Function (the one created in step #1) - - Configure input -> Constant (JSON text) and paste your config (as per previous step) + - Upload ([pgdump-aws-lambda.zip](https://github.com/jameshy/pgdump-aws-lambda/releases/latest)) + - tab "Configuration" -> "General Configuration" -> "Edit" + - Timeout: 15 minutes + - Edit the role and attach the policy "AmazonS3FullAccess" + - Save +3. Give your lambda permissions permissions to write to S3: + - tab "Configuration" -> "Permissions" + - click the existing Execution role + - "Add permissions" -> "Attach policies" + - select "AmazonS3FullAccess" and click "Attach policies" + +4. Test + + - Create new test event, e.g.: + + ```json + { + "PGDATABASE": "dbname", + "PGUSER": "postgres", + "PGPASSWORD": "password", + "PGHOST": "host", + "S3_BUCKET": "db-backups", + "ROOT": "hourly-backups" + } + ``` + + - _Test_ and check the output + +5. Create a CloudWatch rule: + - Event Source: Schedule -> Fixed rate of 1 hour + - Targets: Lambda Function (the one created in step #1) + - Configure input -> Constant (JSON text) and paste your config (as per previous step) #### File Naming @@ -55,13 +65,13 @@ You can add an encryption key to your event, e.g. ```json { - "PGDATABASE": "dbname", - "PGUSER": "postgres", - "PGPASSWORD": "password", - "PGHOST": "host", - "S3_BUCKET" : "db-backups", - "ROOT": "hourly-backups", - "ENCRYPT_KEY": "c0d71d7ae094bdde1ef60db8503079ce615e71644133dc22e9686dc7216de8d0" + "PGDATABASE": "postgres", + "PGUSER": "postgres", + "PGPASSWORD": "password", + "PGHOST": "host", + "S3_BUCKET": "db-backups", + "ROOT": "hourly-backups", + "ENCRYPT_KEY": "c0d71d7ae094bdde1ef60db8503079ce615e71644133dc22e9686dc7216de8d0" } ``` @@ -88,14 +98,13 @@ Your context may require that you use IAM-based authentication to log into the P Support for this can be enabled my making your Cloudwatch Event look like this. ```json - { - "PGDATABASE": "dbname", - "PGUSER": "postgres", - "PGHOST": "host", - "S3_BUCKET" : "db-backups", - "ROOT": "hourly-backups", - "USE_IAM_AUTH": true + "PGDATABASE": "dbname", + "PGUSER": "postgres", + "PGHOST": "host", + "S3_BUCKET": "db-backups", + "ROOT": "hourly-backups", + "USE_IAM_AUTH": true } ``` @@ -111,46 +120,46 @@ NOTE: the execution role for the Lambda function must have access to GetSecretVa Support for this can be enabled by setting the SECRETS_MANAGER_SECRET_ID, so your Cloudwatch Event looks like this: ```json - { - "SECRETS_MANAGER_SECRET_ID": "my/secret/id", - "S3_BUCKET" : "db-backups", - "ROOT": "hourly-backups" + "SECRETS_MANAGER_SECRET_ID": "my/secret/id", + "S3_BUCKET": "db-backups", + "ROOT": "hourly-backups" } ``` -If you supply `SECRETS_MANAGER_SECRET_ID`, you can ommit the 'PG*' keys, and they will be fetched from your SecretsManager secret value instead with the following mapping: +If you supply `SECRETS_MANAGER_SECRET_ID`, you can ommit the 'PG\*' keys, and they will be fetched from your SecretsManager secret value instead with the following mapping: -| Secret Value | PG-Key | -| ------------- | ------------- | -| username | PGUSER | -| password | PGPASSWORD | -| dbname | PGDATABASE | -| host | PGHOST | -| port | PGPORT | +| Secret Value | PG-Key | +| ------------ | ---------- | +| username | PGUSER | +| password | PGPASSWORD | +| dbname | PGDATABASE | +| host | PGHOST | +| port | PGPORT | - -You can provide overrides in your event to any PG* keys as event parameters will take precedence over secret values. +You can provide overrides in your event to any PG\* keys as event parameters will take precedence over secret values. ## Developer #### Bundling a new `pg_dump` binary + 1. Launch an EC2 instance with the Amazon Linux 2 AMI 2. Connect via SSH and: + ```bash -# install postgres 13 +# install postgres 15 sudo amazon-linux-extras install epel sudo tee /etc/yum.repos.d/pgdg.repo<