Merge pull request #1825 from pdineshbalaaji/main

New Pattern for EventBridge to Step Functions and failure events to SQS - CDK Java

cdk-eventbridge-stepfunction-sqs-java/README.md

@@ -0,0 +1,155 @@
+# Amazon EventBridge -> AWS Step Functions -> SQS
+
+This pattern creates an Amazon EventBridge event bus, a AWS Step Functions workflow, and subscribes the AWS Step Functions workflow to be executed when a `customEvent` event is published to the event bus. The AWS Step Functions workflow executes Lambda function for business worflow. AWS Lambda can occasionally experience transient service errors. In this case, invoking Lambda results in a 500 error, such as ServiceException, AWSLambdaException, or SdkClientException. As a best practice, proactively handle these exceptions in your state machine to Retry invoking your Lambda function, or to Catch the error and move to Failed Queue for operational needs. 
+
+Important: this application uses various AWS services and there are costs associated with these services after the Free Tier usage - please see the [AWS Pricing page](https://aws.amazon.com/pricing/) for details. You are responsible for any AWS costs incurred. No warranty is implied in this example.
+
+## Architecture
+![Architecture diagram](docs/images/Architecture.png)
+
+## StateMachine Flow
+![StateMachine Flow](docs/images/StateMachineFlow.svg)
+
+## Requirements
+
+* [Create an AWS account](https://portal.aws.amazon.com/gp/aws/developer/registration/index.html) if you do not already have one and log in. The IAM user that you use must have sufficient permissions to make necessary AWS service calls and manage AWS resources.
+* [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html) installed and configured
+* [Git Installed](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git)
+* [Node.js Installed](https://nodejs.org/en/download/)
+* [AWS CDK](https://docs.aws.amazon.com/cdk/v2/guide/cli.html)
+* [Docker](https://docs.docker.com/get-docker/)
+* [Maven](https://maven.apache.org/download.cgi)
+
+## Deployment Instructions
+
+1. Create a new directory, navigate to that directory in a terminal and clone the GitHub repository:
+    ``` 
+    git clone https://github.com/aws-samples/serverless-patterns
+    ```
+1. Change directory to the pattern directory:
+    ```
+    cd cdk-eventbridge-stepfunction-sqs-java/infrastructure/
+    ```
+2. Inside the `cdk-eventbridge-stepfunction-sqs-java/infrastructure` directory run 
+    ```
+    cdk synth
+    ```
+
+3. Run 
+    ```
+    cdk bootstrap
+    ```
+
+4. From the command line, use CDK to deploy the AWS resources for the pattern. You'll be prompted to approve security related changes during the deployment.
+    ```
+    cdk deploy
+    ```
+5. This command will take sometime to run. After successfully completing, it will print out a few output variables.  Your output should look something like this which contain the resource names which are used for testing.
+```
+Outputs:
+CdkServerlessInfraStack.EventBridgeFailureDLQ = aysncdlqQueue
+CdkServerlessInfraStack.EventCustomBus = CdkServerlessInfraStackcustomEventBusXXXXXXXX
+CdkServerlessInfraStack.StepFunction = EventBridgeCDKStateMachine
+CdkServerlessInfraStack.StepFunctionFailureEventsQueue = Step-function-failure-queue
+```
+## How it works
+
+* Let's create two custom events through AWS Console one for failure and another one for successfull execution. 
+  Successfull event invokes AWS Step Functions and executes lambda function successfully and completes the step function state machine successfully 
+  Failure event invokes AWS Step Functions and executes lambda function which throws custom exception and get retried 2 times using AWS Step Functions error hanlder. After retry is exhausted, Catch handler moves the state machine failure queue. AWS Step Functions writes failure message to the AWS SQS using "SendMessage" task which can be handled for operational needs. 
+
+## Testing
+
+### Send an event to the event bus to trigger the Step Functions workflow
+
+Let's create two custom events for success and failure execution, To do this, log into the AWS Console at [https://console.aws.amazon.com](https://console.aws.amazon.com) you can send a two custom messages separately to the event bus using the AWS Console. 
+
+1) Go to the Amazon EventBridge service page. View a list of event buses by selecting the "Event buses" link on the left navigation menu.(e.g. `CdkServerlessInfraStackcustomEventBusXXXXXX`)
+
+![list of event buses](docs/images/EventBus-Home.png)
+
+1.a Publish "success" event
+    Click on the "Send Events" button at the top right. This brings up a screen that you can use to publish events to the EventBridge.
+    ![publish success event create screen](docs/images/SuccessEvent-Create.png)
+    * Event bus: Select the CustomBus (CdkServerlessInfraStackcustomEventBusXXXXXX).
+    * Event source: Enter `CreateEvent` - This is required field 
+    * Detail type: Enter `CREATE` - This is required field
+    * Event detail: In the event detail, you should craft a JSON payload that sets the `failure` field to be false.
+```json
+{
+  "failure": false
+}
+```
+1.b Click the "Send" button and the confirmation will be displayed:
+
+![publish success event](docs/images/SuccessEvent-Send.png)
+
+2.a Publish "failure" event
+    Click on the "Send Events" button at the top right. This brings up a screen that you can use to publish events to the EventBridge.
+    ![publish failure event create screen](docs/images/FailureEvent-Create.png)
+    * Event bus: Select the CustomBus (CdkServerlessInfraStackcustomEventBusXXXXXX).
+    * Event source: Enter `CreateEvent` - This is required field 
+    * Detail type: Enter `CREATE` - This is required field
+    * Event detail: In the event detail, you should craft a JSON payload that sets the `failure` field to be true.
+```json
+{
+  "failure": true
+}
+```
+2.b Click the "Send" button and the confirmation will be displayed:
+![publish failure event](docs/images/FailureEvent-Send.png)
+
+### Check that the Step Functions workflow started
+
+In the AWS Console, navigate to the AWS Step Functions service. You should see your new state machine in the list.
+
+![View state machines](docs/images/StepFunction-Home.png)
+
+Click on your state machine, and you should see a list of executions. Click on the latest to view the result of the execution and you will notice two recent executions. 
+
+![View Step function executions](docs/images/Stepfunction-failure-home.png)
+
+Click the " Succeeded" status execution to see the state machine, and details. 
+
+![View succeeded execution detail](docs/images/Stepfunction-success-details.png)
+
+Click the "Failed" status execution to see the state machine, and details. After the execution the failed event is send to failed AWS SQS queue.
+
+![View failed execution detail](docs/images/Stepfunction-failure-detail.png)
+
+### Check that the Failure SQS recieved the failed messages
+
+In the AWS Console, navigate to the AWS SQS service. You should see your new failure queue in the list.
+
+![View SQS HomePage](docs/images/FailureSQS.png)
+
+Click the "Step-function-failure-queue", and you should see the "Send and Receive messages" on the top right side. Click the "Send and Receive messages" to navigate to "Poll message" screen. 
+![View SQS HomePage](docs/images/FailureSQS-Home.png)
+![View SQS Send & Recieve Message](docs/images/SQS-Poll-Message-Home.png)
+
+Click the "Poll for messages", and you should see the failure messages displayed on the bottom.
+
+![view SQS Poll Message](docs/images/SQS-Poll-Message.png)
+
+### Making changes
+
+You can customize the AWS Step Functions configuration by editing the code at `./lib/eventbridge-stepfunction-sqs-stack.ts`. To deploy changes, use the `cdk deploy` command.
+
+## Cleanup
+
+1. Delete the stack
+    ```
+    cdk destroy
+    ```
+
+2. Confirm the stack has been deleted. Login to the AWS console and navigate to the AWS Cloudformation service page "CdkServerlessInfraStack" is deleted or run the below 
+    ```bash
+    aws cloudformation list-stacks --query "StackSummaries[?contains(StackName,'CdkServerlessInfraStack')].StackStatus"
+    ```
+
+You should expect to see a message confirming `DELETE_COMPLETE`.
+
+----
+Copyright 2021 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+
+SPDX-License-Identifier: MIT-0

cdk-eventbridge-stepfunction-sqs-java/docs/events/failureEvent.json

@@ -0,0 +1,13 @@
+{
+    "version": "0",
+    "id": "124ec64d-0675-0148-e0ca-4246a813f758",
+    "detail-type": "CREATE",
+    "source": "CustomEvent",
+    "account": "1111111111",
+    "time": "2022-10-31T23:53:37Z",
+    "region": "us-east-2",
+    "resources": [],
+    "detail": {
+      "failure": true
+    }
+}

cdk-eventbridge-stepfunction-sqs-java/docs/events/successEvent.json

@@ -0,0 +1,13 @@
+{
+    "version": "0",
+    "id": "124ec64d-0675-0148-e0ca-4246a813f758",
+    "detail-type": "CREATE",
+    "source": "CustomEvent",
+    "account": "1111111111",
+    "time": "2022-10-31T23:53:37Z",
+    "region": "us-east-2",
+    "resources": [],
+    "detail": {
+      "failure": false
+    }
+}

cdk-eventbridge-stepfunction-sqs-java/example-pattern.json

@@ -0,0 +1,56 @@
+{
+    "title": "EventBridge to Step Functions with SQS failure",
+    "description": "This pattern creates a Step Functions workflow to send failure events to SQS",
+    "language": "Java",
+    "level": "200",
+    "framework": "CDK",
+    "introBox": {
+      "headline": "How it works",
+      "text": [
+        "This pattern creates an Amazon EventBridge event bus, a AWS Step Functions workflow, and subscribes the AWS Step Functions workflow to be executed when a `customEvent` event is published to the event bus. The AWS Step Functions workflow executes Lambda function for business worflow. AWS Lambda can occasionally experience transient service errors. In this case, invoking Lambda results in a 500 error, such as ServiceException, AWSLambdaException, or SdkClientException. As a best practice, proactively handle these exceptions in your state machine to Retry invoking your Lambda function, or to Catch the error and move to Failed Queue for operational needs.", 
+        "Important: this application uses various AWS services and there are costs associated with these services after the Free Tier usage - please see the [AWS Pricing page](https://aws.amazon.com/pricing/) for details. You are responsible for any AWS costs incurred. No warranty is implied in this example."    ]
+    },
+    "gitHub": {
+      "template": {
+        "repoURL": "https://github.com/aws-samples/serverless-patterns/tree/main/cdk-eventbridge-stepfunction-sqs-java",
+        "templateURL": "serverless-patterns/cdk-eventbridge-stepfunction-sqs-java",
+        "projectFolder": "cdk-eventbridge-stepfunction-sqs-java",
+        "templateFile": "infrastructure/src/main/java/com/myorg/CdkInfraStack.java"
+      }
+    },
+    "resources": {
+      "bullets": [
+        {
+          "text": "Getting started with the AWS CDK",
+          "link": "https://docs.aws.amazon.com/cdk/v2/guide/getting_started.html"
+        },
+        {
+          "text": "Call Amazon SQS with Step Functions",
+          "link": "https://docs.aws.amazon.com/step-functions/latest/dg/connect-sqs.html"
+        }
+      ]
+    },
+    "deploy": {
+      "text": [
+        "cdk deploy"
+      ]
+    },
+    "testing": {
+      "text": [
+        "See the GitHub repo for detailed testing instructions."
+      ]
+    },
+    "cleanup": {
+      "text": [
+        "Delete the stack: <code>cdk destroy</code>."
+      ]
+    },
+    "authors": [
+      {
+        "name": "Dinesh Balaaji Prabakaran",
+        "image": "https://avatars.githubusercontent.com/u/43818089?s=400&u=8a1e4fabab318016c485049108a2012b760e9ff7&v=4",
+        "bio": "I am a Senior Technical Account Manager with AWS, I specialize in architecting and developing serverless, event-driven solutions on AWS. AWS Storage Technical Field Community member.",
+        "linkedin": "pdineshbalaaji"
+      }
+    ]
+  }

cdk-eventbridge-stepfunction-sqs-java/infrastructure/.gitignore

@@ -0,0 +1,13 @@
+.classpath.txt
+target
+.classpath
+.project
+.idea
+.settings
+.vscode
+*.iml
+
+# CDK asset staging directory
+.cdk.staging
+cdk.out
+

cdk-eventbridge-stepfunction-sqs-java/infrastructure/README.md

@@ -0,0 +1,18 @@
+# Welcome to your CDK Java project!
+
+This is a blank project for CDK development with Java.
+
+The `cdk.json` file tells the CDK Toolkit how to execute your app.
+
+It is a [Maven](https://maven.apache.org/) based project, so you can open this project with any Maven compatible Java IDE to build and run tests.
+
+## Useful commands
+
+ * `mvn package`     compile and run tests
+ * `cdk ls`          list all stacks in the app
+ * `cdk synth`       emits the synthesized CloudFormation template
+ * `cdk deploy`      deploy this stack to your default AWS account/region
+ * `cdk diff`        compare deployed stack with current state
+ * `cdk docs`        open CDK documentation
+
+Enjoy!

cdk-eventbridge-stepfunction-sqs-java/infrastructure/cdk.json

@@ -0,0 +1,59 @@
+{
+  "app": "mvn -e -q compile exec:java",
+  "watch": {
+    "include": [
+      "**"
+    ],
+    "exclude": [
+      "README.md",
+      "cdk*.json",
+      "target",
+      "pom.xml",
+      "src/test"
+    ]
+  },
+  "context": {
+    "@aws-cdk/aws-lambda:recognizeLayerVersion": true,
+    "@aws-cdk/core:checkSecretUsage": true,
+    "@aws-cdk/core:target-partitions": [
+      "aws",
+      "aws-cn"
+    ],
+    "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true,
+    "@aws-cdk/aws-ec2:uniqueImdsv2TemplateName": true,
+    "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true,
+    "@aws-cdk/aws-iam:minimizePolicies": true,
+    "@aws-cdk/core:validateSnapshotRemovalPolicy": true,
+    "@aws-cdk/aws-codepipeline:crossAccountKeyAliasStackSafeResourceName": true,
+    "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true,
+    "@aws-cdk/aws-sns-subscriptions:restrictSqsDescryption": true,
+    "@aws-cdk/aws-apigateway:disableCloudWatchRole": true,
+    "@aws-cdk/core:enablePartitionLiterals": true,
+    "@aws-cdk/aws-events:eventsTargetQueueSameAccount": true,
+    "@aws-cdk/aws-iam:standardizedServicePrincipals": true,
+    "@aws-cdk/aws-ecs:disableExplicitDeploymentControllerForCircuitBreaker": true,
+    "@aws-cdk/aws-iam:importedRoleStackSafeDefaultPolicyName": true,
+    "@aws-cdk/aws-s3:serverAccessLogsUseBucketPolicy": true,
+    "@aws-cdk/aws-route53-patters:useCertificate": true,
+    "@aws-cdk/customresources:installLatestAwsSdkDefault": false,
+    "@aws-cdk/aws-rds:databaseProxyUniqueResourceName": true,
+    "@aws-cdk/aws-codedeploy:removeAlarmsFromDeploymentGroup": true,
+    "@aws-cdk/aws-apigateway:authorizerChangeDeploymentLogicalId": true,
+    "@aws-cdk/aws-ec2:launchTemplateDefaultUserData": true,
+    "@aws-cdk/aws-secretsmanager:useAttachedSecretResourcePolicyForSecretTargetAttachments": true,
+    "@aws-cdk/aws-redshift:columnId": true,
+    "@aws-cdk/aws-stepfunctions-tasks:enableEmrServicePolicyV2": true,
+    "@aws-cdk/aws-ec2:restrictDefaultSecurityGroup": true,
+    "@aws-cdk/aws-apigateway:requestValidatorUniqueId": true,
+    "@aws-cdk/aws-kms:aliasNameRef": true,
+    "@aws-cdk/aws-autoscaling:generateLaunchTemplateInsteadOfLaunchConfig": true,
+    "@aws-cdk/core:includePrefixInUniqueNameGeneration": true,
+    "@aws-cdk/aws-efs:denyAnonymousAccess": true,
+    "@aws-cdk/aws-opensearchservice:enableOpensearchMultiAzWithStandby": true,
+    "@aws-cdk/aws-lambda-nodejs:useLatestRuntimeVersion": true,
+    "@aws-cdk/aws-efs:mountTargetOrderInsensitiveLogicalId": true,
+    "@aws-cdk/aws-rds:auroraClusterChangeScopeOfInstanceParameterGroupWithEachParameters": true,
+    "@aws-cdk/aws-appsync:useArnForSourceApiAssociationIdentifier": true,
+    "@aws-cdk/aws-rds:preventRenderingDeprecatedCredentials": true
+  }
+}

cdk-eventbridge-stepfunction-sqs-java/infrastructure/pom.xml

@@ -0,0 +1,61 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<project xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"
+         xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
+    <modelVersion>4.0.0</modelVersion>
+
+    <groupId>com.myorg</groupId>
+    <artifactId>serverless-pattern-cdk</artifactId>
+    <version>0.1</version>
+
+    <properties>
+        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
+        <cdk.version>2.103.1</cdk.version>
+        <constructs.version>[10.0.0,11.0.0)</constructs.version>
+        <junit.version>5.7.1</junit.version>
+    </properties>
+
+    <build>
+        <plugins>
+            <plugin>
+                <groupId>org.apache.maven.plugins</groupId>
+                <artifactId>maven-compiler-plugin</artifactId>
+                <version>3.11.0</version>
+                <configuration>
+                    <release>17</release>
+                </configuration>
+            </plugin>
+
+            <plugin>
+                <groupId>org.codehaus.mojo</groupId>
+                <artifactId>exec-maven-plugin</artifactId>
+                <version>3.1.0</version>
+                <configuration>
+                    <mainClass>com.myorg.CdkInfraApp</mainClass>
+                </configuration>
+            </plugin>
+        </plugins>
+    </build>
+
+    <dependencies>
+        <!-- AWS Cloud Development Kit -->
+        <dependency>
+            <groupId>software.amazon.awscdk</groupId>
+            <artifactId>aws-cdk-lib</artifactId>
+            <version>${cdk.version}</version>
+        </dependency>
+
+
+        <dependency>
+            <groupId>software.constructs</groupId>
+            <artifactId>constructs</artifactId>
+            <version>${constructs.version}</version>
+        </dependency>
+
+        <dependency>
+          <groupId>org.junit.jupiter</groupId>
+          <artifactId>junit-jupiter</artifactId>
+          <version>${junit.version}</version>
+          <scope>test</scope>
+        </dependency>
+    </dependencies>
+</project>