diff --git a/cdk-eventbridge-stepfunction-sqs-java/README.md b/cdk-eventbridge-stepfunction-sqs-java/README.md new file mode 100644 index 000000000..d4ef2f004 --- /dev/null +++ b/cdk-eventbridge-stepfunction-sqs-java/README.md @@ -0,0 +1,155 @@ +# Amazon EventBridge -> AWS Step Functions -> SQS + +This pattern creates an Amazon EventBridge event bus, a AWS Step Functions workflow, and subscribes the AWS Step Functions workflow to be executed when a `customEvent` event is published to the event bus. The AWS Step Functions workflow executes Lambda function for business worflow. AWS Lambda can occasionally experience transient service errors. In this case, invoking Lambda results in a 500 error, such as ServiceException, AWSLambdaException, or SdkClientException. As a best practice, proactively handle these exceptions in your state machine to Retry invoking your Lambda function, or to Catch the error and move to Failed Queue for operational needs. + +Important: this application uses various AWS services and there are costs associated with these services after the Free Tier usage - please see the [AWS Pricing page](https://aws.amazon.com/pricing/) for details. You are responsible for any AWS costs incurred. No warranty is implied in this example. + +## Architecture +![Architecture diagram](docs/images/Architecture.png) + +## StateMachine Flow +![StateMachine Flow](docs/images/StateMachineFlow.svg) + +## Requirements + +* [Create an AWS account](https://portal.aws.amazon.com/gp/aws/developer/registration/index.html) if you do not already have one and log in. The IAM user that you use must have sufficient permissions to make necessary AWS service calls and manage AWS resources. +* [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html) installed and configured +* [Git Installed](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git) +* [Node.js Installed](https://nodejs.org/en/download/) +* [AWS CDK](https://docs.aws.amazon.com/cdk/v2/guide/cli.html) +* [Docker](https://docs.docker.com/get-docker/) +* [Maven](https://maven.apache.org/download.cgi) + +## Deployment Instructions + +1. Create a new directory, navigate to that directory in a terminal and clone the GitHub repository: + ``` + git clone https://github.com/aws-samples/serverless-patterns + ``` +1. Change directory to the pattern directory: + ``` + cd cdk-eventbridge-stepfunction-sqs-java/infrastructure/ + ``` +2. Inside the `cdk-eventbridge-stepfunction-sqs-java/infrastructure` directory run + ``` + cdk synth + ``` + +3. Run + ``` + cdk bootstrap + ``` + +4. From the command line, use CDK to deploy the AWS resources for the pattern. You'll be prompted to approve security related changes during the deployment. + ``` + cdk deploy + ``` +5. This command will take sometime to run. After successfully completing, it will print out a few output variables. Your output should look something like this which contain the resource names which are used for testing. +``` +Outputs: +CdkServerlessInfraStack.EventBridgeFailureDLQ = aysncdlqQueue +CdkServerlessInfraStack.EventCustomBus = CdkServerlessInfraStackcustomEventBusXXXXXXXX +CdkServerlessInfraStack.StepFunction = EventBridgeCDKStateMachine +CdkServerlessInfraStack.StepFunctionFailureEventsQueue = Step-function-failure-queue +``` +## How it works + +* Let's create two custom events through AWS Console one for failure and another one for successfull execution. + Successfull event invokes AWS Step Functions and executes lambda function successfully and completes the step function state machine successfully + Failure event invokes AWS Step Functions and executes lambda function which throws custom exception and get retried 2 times using AWS Step Functions error hanlder. After retry is exhausted, Catch handler moves the state machine failure queue. AWS Step Functions writes failure message to the AWS SQS using "SendMessage" task which can be handled for operational needs. + +## Testing + +### Send an event to the event bus to trigger the Step Functions workflow + +Let's create two custom events for success and failure execution, To do this, log into the AWS Console at [https://console.aws.amazon.com](https://console.aws.amazon.com) you can send a two custom messages separately to the event bus using the AWS Console. + +1) Go to the Amazon EventBridge service page. View a list of event buses by selecting the "Event buses" link on the left navigation menu.(e.g. `CdkServerlessInfraStackcustomEventBusXXXXXX`) + +![list of event buses](docs/images/EventBus-Home.png) + +1.a Publish "success" event + Click on the "Send Events" button at the top right. This brings up a screen that you can use to publish events to the EventBridge. + ![publish success event create screen](docs/images/SuccessEvent-Create.png) + * Event bus: Select the CustomBus (CdkServerlessInfraStackcustomEventBusXXXXXX). + * Event source: Enter `CreateEvent` - This is required field + * Detail type: Enter `CREATE` - This is required field + * Event detail: In the event detail, you should craft a JSON payload that sets the `failure` field to be false. +```json +{ + "failure": false +} +``` +1.b Click the "Send" button and the confirmation will be displayed: + +![publish success event](docs/images/SuccessEvent-Send.png) + +2.a Publish "failure" event + Click on the "Send Events" button at the top right. This brings up a screen that you can use to publish events to the EventBridge. + ![publish failure event create screen](docs/images/FailureEvent-Create.png) + * Event bus: Select the CustomBus (CdkServerlessInfraStackcustomEventBusXXXXXX). + * Event source: Enter `CreateEvent` - This is required field + * Detail type: Enter `CREATE` - This is required field + * Event detail: In the event detail, you should craft a JSON payload that sets the `failure` field to be true. +```json +{ + "failure": true +} +``` +2.b Click the "Send" button and the confirmation will be displayed: +![publish failure event](docs/images/FailureEvent-Send.png) + +### Check that the Step Functions workflow started + +In the AWS Console, navigate to the AWS Step Functions service. You should see your new state machine in the list. + +![View state machines](docs/images/StepFunction-Home.png) + +Click on your state machine, and you should see a list of executions. Click on the latest to view the result of the execution and you will notice two recent executions. + +![View Step function executions](docs/images/Stepfunction-failure-home.png) + +Click the " Succeeded" status execution to see the state machine, and details. + +![View succeeded execution detail](docs/images/Stepfunction-success-details.png) + +Click the "Failed" status execution to see the state machine, and details. After the execution the failed event is send to failed AWS SQS queue. + +![View failed execution detail](docs/images/Stepfunction-failure-detail.png) + +### Check that the Failure SQS recieved the failed messages + +In the AWS Console, navigate to the AWS SQS service. You should see your new failure queue in the list. + +![View SQS HomePage](docs/images/FailureSQS.png) + +Click the "Step-function-failure-queue", and you should see the "Send and Receive messages" on the top right side. Click the "Send and Receive messages" to navigate to "Poll message" screen. +![View SQS HomePage](docs/images/FailureSQS-Home.png) +![View SQS Send & Recieve Message](docs/images/SQS-Poll-Message-Home.png) + +Click the "Poll for messages", and you should see the failure messages displayed on the bottom. + +![view SQS Poll Message](docs/images/SQS-Poll-Message.png) + +### Making changes + +You can customize the AWS Step Functions configuration by editing the code at `./lib/eventbridge-stepfunction-sqs-stack.ts`. To deploy changes, use the `cdk deploy` command. + +## Cleanup + +1. Delete the stack + ``` + cdk destroy + ``` + +2. Confirm the stack has been deleted. Login to the AWS console and navigate to the AWS Cloudformation service page "CdkServerlessInfraStack" is deleted or run the below + ```bash + aws cloudformation list-stacks --query "StackSummaries[?contains(StackName,'CdkServerlessInfraStack')].StackStatus" + ``` + +You should expect to see a message confirming `DELETE_COMPLETE`. + +---- +Copyright 2021 Amazon.com, Inc. or its affiliates. All Rights Reserved. + +SPDX-License-Identifier: MIT-0 \ No newline at end of file diff --git a/cdk-eventbridge-stepfunction-sqs-java/docs/events/failureEvent.json b/cdk-eventbridge-stepfunction-sqs-java/docs/events/failureEvent.json new file mode 100644 index 000000000..686c3156a --- /dev/null +++ b/cdk-eventbridge-stepfunction-sqs-java/docs/events/failureEvent.json @@ -0,0 +1,13 @@ +{ + "version": "0", + "id": "124ec64d-0675-0148-e0ca-4246a813f758", + "detail-type": "CREATE", + "source": "CustomEvent", + "account": "1111111111", + "time": "2022-10-31T23:53:37Z", + "region": "us-east-2", + "resources": [], + "detail": { + "failure": true + } +} \ No newline at end of file diff --git a/cdk-eventbridge-stepfunction-sqs-java/docs/events/successEvent.json b/cdk-eventbridge-stepfunction-sqs-java/docs/events/successEvent.json new file mode 100644 index 000000000..50f4e51e5 --- /dev/null +++ b/cdk-eventbridge-stepfunction-sqs-java/docs/events/successEvent.json @@ -0,0 +1,13 @@ +{ + "version": "0", + "id": "124ec64d-0675-0148-e0ca-4246a813f758", + "detail-type": "CREATE", + "source": "CustomEvent", + "account": "1111111111", + "time": "2022-10-31T23:53:37Z", + "region": "us-east-2", + "resources": [], + "detail": { + "failure": false + } +} \ No newline at end of file diff --git a/cdk-eventbridge-stepfunction-sqs-java/docs/images/Architecture.png b/cdk-eventbridge-stepfunction-sqs-java/docs/images/Architecture.png new file mode 100644 index 000000000..d21d86b0e Binary files /dev/null and b/cdk-eventbridge-stepfunction-sqs-java/docs/images/Architecture.png differ diff --git a/cdk-eventbridge-stepfunction-sqs-java/docs/images/EventBus-Home.png b/cdk-eventbridge-stepfunction-sqs-java/docs/images/EventBus-Home.png new file mode 100644 index 000000000..4e63d38ba Binary files /dev/null and b/cdk-eventbridge-stepfunction-sqs-java/docs/images/EventBus-Home.png differ diff --git a/cdk-eventbridge-stepfunction-sqs-java/docs/images/FailureEvent-Create.png b/cdk-eventbridge-stepfunction-sqs-java/docs/images/FailureEvent-Create.png new file mode 100644 index 000000000..f3240772e Binary files /dev/null and b/cdk-eventbridge-stepfunction-sqs-java/docs/images/FailureEvent-Create.png differ diff --git a/cdk-eventbridge-stepfunction-sqs-java/docs/images/FailureEvent-Send.png b/cdk-eventbridge-stepfunction-sqs-java/docs/images/FailureEvent-Send.png new file mode 100644 index 000000000..44421e7c4 Binary files /dev/null and b/cdk-eventbridge-stepfunction-sqs-java/docs/images/FailureEvent-Send.png differ diff --git a/cdk-eventbridge-stepfunction-sqs-java/docs/images/FailureSQS-Home.png b/cdk-eventbridge-stepfunction-sqs-java/docs/images/FailureSQS-Home.png new file mode 100644 index 000000000..bce0940fc Binary files /dev/null and b/cdk-eventbridge-stepfunction-sqs-java/docs/images/FailureSQS-Home.png differ diff --git a/cdk-eventbridge-stepfunction-sqs-java/docs/images/FailureSQS.png b/cdk-eventbridge-stepfunction-sqs-java/docs/images/FailureSQS.png new file mode 100644 index 000000000..28f6cfaae Binary files /dev/null and b/cdk-eventbridge-stepfunction-sqs-java/docs/images/FailureSQS.png differ diff --git a/cdk-eventbridge-stepfunction-sqs-java/docs/images/SQS-Poll-Message-Home.png b/cdk-eventbridge-stepfunction-sqs-java/docs/images/SQS-Poll-Message-Home.png new file mode 100644 index 000000000..0720c9746 Binary files /dev/null and b/cdk-eventbridge-stepfunction-sqs-java/docs/images/SQS-Poll-Message-Home.png differ diff --git a/cdk-eventbridge-stepfunction-sqs-java/docs/images/SQS-Poll-Message.png b/cdk-eventbridge-stepfunction-sqs-java/docs/images/SQS-Poll-Message.png new file mode 100644 index 000000000..4cb908c17 Binary files /dev/null and b/cdk-eventbridge-stepfunction-sqs-java/docs/images/SQS-Poll-Message.png differ diff --git a/cdk-eventbridge-stepfunction-sqs-java/docs/images/StateMachineFlow.svg b/cdk-eventbridge-stepfunction-sqs-java/docs/images/StateMachineFlow.svg new file mode 100644 index 000000000..4aab6babb --- /dev/null +++ b/cdk-eventbridge-stepfunction-sqs-java/docs/images/StateMachineFlow.svg @@ -0,0 +1,211 @@ + + Start Lambda: Invoke Business Execution Job Choice state Does execution successfull? Pass state Cleanup Failure Callback Succeed state Execution Succeed End SQS: SendMessage Failure Queue Fail state Execution Failed $.processedInput.transactionStatus == "f… $.processedInput.transactionStatus == "failed" Default Default , \ No newline at end of file diff --git a/cdk-eventbridge-stepfunction-sqs-java/docs/images/StepFunction-Home.png b/cdk-eventbridge-stepfunction-sqs-java/docs/images/StepFunction-Home.png new file mode 100644 index 000000000..6cb6546f6 Binary files /dev/null and b/cdk-eventbridge-stepfunction-sqs-java/docs/images/StepFunction-Home.png differ diff --git a/cdk-eventbridge-stepfunction-sqs-java/docs/images/Stepfunction-failure-detail.png b/cdk-eventbridge-stepfunction-sqs-java/docs/images/Stepfunction-failure-detail.png new file mode 100644 index 000000000..f995d11ab Binary files /dev/null and b/cdk-eventbridge-stepfunction-sqs-java/docs/images/Stepfunction-failure-detail.png differ diff --git a/cdk-eventbridge-stepfunction-sqs-java/docs/images/Stepfunction-failure-home.png b/cdk-eventbridge-stepfunction-sqs-java/docs/images/Stepfunction-failure-home.png new file mode 100644 index 000000000..7e2a8faf8 Binary files /dev/null and b/cdk-eventbridge-stepfunction-sqs-java/docs/images/Stepfunction-failure-home.png differ diff --git a/cdk-eventbridge-stepfunction-sqs-java/docs/images/Stepfunction-success-details.png b/cdk-eventbridge-stepfunction-sqs-java/docs/images/Stepfunction-success-details.png new file mode 100644 index 000000000..1ef0217f3 Binary files /dev/null and b/cdk-eventbridge-stepfunction-sqs-java/docs/images/Stepfunction-success-details.png differ diff --git a/cdk-eventbridge-stepfunction-sqs-java/docs/images/Stepfunction-success-home.png b/cdk-eventbridge-stepfunction-sqs-java/docs/images/Stepfunction-success-home.png new file mode 100644 index 000000000..ce387eccb Binary files /dev/null and b/cdk-eventbridge-stepfunction-sqs-java/docs/images/Stepfunction-success-home.png differ diff --git a/cdk-eventbridge-stepfunction-sqs-java/docs/images/SuccessEvent-Create.png b/cdk-eventbridge-stepfunction-sqs-java/docs/images/SuccessEvent-Create.png new file mode 100644 index 000000000..162541197 Binary files /dev/null and b/cdk-eventbridge-stepfunction-sqs-java/docs/images/SuccessEvent-Create.png differ diff --git a/cdk-eventbridge-stepfunction-sqs-java/docs/images/SuccessEvent-Send.png b/cdk-eventbridge-stepfunction-sqs-java/docs/images/SuccessEvent-Send.png new file mode 100644 index 000000000..ec5fdd154 Binary files /dev/null and b/cdk-eventbridge-stepfunction-sqs-java/docs/images/SuccessEvent-Send.png differ diff --git a/cdk-eventbridge-stepfunction-sqs-java/example-pattern.json b/cdk-eventbridge-stepfunction-sqs-java/example-pattern.json new file mode 100644 index 000000000..e08e00ccc --- /dev/null +++ b/cdk-eventbridge-stepfunction-sqs-java/example-pattern.json @@ -0,0 +1,56 @@ +{ + "title": "EventBridge to Step Functions with SQS failure", + "description": "This pattern creates a Step Functions workflow to send failure events to SQS", + "language": "Java", + "level": "200", + "framework": "CDK", + "introBox": { + "headline": "How it works", + "text": [ + "This pattern creates an Amazon EventBridge event bus, a AWS Step Functions workflow, and subscribes the AWS Step Functions workflow to be executed when a `customEvent` event is published to the event bus. The AWS Step Functions workflow executes Lambda function for business worflow. AWS Lambda can occasionally experience transient service errors. In this case, invoking Lambda results in a 500 error, such as ServiceException, AWSLambdaException, or SdkClientException. As a best practice, proactively handle these exceptions in your state machine to Retry invoking your Lambda function, or to Catch the error and move to Failed Queue for operational needs.", + "Important: this application uses various AWS services and there are costs associated with these services after the Free Tier usage - please see the [AWS Pricing page](https://aws.amazon.com/pricing/) for details. You are responsible for any AWS costs incurred. No warranty is implied in this example." ] + }, + "gitHub": { + "template": { + "repoURL": "https://github.com/aws-samples/serverless-patterns/tree/main/cdk-eventbridge-stepfunction-sqs-java", + "templateURL": "serverless-patterns/cdk-eventbridge-stepfunction-sqs-java", + "projectFolder": "cdk-eventbridge-stepfunction-sqs-java", + "templateFile": "infrastructure/src/main/java/com/myorg/CdkInfraStack.java" + } + }, + "resources": { + "bullets": [ + { + "text": "Getting started with the AWS CDK", + "link": "https://docs.aws.amazon.com/cdk/v2/guide/getting_started.html" + }, + { + "text": "Call Amazon SQS with Step Functions", + "link": "https://docs.aws.amazon.com/step-functions/latest/dg/connect-sqs.html" + } + ] + }, + "deploy": { + "text": [ + "cdk deploy" + ] + }, + "testing": { + "text": [ + "See the GitHub repo for detailed testing instructions." + ] + }, + "cleanup": { + "text": [ + "Delete the stack: cdk destroy." + ] + }, + "authors": [ + { + "name": "Dinesh Balaaji Prabakaran", + "image": "https://avatars.githubusercontent.com/u/43818089?s=400&u=8a1e4fabab318016c485049108a2012b760e9ff7&v=4", + "bio": "I am a Senior Technical Account Manager with AWS, I specialize in architecting and developing serverless, event-driven solutions on AWS. AWS Storage Technical Field Community member.", + "linkedin": "pdineshbalaaji" + } + ] + } \ No newline at end of file diff --git a/cdk-eventbridge-stepfunction-sqs-java/infrastructure/.gitignore b/cdk-eventbridge-stepfunction-sqs-java/infrastructure/.gitignore new file mode 100644 index 000000000..1db21f162 --- /dev/null +++ b/cdk-eventbridge-stepfunction-sqs-java/infrastructure/.gitignore @@ -0,0 +1,13 @@ +.classpath.txt +target +.classpath +.project +.idea +.settings +.vscode +*.iml + +# CDK asset staging directory +.cdk.staging +cdk.out + diff --git a/cdk-eventbridge-stepfunction-sqs-java/infrastructure/README.md b/cdk-eventbridge-stepfunction-sqs-java/infrastructure/README.md new file mode 100644 index 000000000..516ef71a2 --- /dev/null +++ b/cdk-eventbridge-stepfunction-sqs-java/infrastructure/README.md @@ -0,0 +1,18 @@ +# Welcome to your CDK Java project! + +This is a blank project for CDK development with Java. + +The `cdk.json` file tells the CDK Toolkit how to execute your app. + +It is a [Maven](https://maven.apache.org/) based project, so you can open this project with any Maven compatible Java IDE to build and run tests. + +## Useful commands + + * `mvn package` compile and run tests + * `cdk ls` list all stacks in the app + * `cdk synth` emits the synthesized CloudFormation template + * `cdk deploy` deploy this stack to your default AWS account/region + * `cdk diff` compare deployed stack with current state + * `cdk docs` open CDK documentation + +Enjoy! diff --git a/cdk-eventbridge-stepfunction-sqs-java/infrastructure/cdk.json b/cdk-eventbridge-stepfunction-sqs-java/infrastructure/cdk.json new file mode 100644 index 000000000..02013cc55 --- /dev/null +++ b/cdk-eventbridge-stepfunction-sqs-java/infrastructure/cdk.json @@ -0,0 +1,59 @@ +{ + "app": "mvn -e -q compile exec:java", + "watch": { + "include": [ + "**" + ], + "exclude": [ + "README.md", + "cdk*.json", + "target", + "pom.xml", + "src/test" + ] + }, + "context": { + "@aws-cdk/aws-lambda:recognizeLayerVersion": true, + "@aws-cdk/core:checkSecretUsage": true, + "@aws-cdk/core:target-partitions": [ + "aws", + "aws-cn" + ], + "@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true, + "@aws-cdk/aws-ec2:uniqueImdsv2TemplateName": true, + "@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true, + "@aws-cdk/aws-iam:minimizePolicies": true, + "@aws-cdk/core:validateSnapshotRemovalPolicy": true, + "@aws-cdk/aws-codepipeline:crossAccountKeyAliasStackSafeResourceName": true, + "@aws-cdk/aws-s3:createDefaultLoggingPolicy": true, + "@aws-cdk/aws-sns-subscriptions:restrictSqsDescryption": true, + "@aws-cdk/aws-apigateway:disableCloudWatchRole": true, + "@aws-cdk/core:enablePartitionLiterals": true, + "@aws-cdk/aws-events:eventsTargetQueueSameAccount": true, + "@aws-cdk/aws-iam:standardizedServicePrincipals": true, + "@aws-cdk/aws-ecs:disableExplicitDeploymentControllerForCircuitBreaker": true, + "@aws-cdk/aws-iam:importedRoleStackSafeDefaultPolicyName": true, + "@aws-cdk/aws-s3:serverAccessLogsUseBucketPolicy": true, + "@aws-cdk/aws-route53-patters:useCertificate": true, + "@aws-cdk/customresources:installLatestAwsSdkDefault": false, + "@aws-cdk/aws-rds:databaseProxyUniqueResourceName": true, + "@aws-cdk/aws-codedeploy:removeAlarmsFromDeploymentGroup": true, + "@aws-cdk/aws-apigateway:authorizerChangeDeploymentLogicalId": true, + "@aws-cdk/aws-ec2:launchTemplateDefaultUserData": true, + "@aws-cdk/aws-secretsmanager:useAttachedSecretResourcePolicyForSecretTargetAttachments": true, + "@aws-cdk/aws-redshift:columnId": true, + "@aws-cdk/aws-stepfunctions-tasks:enableEmrServicePolicyV2": true, + "@aws-cdk/aws-ec2:restrictDefaultSecurityGroup": true, + "@aws-cdk/aws-apigateway:requestValidatorUniqueId": true, + "@aws-cdk/aws-kms:aliasNameRef": true, + "@aws-cdk/aws-autoscaling:generateLaunchTemplateInsteadOfLaunchConfig": true, + "@aws-cdk/core:includePrefixInUniqueNameGeneration": true, + "@aws-cdk/aws-efs:denyAnonymousAccess": true, + "@aws-cdk/aws-opensearchservice:enableOpensearchMultiAzWithStandby": true, + "@aws-cdk/aws-lambda-nodejs:useLatestRuntimeVersion": true, + "@aws-cdk/aws-efs:mountTargetOrderInsensitiveLogicalId": true, + "@aws-cdk/aws-rds:auroraClusterChangeScopeOfInstanceParameterGroupWithEachParameters": true, + "@aws-cdk/aws-appsync:useArnForSourceApiAssociationIdentifier": true, + "@aws-cdk/aws-rds:preventRenderingDeprecatedCredentials": true + } +} diff --git a/cdk-eventbridge-stepfunction-sqs-java/infrastructure/pom.xml b/cdk-eventbridge-stepfunction-sqs-java/infrastructure/pom.xml new file mode 100644 index 000000000..7bcdd0c12 --- /dev/null +++ b/cdk-eventbridge-stepfunction-sqs-java/infrastructure/pom.xml @@ -0,0 +1,61 @@ + + + 4.0.0 + + com.myorg + serverless-pattern-cdk + 0.1 + + + UTF-8 + 2.103.1 + [10.0.0,11.0.0) + 5.7.1 + + + + + + org.apache.maven.plugins + maven-compiler-plugin + 3.11.0 + + 17 + + + + + org.codehaus.mojo + exec-maven-plugin + 3.1.0 + + com.myorg.CdkInfraApp + + + + + + + + + software.amazon.awscdk + aws-cdk-lib + ${cdk.version} + + + + + software.constructs + constructs + ${constructs.version} + + + + org.junit.jupiter + junit-jupiter + ${junit.version} + test + + + diff --git a/cdk-eventbridge-stepfunction-sqs-java/infrastructure/src/main/java/com/myorg/CdkInfraApp.java b/cdk-eventbridge-stepfunction-sqs-java/infrastructure/src/main/java/com/myorg/CdkInfraApp.java new file mode 100644 index 000000000..56fca607c --- /dev/null +++ b/cdk-eventbridge-stepfunction-sqs-java/infrastructure/src/main/java/com/myorg/CdkInfraApp.java @@ -0,0 +1,40 @@ +package com.myorg; + +import software.amazon.awscdk.App; +import software.amazon.awscdk.Environment; +import software.amazon.awscdk.StackProps; + +public class CdkInfraApp { + public static void main(final String[] args) { + App app = new App(); + + new CdkInfraStack(app, "CdkServerlessInfraStack", StackProps.builder() + // If you don't specify 'env', this stack will be environment-agnostic. + // Account/Region-dependent features and context lookups will not work, + // but a single synthesized template can be deployed anywhere. + + // Uncomment the next block to specialize this stack for the AWS Account + // and Region that are implied by the current CLI configuration. + + // .env(Environment.builder() + // .account(System.getenv("CDK_DEFAULT_ACCOUNT")) + // .region(System.getenv("CDK_DEFAULT_REGION")) + // .build()) + + + // Uncomment the next block if you know exactly what Account and Region you + // want to deploy the stack to. + /* + .env(Environment.builder() + .account("123456789012") + .region("us-east-1") + .build()) + */ + + // For more information, see https://docs.aws.amazon.com/cdk/latest/guide/environments.html + .build()); + + app.synth(); + } +} + diff --git a/cdk-eventbridge-stepfunction-sqs-java/infrastructure/src/main/java/com/myorg/CdkInfraStack.java b/cdk-eventbridge-stepfunction-sqs-java/infrastructure/src/main/java/com/myorg/CdkInfraStack.java new file mode 100644 index 000000000..83e4f7842 --- /dev/null +++ b/cdk-eventbridge-stepfunction-sqs-java/infrastructure/src/main/java/com/myorg/CdkInfraStack.java @@ -0,0 +1,172 @@ +package com.myorg; + +import static java.util.Collections.singletonList; +import static software.amazon.awscdk.BundlingOutput.ARCHIVED; + +import java.util.Arrays; +import java.util.List; + +import software.amazon.awscdk.BundlingOptions; +import software.amazon.awscdk.CfnOutput; +import software.amazon.awscdk.CfnOutputProps; +import software.amazon.awscdk.DockerVolume; +import software.amazon.awscdk.Duration; +import software.amazon.awscdk.Stack; +import software.amazon.awscdk.StackProps; +import software.amazon.awscdk.services.events.EventBus; +import software.amazon.awscdk.services.events.EventPattern; +import software.amazon.awscdk.services.events.Rule; +import software.amazon.awscdk.services.events.RuleProps; +import software.amazon.awscdk.services.events.targets.SfnStateMachine; +import software.amazon.awscdk.services.events.targets.SfnStateMachineProps; +import software.amazon.awscdk.services.lambda.Code; +import software.amazon.awscdk.services.lambda.Function; +import software.amazon.awscdk.services.lambda.FunctionProps; +import software.amazon.awscdk.services.lambda.Runtime; +import software.amazon.awscdk.services.logs.RetentionDays; +import software.amazon.awscdk.services.s3.assets.AssetOptions; +import software.amazon.awscdk.services.sqs.Queue; +import software.amazon.awscdk.services.sqs.QueueProps; +import software.amazon.awscdk.services.stepfunctions.CatchProps; +import software.amazon.awscdk.services.stepfunctions.Choice; +import software.amazon.awscdk.services.stepfunctions.Condition; +import software.amazon.awscdk.services.stepfunctions.Fail; +import software.amazon.awscdk.services.stepfunctions.Pass; +import software.amazon.awscdk.services.stepfunctions.RetryProps; +import software.amazon.awscdk.services.stepfunctions.StateMachine; +import software.amazon.awscdk.services.stepfunctions.Succeed; +import software.amazon.awscdk.services.stepfunctions.TaskInput; +import software.amazon.awscdk.services.stepfunctions.tasks.LambdaInvoke; +import software.amazon.awscdk.services.stepfunctions.tasks.SqsSendMessage; +import software.amazon.awscdk.services.stepfunctions.tasks.SqsSendMessageProps; +import software.constructs.Construct; + +public class CdkInfraStack extends Stack { + + public CdkInfraStack(final Construct scope, final String id) { + this(scope, id, null); + } + + public CdkInfraStack(final Construct scope, final String id, final StackProps props) { + super(scope, id, props); + + // The code that defines your stack goes here + + List functionOnePackagingInstructions = Arrays.asList("/bin/sh", "-c", + "cd lambda-handler " + "&& mvn clean install " + + "&& cp /asset-input/lambda-handler/target/lambda-handler.jar /asset-output/"); + + BundlingOptions.Builder builderOptions = BundlingOptions.builder().command(functionOnePackagingInstructions) + .image(Runtime.JAVA_11.getBundlingImage()).volumes(singletonList( + // Mount local .m2 repo to avoid download all the dependencies again inside the + // container + DockerVolume.builder().hostPath(System.getProperty("user.home") + "/.m2/") + .containerPath("/root/.m2/").build())) + .user("root").outputType(ARCHIVED); + + Function executionLambda = new Function(this, "lambda-handler", + FunctionProps.builder().runtime(Runtime.JAVA_11) + .code(Code.fromAsset("../software/", AssetOptions.builder() + .bundling(builderOptions.command(functionOnePackagingInstructions).build()).build())) + .handler("com.myorg.App").memorySize(1024).timeout(Duration.seconds(900)) + .logRetention(RetentionDays.ONE_WEEK).build()); + + // Create a Succeed state + Succeed succeeded = Succeed.Builder.create(this, "Execution Succeed").build(); + Fail fail = Fail.Builder.create(this, "Execution Failed").build(); + + // Create a Failure Queue + Queue failureQueue = Queue.Builder.create(this, "StepFunctionFailureQueue") + .queueName("Step-function-failure-queue").build(); + // Create an SQS SendMessage task + SqsSendMessageProps failureQueueStepProps = SqsSendMessageProps.builder().queue(failureQueue) + .messageBody(TaskInput.fromJsonPathAt("$")).build(); + SqsSendMessage failureQueueStep = new SqsSendMessage(this, "FailureQueue", failureQueueStepProps); + failureQueueStep.next(fail); + + // Create a "Failure Callback" Pass state + Pass failureCallback = Pass.Builder.create(this, "FailureCallback").build(); + + // Chain the "Failure Callback" state to the failureQueueStep task + failureCallback.next(failureQueueStep); + + // Create the LambdaInvoke task + LambdaInvoke executionFunction = LambdaInvoke.Builder.create(this, "ExecutionJob") + .lambdaFunction(executionLambda).retryOnServiceExceptions(true).outputPath("$.Payload").build(); + + // Add retry configuration + executionFunction + .addRetry(RetryProps.builder().errors(Arrays.asList("Failure Exception")).maxAttempts(1).build()); + + // Add catch configuration + executionFunction.addCatch(failureCallback, CatchProps.builder().resultPath("$.message.errorMessage").build()); + + // Create a Choice state + Choice choiceState = Choice.Builder.create(this, "DoesExecutionSuccessful?").build(); + + // Define transitions + choiceState.when(Condition.stringEquals("$.processedInput.transactionStatus", "completed"), succeeded) + .otherwise(failureCallback); + + // Set the next state of the executionFunction to the Choice state + executionFunction.next(choiceState); + + // Create a Step Functions state machine + StateMachine stateMachine = StateMachine.Builder.create(this, "EventBridgeCDKStateMachine") + .stateMachineName("EventBridgeCDKStateMachine").definition(executionFunction).build(); + + // Grant lambda execution roles + executionLambda.grantInvoke(stateMachine.getRole()); + + // Create an Event Bus + EventBus customEventBus = new EventBus(this, "customEventBus"); + + // Define the event rule and event pattern + RuleProps eventRuleProps = RuleProps.builder().eventPattern(EventPattern.builder() + .detailType(Arrays.asList("CREATE", "UPDATE", "DELETE")).source(List.of("CustomEvent")).build()) + .eventBus(customEventBus).build(); + + // Create the event rule + Rule eventRule = new Rule(this, "stepfunctionexecution-rule", eventRuleProps); + + // Create an asynchronous DLQ Queue + Queue aysncdlqQueue = new Queue(this, "aysncdlqQueue", QueueProps.builder().queueName("aysncdlqQueue").build()); + + // Add a target to the event rule with a Dead Letter Queue and retry policy + SfnStateMachine sfnStateMachineTarget = new SfnStateMachine(stateMachine, + SfnStateMachineProps.builder().deadLetterQueue(aysncdlqQueue) + .maxEventAge(Duration.hours(2)).retryAttempts(3).build()); + + eventRule.addTarget(sfnStateMachineTarget); + + // Create an output for the custom event bus + new CfnOutput(this, "EventCustomBus", CfnOutputProps.builder() + .value(customEventBus.getEventBusName()) + .description("The custom Event Bus Name") + .exportName("customEventBus") + .build()); + + // Create an output for the Step Function state machine + new CfnOutput(this, "StepFunction", CfnOutputProps.builder() + .value(stateMachine.getStateMachineName()) + .description("The name of the stepfunction workflow") + .exportName("stepFunctionName") + .build()); + + // Create an output for the EventBridge failure DLQ + new CfnOutput(this, "EventBridgeFailureDLQ", CfnOutputProps.builder() + .value(aysncdlqQueue.getQueueName()) + .description("EventBridge Step function Failure innovation DLQ") + .exportName("failureDLQName") + .build()); + + // Create an output for the Step Function failure events queue + new CfnOutput(this, "StepFunctionFailureEventsQueue", CfnOutputProps.builder() + .value(failureQueue.getQueueName()) + .description("Step function Failure Events Queue") + .exportName("failureEventQueueName") + .build()); + + } + +} diff --git a/cdk-eventbridge-stepfunction-sqs-java/infrastructure/src/test/java/com/myorg/CdkTest.java b/cdk-eventbridge-stepfunction-sqs-java/infrastructure/src/test/java/com/myorg/CdkTest.java new file mode 100644 index 000000000..becbd0b6d --- /dev/null +++ b/cdk-eventbridge-stepfunction-sqs-java/infrastructure/src/test/java/com/myorg/CdkTest.java @@ -0,0 +1,26 @@ +// package com.myorg; + +// import software.amazon.awscdk.App; +// import software.amazon.awscdk.assertions.Template; +// import java.io.IOException; + +// import java.util.HashMap; + +// import org.junit.jupiter.api.Test; + +// example test. To run these tests, uncomment this file, along with the +// example resource in java/src/main/java/com/myorg/CdkStack.java +// public class CdkTest { + +// @Test +// public void testStack() throws IOException { +// App app = new App(); +// CdkStack stack = new CdkStack(app, "test"); + +// Template template = Template.fromStack(stack); + +// template.hasResourceProperties("AWS::SQS::Queue", new HashMap() {{ +// put("VisibilityTimeout", 300); +// }}); +// } +// } diff --git a/cdk-eventbridge-stepfunction-sqs-java/package.json b/cdk-eventbridge-stepfunction-sqs-java/package.json new file mode 100644 index 000000000..e3d7e7123 --- /dev/null +++ b/cdk-eventbridge-stepfunction-sqs-java/package.json @@ -0,0 +1,5 @@ +{ + "dependencies": { + "aws-cdk": "2.80.0" + } + } \ No newline at end of file diff --git a/cdk-eventbridge-stepfunction-sqs-java/software/lambda-handler/.factorypath b/cdk-eventbridge-stepfunction-sqs-java/software/lambda-handler/.factorypath new file mode 100644 index 000000000..855ecc50c --- /dev/null +++ b/cdk-eventbridge-stepfunction-sqs-java/software/lambda-handler/.factorypath @@ -0,0 +1,44 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/cdk-eventbridge-stepfunction-sqs-java/software/lambda-handler/.gitignore b/cdk-eventbridge-stepfunction-sqs-java/software/lambda-handler/.gitignore new file mode 100644 index 000000000..1db21f162 --- /dev/null +++ b/cdk-eventbridge-stepfunction-sqs-java/software/lambda-handler/.gitignore @@ -0,0 +1,13 @@ +.classpath.txt +target +.classpath +.project +.idea +.settings +.vscode +*.iml + +# CDK asset staging directory +.cdk.staging +cdk.out + diff --git a/cdk-eventbridge-stepfunction-sqs-java/software/lambda-handler/pom.xml b/cdk-eventbridge-stepfunction-sqs-java/software/lambda-handler/pom.xml new file mode 100644 index 000000000..0b22336b3 --- /dev/null +++ b/cdk-eventbridge-stepfunction-sqs-java/software/lambda-handler/pom.xml @@ -0,0 +1,105 @@ + + 4.0.0 + cdk-lambda + LambdaHandler + 1.0 + jar + Java Function handler + + 11 + 11 + 2.19.0 + UTF-8 + + + + + software.amazon.lambda + powertools-tracing + 1.12.3 + + + software.amazon.lambda + powertools-metrics + 1.12.3 + + + com.amazonaws + aws-lambda-java-core + 1.2.1 + + + com.amazonaws + aws-lambda-java-events + 3.11.0 + + + + org.apache.logging.log4j + log4j-core + ${log4j.version} + + + org.apache.logging.log4j + log4j-api + ${log4j.version} + + + + junit + junit + 4.13.2 + test + + + + + + + org.codehaus.mojo + aspectj-maven-plugin + 1.14.0 + + ${maven.compiler.source} + ${maven.compiler.target} + ${maven.compiler.target} + + + software.amazon.lambda + powertools-tracing + + + software.amazon.lambda + powertools-metrics + + + + + + + compile + + + + + + org.apache.maven.plugins + maven-shade-plugin + 3.4.0 + + false + lambda-handler + + + + package + + shade + + + + + + + \ No newline at end of file diff --git a/cdk-eventbridge-stepfunction-sqs-java/software/lambda-handler/src/main/java/com/myorg/App.java b/cdk-eventbridge-stepfunction-sqs-java/software/lambda-handler/src/main/java/com/myorg/App.java new file mode 100644 index 000000000..27faf29f1 --- /dev/null +++ b/cdk-eventbridge-stepfunction-sqs-java/software/lambda-handler/src/main/java/com/myorg/App.java @@ -0,0 +1,52 @@ +package com.myorg; +import java.io.IOException; +import java.io.InputStream; +import java.io.OutputStream; + +import com.amazonaws.services.lambda.runtime.Context; +import com.amazonaws.services.lambda.runtime.LambdaLogger; +import com.amazonaws.services.lambda.runtime.RequestStreamHandler; +import com.fasterxml.jackson.databind.JsonNode; +import com.fasterxml.jackson.databind.ObjectMapper; +import com.fasterxml.jackson.databind.node.ObjectNode; + +public class App implements RequestStreamHandler { + private static final ObjectMapper objectMapper = new ObjectMapper(); + + @Override + public void handleRequest(InputStream input, OutputStream output, Context context) throws IOException { + // Parse the input stream to JsonNode + JsonNode event = objectMapper.readTree(input); + + LambdaLogger logger = context.getLogger(); + + try { + // Check if the event indicates a failure + if (event.has("detail") && event.get("detail").has("failure") + && event.get("detail").get("failure").asBoolean()) { + throw new Exception("Failure Exception"); + } else { + // Log the request + logger.log("Request: " + objectMapper.writeValueAsString(event)); + + // Create a success response + ObjectNode response = objectMapper.createObjectNode(); + response.put("statusCode", 200); + + ObjectNode processedInput = objectMapper.createObjectNode(); + processedInput.put("transactionStatus", "completed"); + + response.set("processedInput", processedInput); + response.set("sourceEvent", event); + + // Write the response to the output stream + objectMapper.writeValue(output, response); + } + } catch (Exception e) { + // Log the error + logger.log("Error: " + e.getMessage()); + + throw new RuntimeException(e); + } + } +} diff --git a/cdk-eventbridge-stepfunction-sqs-java/software/lambda-handler/src/test/java/com/myorg/CdkTest.java b/cdk-eventbridge-stepfunction-sqs-java/software/lambda-handler/src/test/java/com/myorg/CdkTest.java new file mode 100644 index 000000000..becbd0b6d --- /dev/null +++ b/cdk-eventbridge-stepfunction-sqs-java/software/lambda-handler/src/test/java/com/myorg/CdkTest.java @@ -0,0 +1,26 @@ +// package com.myorg; + +// import software.amazon.awscdk.App; +// import software.amazon.awscdk.assertions.Template; +// import java.io.IOException; + +// import java.util.HashMap; + +// import org.junit.jupiter.api.Test; + +// example test. To run these tests, uncomment this file, along with the +// example resource in java/src/main/java/com/myorg/CdkStack.java +// public class CdkTest { + +// @Test +// public void testStack() throws IOException { +// App app = new App(); +// CdkStack stack = new CdkStack(app, "test"); + +// Template template = Template.fromStack(stack); + +// template.hasResourceProperties("AWS::SQS::Queue", new HashMap() {{ +// put("VisibilityTimeout", 300); +// }}); +// } +// }