In this workshop, you will explore several patterns for stream processing at scale in AWS. The methods used here are based on the Serverless Streaming and Architecture Best Practices White Paper.
Imagine that you are at a health care company that deploys equipment in hospitals across the globe. The equipment collects patient data, and you need to ingest and analyze the data at scale, in near real-time, in a HIPAA compliant manner.
In this workshop, you will deploy and explore the architecture below, which takes in simulated life support device sensor data and processes it. The entire architecture is serverless and uses HIPAA-eligable services.
This workshop is divided into three modules that are meant to be completed in order.
You will not deploy the architecture manually. Instead, you will be able to explore the architecture, understand architecture patterns, and apply changes as necessary.
If you are already familiar with the services in question, as an extra challenge, try to complete the workshop without reading the step-by-step instructions provided in the dropdowns.
Whatever level you're at, make sure you read the workshop fully to get the most out of it. And, if you finish ahead of time, feel free to change or add onto the pipeline to see what happens!
If you are using an account that was pre-set up for this workshop, you will have limited permissions, and may not be able to alter certain aspects of the pipeline.
| Module | Description | |
|---|---|---|
| 0 | Set Up | Deploy the architecture using a CloudFormation template. |
| 1 | Collect & De-Identify Data | Ingest simulated real-time device data into IoT Core, de-identify the data using an IoT Lambda Action, saving the PHI/PII data to an encrypted DynamoDB table while sending PHI/PII-free data on to S3 via Kinesis Firehose. |
| 2 | Enrich Data | Enrich streaming data using a Kinesis Firehose Record Transformation with metadata from DynamoDB, and then store the enriched records in S3. |
| 3 | Detect Anomalies | Use Kinesis Analytics to calculate anomaly scores with the Random Cut Forest algorithm, and automatically send an SNS text message if an anomaly is found. |
| 4 | Clean Up | Delete the entire architecture. |