psm

Security enabled simple REST service to manage application configuration in AWS SSM Parameter Store.

Related blog post can be found on the Neiman Marcus Medium page.

Details

Cloud applications often require runtime and deployment configuration which must be managed. Following infrastructure-as-code best practices, storing this configuration alongside the application is crucial.

Psm, short for parameter store manager, aims to allow product teams, developers, and cloud architects to easily maintain and deploy configuration in a secure, reliable, and templated manner, inside source control.

Deployed with serverless framework, this application can be easily modified for any AWS environment. If you have an improvement or run into an issue please participate on the github page.

Configuration Convention

Psm assumes that application configuration is stored in a specific heirarchy in AWS SSM Parameter Store. By default psm deploys and handles configuration under /{application}/{stage}/... with each application or service having multiple stages for different environments. It is common to have a dev, staging and prod stage of an application.

Path Override

You can override the default path, by passing a hardcoded string in the x-path-override header.

Block value parser

You can override the default parser by passing a x-kv-block-parser: true in the header. This will use parent nodes of the json as keys (see example below).

eg. input json:

{
  "keyOne": {
    "DOB": "02/19/1983",
    "Phone": "+1 (214) 999999"
  },
  "keyTwo": {
    "VehicleID": "1XBYUT"
  },
  "keyThree": {
    "API_ENDPOINT": {
      "uri": "/dbcy.com/US",
      "toggles": ["ORDER_SORT_OPTION_1", "ORDER_SORT_OPTION_2"]
    }
  }
}

output ssm key value:

key	value
keyOne	{"DOB": "02/19/1983","Phone": "+1 (214) 999999"
keyTwo	{"VehicleID": "1XBYUT"}
keyThree	{"API_ENDPOINT": {"uri": "/dbcy.com/US","toggles": ["ORDER_SORT_OPTION_1","ORDER_SORT_OPTION_2"]}}

Psm functions

Psm uses the following three lambda functions:

• Encrypt - Encrypt secrets via KMS CMK to securely store in source control.
• Update - Update configuration in AWS SSM Parameter Store
• View - Retrieve and view configuration in AWS SSM Parameter Store

Encrypt

The encrypt function allows a developer to encrypt a supplied value for storage in source control. The function takes the input and encrypts it against psm's CMK. The returned value is prefixed with cipher: and is used when pushing configuration to parameter store. It is not necessary to encrypt all values in source control, only secrets.

You have the option to POST a secret to be encrypted, or to use the GET method for a encrypted, randomly generated value.

This function requires no API key.

Examples

Example API calls and .http files can be used with the REST VSCode Extension.

POST Method

POST https://abcdef0123.execute-api.us-west-2.amazonaws.com/prod/encrypt HTTP/1.1

Hello World!

GET Method

GET https://abcdef0123.execute-api.us-west-2.amazonaws.com/prod/encrypt HTTP/1.1

Example Configuration File

{
  "foo": "cipher:AQICAHjumw2gwZTBa2YnLcaUczMcoU..."
}

Update

The update function pushes configuration to SSM Parameter Store. It will crawl the json formatted data input, and if configuration is changed, push the new configuration. If the value is encrypted, using the encrypt function, and prefixed with cipher:, it will decrypt the value before comparing the parameters.

The update function uses query string parameters to place the configuration. Psm is looking for the appId or name of the application, and stage. See examples below.

This function flattens the json input data. For example, {"foo": {"bar": "buzz"}} will be transformed to the parameter key, /{application}/{stage}/foo.bar.

This function requires use of the API key.

Metadata and Tags

You can include tags by including a dedicated metadata.tags section at the top level of the data input. The application will tag each parameter accordingly. It is advisable to keep tags at a minimum to avoid timeouts, and avoid too many tags which loses much of the value tags provide.

Tags reside within the metadata section, to provide future extensibility within the metadata heirarchy.

To disable maintaining metadata in SSM Parameter Store, supply the --metadataAsParam false cli option when deploying.

Examples

POST https://abcdef0123.execute-api.us-west-2.amazonaws.com/prod/update?appId=psm-test&stage=dev HTTP/1.1
x-api-key: abcdef0123456789ABCDEF0123456789abcdef01

{
    "metadata": {
        "tags": {
            "Application": "Application1",
            "Environment": "dev",
            "Owner": "[email protected]"
        }
    },
    "foo": {
        "bar": "buzz"
    },
    "baz": "cipher:AQICAHjumw2gwZTBa2YnLcaUczMcoU..."
}

curl -X POST 'https://abcdef0123.execute-api.us-west-2.amazonaws.com/prod/update?appId=psm-test&stage=dev' \
  -H 'x-api-key: abcdef0123456789ABCDEF0123456789abcdef01' \
  -d '@config/dev.json'

with x-path-override header:

curl -X POST 'https://abcdef0123.execute-api.us-west-2.amazonaws.com/prod/update?appId=psm-test&stage=dev' \
  -H 'x-api-key: abcdef0123456789ABCDEF0123456789abcdef01' \
  -H 'x-path-override: /org/service/dev5/' \
  -d '@config/dev.json'

View

The view function will retrieve the configuration from parameter store, for review. SecureString parameters are encrypted with the CMK during the process. Similar to the update function, the view function leverages query string parameters, which are appId and stage.

This function requires use of the API key.

Example

GET https://abcdef0123.execute-api.us-west-2.amazonaws.com/prod/view?appId=psm-test&stage=dev HTTP/1.1
x-api-key: abcdef0123456789ABCDEF0123456789abcdef01

Installation && Deployment

• Install the necessary sls plugins with npm install.
• Install the necessary python requirements pip install -r requirements.txt.
• Deploy with serverless deploy --stage prod

Usage

Developers may start encrypting their secrets with the encrypt function and push configuration to source control after deployment.

Next, within deployment pipelines, the build server should look for configuration files to push with each stage. For example, the repository can have a config directory, with json files for each stage. Example: ./config/{stage}.json, or ./config/dev.json and /config/prod.json.

The pipeline can make REST calls to push the parameters within those files, calling the update function. Use parameters in your pipeline to pass in the appId and stage. Neiman Marcus matches the appId to the repository name.

Severless Framework

Serverless Framework is one of the tools that can leverage these parameters from SSM Parameter Store. It is common to push configuration to SSM Parameter Store, and then immediately call it when deploying. Local configuration files can be used with Serverless Framework, but would not leverage secret storage.

Example

• With the above example

provider:
  environment:
    FOO: ${ssm:/${self:service}/${self:provider.stage}/foo.bar}
    SECRET: ${ssm:/${self:service}/${self:provider.stage}/baz~true}

Query String Parameters

parameter	Description	Required
appId	The name of the application or service to prefix parameters with.	yes
stage	The name of the stage to prefix parameters with.	yes

Custom Headers

header	Description	Required
x-path-override	The custom path to override the PSM default.	no
x-kv-block-parser	An alternate way to parse input json.	no

Security

• The encrypt function does not require an API key since it is fairly simple and non-impacting.
• The update and view functions require API keys.
• The API key should be stored securely within your build server's credential store and not shared.
• It is suggested that you modify the ApiGatewayRestApi to white list specific IP addresses if possible. Alternatively you can run the Api Gateway as a private endpoint.
• Update the KMSKey.yml resource file with the appropriate IAM roles to manange the CMK.

Known issues

• Suppliying a list will map each item in the list to it's index.
  • For example, itme {"foo": ["bar", "baz"]} will be flattened to /{application}/{stage}/foo.0 and /{application}/{stage}/foo.1 with the value of bar and baz respectedly.
  • The view function will return the parameters as {"foo": {"0": bar", "1": baz"}}
  • It is best to avoid lists in the mean time.
• StringLists are not currently supported.

Items to Add

• Optional parameter clean up
• SSM Parameter Store backup and logging of changes
• Better error handling
• String Lists

Authors

• Clay Danford - Project creation and development.

Conduct / Contributing / License

• Refer to our contribution guidelines to contribute to this project. See CONTRIBUTING.md.
• All contributions must follow our code of conduct. See CONDUCT.md.
• This project is licensed under the Apache 2.0 license. See LICENSE.