Skip to content

Operations

Jump to bottom
Ryan Wold edited this page Mar 23, 2021 · 4 revisions

Deployment

This document provides explanation of the Touchpoints agile development process and the production deployment process.

Touchpoints is currently deployed in at least 3 environments.

  1. Staging - updated multiple times daily. used for continually integrating in-development code
  2. Demo - a stable, longer-lived environment for Touchpoint's gov customers to test in. Demo may be used to preview a pre-release version of Touchpoints.
  3. Production - the live site

These environments are managed via cloud.gov, and are currently configured to run on Cloud Foundry, but can run elsewhere with relatively little modification.

Each Developer is expected to be able to develop Touchpoints on their own cloud.gov instance as well.

How to Deploy Touchpoints

Deploying Touchpoints requires pushing a directory of code (the Touchpoints repo or CI build output) to an application instance in cloud.gov's Cloud Foundry using the cf push command.

This can be done 1) manually via command line or 2) in an automated way, using a continuous integration tool like CircleCI.

Using Cloud Foundry

cf push references a local manifest.yml that declares one or more applications. Thus, having a "properly configured" manifest.yml file is a prerequisite dependency; a requirement.

To deploy a specific instance, use cf push. For example: cf push touchpoints-staging.

Touchpoints Development Process

Software Product teams develop conventions related to the development process. The steps below represent the flow of code from a developer's machine through multiple gates, to production.

How git branches are used
  • Code that is releasable lives in the production branch on GitHub.
  • Code that is stable lives in the main branch.
  • Code that is in-development lives in the develop branch.
  • Developers work in feature-branches, created off the develop branch.
How code flows from feature to develop to main to production
  • LOCAL DEVELOPMENT
  • feature branches are pushed to https://touchpoints-dev-name-of-dev.app.cloud.gov (each Dev has their own Dev environment)
  • feature branches are submitted as Pull Requests and reviewed in GitHub
  • all commits are pushed here, to this Touchpoints code repository
  • upon code commit, CircleCI runs automated tests
  • upon code commit, Snyk runs automated code scans
  • When a feature branch is code reviewed and clicked-through on , it may be merged to develop
  • When a story is completed and ready for Acceptance, a Developer pushes the develop or feature branch to the Staging environment
    • green (passing) main builds get deployed to Staging automatically via CircleCI
  • ON STAGING...
  • Product Owner accepts or rejects stories and/or features on the Staging environment.
  • 👀 CODE REVIEW
  • When stories in develop are Accepted by the Product Owner on Staging, they can be merged to main with the Team's discretion
  • the develop branch is merged to main via Pull Request
    • green (passing) main builds get deployed to Demo automatically via CircleCI
  • ON DEMO...
  • Users can use the product and share feedback continuously ➰ in a Demo environment
  • CREATING A RELEASE:
  • releases are tagged using semantic versioning. for example: git tag 0.0.1
  • 🚢 TO PRODUCTION!
    • green (passing) main builds can be merged to production and then deployed to Production by coordinating with Ryan Wold

Environment variables

Touchpoints requires multiple environment variables to be setup for each application instance. Environment variables are conventionally set in manifest.yml. See manifest.sample.yml for an example.

Configuration values are to be set in the cloud.gov application instance as environment variables.

There is an audit trail for changing cloud.gov environment variables via the Cloud Foundry API. https://apidocs.cloudfoundry.org/253/events/list_user_provided_service_instance_update_events.html

name description required default
AWS_ACCESS_KEY_ID IAM Account Key used for Simple email service no
AWS_SECRET_ACCESS_KEY IAM Account Access Key used for Simple email service no
AWS_REGION specifies AWS region no us-east-1
NEW_RELIC_KEY API Key for New Relic. The New Relic Key is the same for Staging and Production. no
TOUCHPOINTS_EMAIL_SENDER email address when Touchpoints sends email. Account Confirmation, Password Reset, Submission Notification yes
TOUCHPOINTS_GTM_CONTAINER_ID GTM to deliver analytics for the deployed app/Product itself no

Deploying the Touchpoints Application to Cloud.gov

Assuming you've setup the environment variables above.

Assuming manifest.yml is properly configured Assuming you have /tmp/goggle_service_account_ENV.json in place for your respective ENV.

  • note: in test environment, the .json string is copied to CircleCI as GOOGLE_CONFIG Assuming you have other ENVs set (see Section above)

  • push to staging using cf push touchpoints-staging

  • push to staging using cf push touchpoints-demo

Additional Notes for Developers

Running Touchpoints requires a Google Service Account with access to Tag Manager, Drive, and Sheets API. Be sure to add the Service Account email address to the Google Tag Manager Account User Management Panel.

General Developer Prerequisites

An onboarding Touchpoints Developer should have access to the following tools.

  • GSA Email
  • cloud.gov
  • GitHub
  • Google Tag Manager
  • AWS Simple Email Service
  • Circle CI
  • Snyk

Configuring s3

Touchpoints uses s3 buckets, which are provided as Cloud.gov service.

Touchpoints stores Organization logo image files, and .pdfs of Service Maps for a given Touchpoint.

Here's how to configure cloud.gov S3 for Touchpoints:

  1. Create a public S3 service in cloud.gov
  2. Bind that s3 instance to your Touchpoints application application
  3. Then look at the VCAP settings of that cloud.gov application instance, and set those configs in Touchpoints /config/initializers/carrierwave.rb
  • aws_access_key_id
  • aws_secret_access_key
  • region
  • host
  • fog_directory (the name of the S3 bucket)

Configuring CircleCI to deploy to cloud.gov

Setup cloud.gov Service Accounts. Use the cloud.gov Service Accounts in CircleCI to push successful builds to environments automatically (continuous deployment), or with a click of a button (when 🚢 shipping software is a business decision).

Deployer Accounts

Touchpoints uses Cloud.gov Service Accounts are used to deploy the application from the Continuous Integration server.

for Staging and Demo environments

  • cf create-service cloud-gov-service-account space-deployer touchpoints-staging-deployer - creates the Service Account service
  • cf create-service-key touchpoints-staging-deployer staging-deploy-key - creates the Service Account's Service Key
  • cf service-key touchpoints-staging-deployer staging-deploy-key - diplays the username & password of the Service Account's Service Key

The Staging Service Account's username and password are used on CI to deploy to Staging and Demo.

for Production environment

  • cf create-service cloud-gov-service-account space-deployer touchpoints-deployer
  • cf create-service-key touchpoints-deployer staging-deploy-key
  • cf service-key touchpoints-deployer staging-deploy-key

Touchpoints Maintenance Page

Touchpoints has a Maintenance Page for display when the app is under extended maintenance or otherwise experiencing downtime.

The maintenance page is a separate application; with 1 html file. 1 maintenance page is maintained for each Cloud.gov space. In cloud.gov, the web application simply updates a virtual route to point the maintenance page app. When complete, the virtual route is pointed back at the web application.

Because Touchpoints maintains 3 "Spaces" (dev, staging, prod), the 3 maintenance pages exist as:

  • touchpoints-dev-maintenance-page
  • touchpoints-staging-maintenance-page
  • touchpoints-prod-maintenance-page

Disabling Health Checks for Workers

Because Worker instances do not have publicly accessible HTTP endpoints, health checks should be disabled for the app. Otherwise, cloud.gov will kill the application instance when it fails a health check.

  • Staging - cf set-health-check touchpoints-staging-sidekiq-worker none
  • Demo - cf set-health-check touchpoints-demo-sidekiq-worker none
  • Production - cf set-health-check touchpoints-production-sidekiq-worker none

See the README at https://github.com/gsa/touchpoints-maintenance-page for information about the specific commands to create and use the maintenance page.

Scaling

To scale Touchpoints instances on cloud.gov:

  • To view how many instances are set to run - cf scale instance-name
    • example: cf scale touchpoints
  • To set how many instances should run - cf scale instance-name -i number-of-instances
    • example: cf scale touchpoints-staging -i 2
References:
Clone this wiki locally