Rycar/dca mvp

terraform/dca/aws/README.md

@@ -0,0 +1,80 @@
+# Detect, Correct, Automate - Effortless Automaton Demo
+
+This directory contains some minimal terraform capible of spinning up 1-n linux servers (defined in a `node_count` variable in `terraform.tfvars`) on which we can apply some simple detect & correct content.
+
+## Usage
+
+### Before you begin
+
+This project is designed to work with Terraform >= 0.12.
+
+Before you launch any servers, you must decide whether or not you wish to launch a Chef Automate instance. Running Chef Automate is optional, but recommended.
+
+To run **without** a Chef Automate server:
+- Ensure the `event-stream-enabled` parameter is set to `false` in your `terraform.tfvars` file.
+
+To run **with** a Chef Automate server:
+- First launch a Chef Automate instance (this can be done with the tf plan found in `../../chef-automate/aws`). Save the output variables once complete.
+- Ensure the `event-stream-enabled` parameter is set to `false` in your `terraform.tfvars` file.
+- Ensure the `automate_hostname`, `automate_url`, `automate_ip`, and `automate_token` are set with the proper information (NOTE: `automate_ip` refers to the public IP of the automate server)
+- Set the `node_count` variable to your prefered number of nodes.
+
+### OPTIONAL: Define your detect/correct Chef Habitat artifacts
+
+I have created two packages to run through content quickly & easily:
+- `nrycar/dca-audit`: Identical to the audit-baseline package, but without the patching scan
+- `nrycar/dca-hardening`: As above for config-baseline, skips the yum update to make things speedy
+
+If this serves your purposes, you don't need to make any changes to `terraform.tfvars`. However, the origin and package are configurable, so long as they're publicly available on `bldr.habitat.sh`. If you'd like to substitute an effortless package of your own, it's supported! Just make sure you test it ahead of time.
+
+### Launch your server(s)
+
+Once your `terraform.tfvars` has been updated with the above variables, and your usual keys & tags, you should now be able to run `terraform apply`
+
+### Running the demo
+
+Once your instances are provisioned, you can run a Detect/Correct/Automate motnion via some shell scripts provided in the ./scripts/ directory.
+
+All scripts will use the SSH information provided in your local SSH config, or optionally you can pass the path to your ssh key of choice as a parameter like so:
+`./scripts/detect.sh ~/.ssh/my_ssh_key.pem`
+
+To run through the demo narrative, there are three scripts you'll be using:
+
+- `./scripts/detect.sh`: Loads the audit package, and sends the output to Chef Automate
+- `./scripts/waivers.sh`: Loads `waivers.toml` from `/.effortless_dca/files/` and applies it to the running audit package. By default, sets up a wavier for the `sysctl-14` and `os-08` controls (inhereted from the `linux-baseline` profile).
+- `./scripts/correct.sh`: Loads the infra package as above. Takes ~30-60 seconds for results and audits to show up in Automate by default.
+
+### Re-Running the demo
+
+The easiest way to re-run the demo after running through the correct step is to spin everything down and back up (`terraform destroy` followed by a `terraform apply`).
+
+Your old instances will still show up in Automate, but will go stale at whatever setting you have for missing nodes in your automate config (default: 1 day). If you have the EAS dashboard enabled, you'll also see those instances show up as "disconnected" in fairly short order once they're spun down.
+
+## Under the hood
+
+### Terraform Plans
+
+The terraform is pretty lightweight and straightforward. The code is largely ripped from the national-parks instances with a few differences:
+
+- Chef Habitat and Chef Infra Client are pre-installed to ensure the included shell scripts execute quickly
+- The effortless audit/infra packages are installed (again for speed), but not loaded. This ensures no data shows up in Chef Automate until you're ready for it to.
+
+### The effortless_dca cookbook
+
+The cookbook has been kept very simple for ease of use and readability. The bulk of the content is handed via simple execute blocks with guards to load the appropriate hab packages if they're not already loaded like so:
+
+```
+execute 'Run Audits' do
+  command "hab svc load #{node['effortless_dca']['audit_origin']}/#{node['effortless_dca']['audit_package']}"
+  action :run
+  not_if "hab sup status #{node['effortless_dca']['audit_origin']}/#{node['effortless_dca']['audit_package']}"
+end
+```
+
+### The shell scripts
+
+The shell scripts are all using a combination of `chef-run` with the outputs of your `terraform apply` to ensure that no matter how many nodes you spin up, you can run the dca audits on the lot of them. The commands all run some form of:
+
+`chef-run ``terraform output dca_public_ips`` effortless_dca::correct --user centos`
+
+While the number of nodes is arbitrary, please note that chef-run will perform updates in parallel from your local machine. This shouldn't make much difference with default settings, but if you run a custom cookbook that needs to transfer any large files/binaries, you may see slowdown with higher numbers of nodes.

terraform/dca/aws/dca.tf

@@ -0,0 +1,75 @@
+# Launch var.node_count CentOS instances for DCA Demos
+resource "aws_instance" "dca" {
+  connection {
+    host        = coalesce(self.public_ip, self.private_ip)
+    type        = "ssh"
+    user        = var.aws_ami_user
+    private_key = file(var.aws_key_pair_file)
+  }
+
+  ami                         = data.aws_ami.centos.id
+  instance_type               = var.test_server_instance_type
+  key_name                    = var.aws_key_pair_name
+  subnet_id                   = aws_subnet.dca_subnet.id
+  vpc_security_group_ids      = [aws_security_group.dca.id]
+  associate_public_ip_address = true
+  count                       = var.node_count
+
+  tags = {
+    Name          = "dca_${random_id.instance_id.hex}"
+    X-Dept        = var.tag_dept
+    X-Customer    = var.tag_customer
+    X-Project     = var.tag_project
+    X-Application = var.tag_application
+    X-Contact     = var.tag_contact
+    X-TTL         = var.tag_ttl
+  }
+
+  provisioner "file" {
+    content     = data.template_file.install_hab.rendered
+    destination = "/tmp/install_hab.sh"
+  }
+
+  provisioner "file" {
+    content     = data.template_file.sup_service.rendered
+    destination = "/home/${var.aws_ami_user}/hab-sup.service"
+  }
+
+  provisioner "file" {
+    content     = data.template_file.audit_toml_linux.rendered
+    destination = "/home/${var.aws_ami_user}/audit_linux.toml"
+  }
+
+  provisioner "file" {
+    content     = data.template_file.config_toml_linux.rendered
+    destination = "/home/${var.aws_ami_user}/config_linux.toml"
+  }
+
+  provisioner "remote-exec" {
+    inline = [
+      "sudo rm -rf /etc/machine-id",
+      "sudo systemd-machine-id-setup",
+      "sudo hostname dca-${count.index}",
+      "curl -L https://omnitruck.chef.io/install.sh | sudo bash",
+      "sudo groupadd hab",
+      "sudo adduser hab -g hab",
+      "chmod +x /tmp/install_hab.sh",
+      "sudo /tmp/install_hab.sh",
+      "sudo hab license accept",
+      "sudo hab pkg install ${var.hab-sup-version}",
+      "sudo mv /home/${var.aws_ami_user}/hab-sup.service /etc/systemd/system/hab-sup.service",
+      "sudo systemctl daemon-reload",
+      "sudo systemctl start hab-sup",
+      "sudo systemctl enable hab-sup",
+      "sudo /sbin/sysctl -w net.ipv4.conf.all.accept_source_route=0",
+      "sudo /sbin/sysctl -w net.ipv4.conf.default.accept_source_route=0",
+      "sudo /sbin/sysctl -w net.ipv4.conf.default.accept_redirects=0",
+      "sudo /sbin/sysctl -w net.ipv4.conf.all.accept_redirects=0",
+      "sudo mkdir -p /hab/user/${var.infra_package}/config /hab/user/${var.audit_package}/config",
+      "sudo cp /home/${var.aws_ami_user}/audit_linux.toml /hab/user/${var.audit_package}/config/user.toml",
+      "sudo cp /home/${var.aws_ami_user}/config_linux.toml /hab/user/${var.infra_package}/config/user.toml",
+      "sudo hab pkg install ${var.infra_origin}/${var.infra_package} --channel stable",
+      "sudo hab pkg install ${var.audit_origin}/${var.audit_package} --channel stable",
+    ]
+  }
+}

terraform/dca/aws/effortless_dca/.delivery/project.toml

@@ -0,0 +1,34 @@
+# Delivery for Local Phases Execution
+#
+# This file allows you to execute test phases locally on a workstation or
+# in a CI pipeline. The delivery-cli will read this file and execute the
+# command(s) that are configured for each phase. You can customize them
+# by just modifying the phase key on this file.
+#
+# By default these phases are configured for Cookbook Workflow only
+#
+
+[local_phases]
+unit = "chef exec rspec spec/"
+lint = "chef exec cookstyle"
+# Foodcritic includes rules only appropriate for community cookbooks
+# uploaded to Supermarket. We turn off any rules tagged "supermarket"
+# by default. If you plan to share this cookbook you should remove
+# '-t ~supermarket' below to enable supermarket rules.
+syntax = "chef exec foodcritic . -t ~supermarket"
+provision = "chef exec kitchen create"
+deploy = "chef exec kitchen converge"
+smoke = "chef exec kitchen verify"
+# The functional phase is optional, you can define it by uncommenting
+# the line below and running the command: `delivery local functional`
+# functional = ""
+cleanup = "chef exec kitchen destroy"
+
+# Remote project.toml file
+#
+# Instead of the local phases above, you may specify a remote URI location for
+# the `project.toml` file. This is useful for teams that wish to centrally
+# manage the behavior of the `delivery local` command across many different
+# projects.
+#
+# remote_file = "https://url/project.toml"

terraform/dca/aws/effortless_dca/.gitignore

@@ -0,0 +1,22 @@
+.vagrant
+*~
+*#
+.#*
+\#*#
+.*.sw[a-z]
+*.un~
+
+# Bundler
+Gemfile.lock
+gems.locked
+bin/*
+.bundle/*
+
+# test kitchen
+.kitchen/
+kitchen.local.yml
+
+# Chef
+Berksfile.lock
+.zero-knife.rb
+Policyfile.lock.json

terraform/dca/aws/effortless_dca/CHANGELOG.md

@@ -0,0 +1,11 @@
+# effortless_dca CHANGELOG
+
+This file is used to list changes made in each version of the effortless_dca cookbook.
+
+# 0.1.0
+
+Initial release.
+
+- change 0
+- change 1
+

terraform/dca/aws/effortless_dca/LICENSE

@@ -0,0 +1,3 @@
+Copyright 2019 The Authors
+
+All rights reserved, do not redistribute.

terraform/dca/aws/effortless_dca/Policyfile.rb

@@ -0,0 +1,16 @@
+# Policyfile.rb - Describe how you want Chef Infra Client to build your system.
+#
+# For more information on the Policyfile feature, visit
+# https://docs.chef.io/policyfile.html
+
+# A name that describes what the system you're building with Chef does.
+name 'effortless_dca'
+
+# Where to find external cookbooks:
+default_source :supermarket
+
+# run_list: chef-client will run these recipes in the order specified.
+run_list 'effortless_dca::default'
+
+# Specify a custom source for a single cookbook:
+cookbook 'effortless_dca', path: '.'

terraform/dca/aws/effortless_dca/README.md

@@ -0,0 +1,4 @@
+# effortless_dca
+
+TODO: Enter the cookbook description here.
+

terraform/dca/aws/effortless_dca/attributes/default.rb

@@ -0,0 +1,4 @@
+default['effortless_dca']['audit_origin'] = 'nrycar'
+default['effortless_dca']['infra_origin'] = 'nrycar'
+default['effortless_dca']['audit_package'] = 'dca-audit'
+default['effortless_dca']['infra_package'] = 'dca-hardening'

terraform/dca/aws/effortless_dca/chefignore

@@ -0,0 +1,110 @@
+# Put files/directories that should be ignored in this file when uploading
+# to a Chef Infra Server or Supermarket.
+# Lines that start with '# ' are comments.
+
+# OS generated files #
+######################
+.DS_Store
+ehthumbs.db
+Icon?
+nohup.out
+Thumbs.db
+
+# SASS #
+########
+.sass-cache
+
+# EDITORS #
+###########
+.#*
+.project
+.settings
+*_flymake
+*_flymake.*
+*.bak
+*.sw[a-z]
+*.tmproj
+*~
+\#*
+mkmf.log
+REVISION
+TAGS*
+tmtags
+
+## COMPILED ##
+##############
+*.class
+*.com
+*.dll
+*.exe
+*.o
+*.pyc
+*.so
+*/rdoc/
+a.out
+
+# Testing #
+###########
+.circleci/*
+.codeclimate.yml
+.foodcritic
+.kitchen*
+.rspec
+.rubocop.yml
+.travis.yml
+.watchr
+azure-pipelines.yml
+examples/*
+features/*
+Guardfile
+kitchen.yml*
+Procfile
+Rakefile
+spec/*
+spec/*
+spec/fixtures/*
+test/*
+
+# SCM #
+#######
+.git
+.gitattributes
+.gitconfig
+.github/*
+.gitignore
+.gitmodules
+.svn
+*/.bzr/*
+*/.git
+*/.hg/*
+*/.svn/*
+
+# Berkshelf #
+#############
+Berksfile
+Berksfile.lock
+cookbooks/*
+tmp
+
+# Bundler #
+###########
+vendor/*
+Gemfile
+Gemfile.lock
+
+# Policyfile #
+##############
+Policyfile.rb
+Policyfile.lock.json
+
+# Cookbooks #
+#############
+CHANGELOG*
+CONTRIBUTING*
+TESTING*
+CODE_OF_CONDUCT*
+
+# Vagrant #
+###########
+.vagrant
+Vagrantfile

terraform/dca/aws/effortless_dca/files/waivers.toml

@@ -0,0 +1,11 @@
+interval = 30
+splay = 5
+
+[waivers]
+[waivers.'sysctl-14']
+run = false
+justification = "Conflict with corporate networking policy. Skip."
+[waivers.'os-08']
+expiration_date = "2020-04-01"
+run = false
+justification = "We have a plan to fix this in Q1. Alert after spring."