documentation

README.md

@@ -3,56 +3,62 @@
 ## Prerequisites
 
 * Python 3
-* AWS CLI
-* Vault CLI
-* Terraform CLI
-* Env vars:
-  * `VAULT_ADDR`
-  * `VAULT_TOKEN`
+* [Vault CLI](https://developer.hashicorp.com/vault/downloads)
 
 Install Python Dependencies:
 
 ```
 python3 -m pip install -r requirements.txt
 ```
 
-## Apps
-
-Apps are deployed from `apps/` directory using docker compose.
-
-### Deploy
+Create a directory for secret files:
 
 ```
-bin/apps.py deploy_app APP_NAME
+sudo mkdir -p /etc/dfc
+sudo chown $USER -R /etc/dfc
 ```
 
-### Docker Compose Management
+## Apps
+
+The apps are defined using Docker Compose with some additional configurations under `apps/`
 
-```
-bin/apps.py compose APP_NAME COMMAND
-```
+Anyway app is continuously deployed on any change to `apps/anyway`, the other apps need to be deployed manually.
 
-for example:
+To manage the apps you need to get a Vault TOKEN, access vault at https://vault.dataforchange.gov.il and login.
+Once logged-in, click on your name in the top right corner and select "Copy token".
+Set the Token in an env var:
 
 ```
-bin/apps.py compose anyway ps
-bin/apps.py compose anyway logs -f
-bin/apps.py compose anyway logs -f anyway-main
+export VAULT_TOKEN=<paste token here>
 ```
 
-### Server Management
+You might need to get a new token occasionally, if you get an error message about an invalid token, just repeat the above steps.
 
-SSH to the server:
+Now you can run the apps management script to manage the apps, following are some examples:
 
 ```
+# Run Docker Compose commands
+bin/apps.py compose APP_NAME COMMAND
+# for example:
+bin/apps.py compose anyway ps
+bin/apps.py compose anyway logs -f anyway-main
+
+# SSH to the server
 bin/apps.py ssh
+
+# Run a command on the server
 bin/apps.py ssh docker ps
+
+# Deploy an app
+bin/apps.py deploy_app APP_NAME
 ```
 
 ## Terraform
 
-Every commit to the repo will run terraform plan in GitHub actions, you can check the actions log for details.
-To prevent destructive actions, apply the changes you have to run locally as described below.
+Prerequisites:
+
+* [Terraform CLI](https://www.terraform.io/downloads.html)
+* [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html)
 
 ### Running locally
 

apps/redash/compose.yaml

@@ -64,8 +64,8 @@ services:
       - "traefik.enable=true"
       - "traefik.http.services.redash-nginx.loadbalancer.server.port=80"
       - "traefik.http.routers.redash-nginx.rule=Host(`redash.dataforchange.org.il`)"
-      # - "traefik.http.routers.anyway-reports.tls=true"
-      # - "traefik.http.routers.anyway-reports.tls.certresolver=dfc"
+      - "traefik.http.routers.redash-nginx.tls=true"
+      - "traefik.http.routers.redash-nginx.tls.certresolver=dfc"
     # pulled Nov 1, 2021
     image: redash/nginx:latest@sha256:4eaaa7af6476b0422058b0022661ad6129dfbf9065c506fb0904bbf0a16f2007
     restart: unless-stopped

bin/apps.py

@@ -3,21 +3,26 @@
 import sys
 import json
 import subprocess
-import tempfile
-
 from ruamel import yaml
 from functools import lru_cache
 from tempfile import TemporaryDirectory
 
 
+VAULT_ADDR = os.environ.get('VAULT_ADDR', 'https://vault.dataforchange.org.il')
+VAULT_TOKEN = os.environ.get('VAULT_TOKEN')
 ETC_DFC_DOCKER = '/etc/dfc/docker' if os.environ.get("CI") != "true" else '/tmp/dfc/docker'
 
 
 @lru_cache()
 def get_vault_kv_path(path):
+    assert VAULT_TOKEN, "VAULT_TOKEN env var is required, see README for how to get it"
     return json.loads(subprocess.check_output([
         'vault', 'kv', 'get', '-format=json', f'kv/{path}'
-    ]))['data']['data']
+    ], env={
+        **os.environ,
+        'VAULT_ADDR': VAULT_ADDR,
+        'VAULT_TOKEN': VAULT_TOKEN,
+    }))['data']['data']
 
 
 def ssh_init():