From a8688bdf8837b0cabca35593bc26b933e5123b9a Mon Sep 17 00:00:00 2001 From: Emma Paz Date: Fri, 8 Mar 2024 11:49:41 -0500 Subject: [PATCH] Reorganized content to match using docker compose configuration --- README.md | 281 ++++++++++++++---------------------------------------- 1 file changed, 73 insertions(+), 208 deletions(-) diff --git a/README.md b/README.md index 5d60ecad..64184356 100644 --- a/README.md +++ b/README.md @@ -1,115 +1,104 @@ -# Contents of this readme +# ADMG Project Repo +Welcome to the ADMG project repository. This repository contains the code for the ADMG web application, which is a Django web application that provides a user interface for the ADMG database. The application is built using the Django web framework and is designed to be deployed using Docker. -1. `admg_webapp` backend documentation -2. `admin_ui` frontend setup -# `admg_webapp` backend documentation +## README Contents +- Project Structure +- Local Development +- Additional Dev Tools +- Automated Deployment +- Configuring system to deploy CASEI -## How to use the interactive Query -``` -pip install notebook
-pip install django-extensions -``` +## Project Structure +- frontend: `/app/admin_ui` +- backend: `/app/admg_webapp` + +## Local Development -Add django_extensions to installed apps unless using cookiecutter
+### Project Setup +1. Install docker +2. Copy `.env.sample_local` to `.env` +3. Run docker compose
+ `docker compose up` + +#### If this is your first time setting up & running the application, you will also need to: +**Run migrations** to create the database schema: +```sh +docker compose run --rm web sh -c "python manage.py migrate" ``` -python manage.py shell_plus --notebook +**Create a superuser** to access the admin interface: +``` sh +docker compose run --rm web sh -c "python manage.py createsuperuser" +``` +**Load a dump** of the database to view the application with some data: +Download the latest zip file of example data (or get this from one of the database maintainers) & load into the database. The following command will load the data into the database: +```sh +cat ./production_dump-2020.01.28.sql | psql admg_webapp ``` -in the notebook, import your models file - -## How to get the token - -- go to /authenticate/applications/register/ -- create a user and verify the email address by clicking on the link that shows up in the terminal where you've done `python manage.py runserver` -- register the app -- Use Client Type: confidential, Authorization Grant Type: Resource owner password-based -- get the `client_id` and `client_secret` -- `curl -X POST -d "grant_type=password&username=&password=" -u":" http://domain/authenticate/token/` -- You will get something like: - - ```javascript - { - "access_token": "access_token", - "expires_in": 36000, - "token_type": "Bearer", - "scope": "read write", - "refresh_token": "refresh_token" - } - ``` - -- Use this `access_token` to hit on APIs - - `curl -H "Authorization: Bearer " http://localhost:8000/your_end_point_here` -- To refresh your token - - `curl -X POST -d "grant_type=refresh_token&refresh_token=&client_id=&client_secret=" http://localhost:8000/authenticate/token` +> ^ These commands should be run in a new terminal window, while the application is running. -Example JavaScript code +## Additional Dev Tools -``` -const url = 'http[s]://www.domain.com/authenticate/token/'; -const cId = '' -const cSecret = '' -const data = new FormData(); -data.append('username', ''); -data.append('password', ''); -data.append('grant_type', 'password'); -const config = { - method: 'post', - url, - data, - auth: { - username: cId, - password: cSecret, - } -}; -axios(config) -.then(function (response) { - // response.access_token will have the token -}) -``` -## Automatic deployment +### Understanding `python manage.py` -- Update the webserver IP in the hosts/ file. If no hosts file exists, create one [see hosts/.sample file] -- Run the command `ansible-playbook --private-key private_key_file.pem -i hosts/ playbook.yml -v [-e 'branch=']` - - `private_key_file.pem` is the private key for the webserver - - `environment` a choice of staging or production - - `[-e 'branch=']` part is optional and can be used in case some another branch is desired +`python manage.py ` -## Local Setup +`manage.py` is your entry point into the django app. It has several commands, including: + - `test` + - `migrate` + - `makemigrations` + - `runserver_plus` + - `shell_plus` + - django extensions — third party modules -- Install docker -- Copy `.env.sample_local` to `.env` -- Run docker compose - - `docker compose up` +To run python manage.py commands using docker compose, use the following command structure: + ``` + docker compose run --rm -it web python manage.py + ``` -## Shell Access +### Shell Access Utilize Django's shell for experimentation with queries & other Django functionality: -`docker compose run --rm -it web python manage.py shell_plus` +```sh +docker compose run --rm -it web python manage.py shell_plus +``` +### Project URLs -## Running Tests +List all of the URL patterns for the project: +```sh +docker compose run --rm it web python manage.py show_urls +``` +### Running Tests -`docker compose run --rm -it web pytest` +Run your tests: +```sh +docker compose run --rm -it web pytest +``` ### Reporting test coverage Run your tests with coverage: - -`docker compose run --rm -it web python -m coverage run -m pytest` +```sh +docker compose run --rm -it web python -m coverage run -m pytest +``` Generate coverage report: - -`docker compose run --rm -it web python -m coverage report -m --skip-covered` +```sh +docker compose run --rm -it web python -m coverage report -m --skip-covered +``` If you want to view coverage in your editor using, for example, VSCode's Coverage Gutters plugin, export the coverage report to a supported format: -`docker compose run --rm -it web python -m coverage lcov -o coverage.lcov` +```sh +docker compose run --rm -it web python -m coverage lcov -o coverage.lcov +``` -## Sass +### Sass To build Sass files for the project: @@ -117,137 +106,13 @@ To build Sass files for the project: python manage.py sass admin_ui/static/scss admin_ui/static/css --watch ``` -# `admin_ui` setup - -## Installation - -1. Install prerequisite technologies (for example, using `brew` on a mac): postgres, postgis - -2. Create a virtual environment - - Set up the env (only need to do once) - - ``` - python3 -m venv .venv - ``` - -3. Activate the virtual environment (do this every time you start the project) - - ``` - source .venv/bin/activate - ``` - -4. Install requirements - - 1. general requirements - - ``` - pip install -r requirements/base.txt - ``` - - 2. local requirements - - ``` - pip install -r local.txt - ``` - -5. Start postgres - - To get a path that you can use to start Postgres: - - ``` - brew info posgresql - ``` - - (It will probably look something like `pg_ctl -D /usr/local/var postgres start`) - -6. Check that postgres is working - If `psql -l` gives you a list of tables then all is well. - -7. Create a database - - ``` - createdb admg_prod - ``` - -8. Load a dump of the database - - Download the latest zip file of example data (get this from one of the database maintainers) & load into the database. For example: - - ``` - cat ./production_dump-2020.01.28.sql | psql admg_prod - ``` - -9. Run migrations - - ``` - python manage.py migrate - ``` - -10. Create yourself a user - - ``` - python manage.py creatersuperuser - ``` - -## Running the application - -1. With the virual environment activated, run the server - - ``` - python manage.py runserver_plus - ``` - -2. Open the website - http://localhost:8000/ - -### Understanding `python manage.py` - -`python manage.py ` - -- `manage.py` is your entry point into the django app. It has several commands, including: - - `test` - - `migrate` - - `makemigrations` - - `runserver_plus` - - `shell_plus` - - django extensions — third party modules - -### Optional additional tools - -interactive way to interact with the database and the database models. - -`python manage.py shell_plus` - -### Running the infrastructure for DOI fetching - -DOI fetching uses rabbitmq and celery. - -#### Installation - -Install `rabbitmq` (probably using `brew` if you’re on a Mac) - -#### Starting the service - -1. start rabbitmq: - - ``` - rabbitmq-server - ``` - -2. start the celery worker: - ``` - celery -A config.celery_app worker --beat --scheduler django -l DEBUG - ``` +## Automated deployment - _Note: If running locally (ie not in Docker), you may need to overwrite the `CELERY_BROKER_URL` setting:_ +Several automated workflows are already configured. These can be found within the .github/workflows directory. - ``` - CELERY_BROKER_URL=amqp://guest:guest@localhost:5672 celery -A config celery_app worker --beat --scheduler django -l DEBUG` - ``` -### Configuring system to deploy CASEI +## Configuring system to deploy CASEI The Maintenance Interface is able to initiate a deployment of [CASEI](https://github.com/NASA-IMPACT/admg-inventory/). This works by triggering a [workflow dispatch event](https://docs.github.com/en/rest/reference/actions#create-a-workflow-dispatch-event) on CASEI's [`deploy-to-production` workflow](https://github.com/NASA-IMPACT/admg-inventory/actions/workflows/deploy-to-production.yml). To allow the Maintenance Interface to trigger CASEI, a [Personal Access Token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens) with `actions:write` permissions should be provided via the `CASEI_GH_TOKEN` environment secret. The following environment variables may optionally be provided to override default configuration: