template update

.description.md

@@ -0,0 +1,25 @@
+<!-- Please provide a short overview of the features of this service.-->
+
+This repo is a template for creating a new microservice.
+
+The directories, files, and their structure herein are recommendations
+from the GHGA Dev Team.
+
+### Naming Conventions
+The github repository contains only lowercase letters, numbers, and hyphens "-",
+e.g.: `my-microservice`
+
+The python package (and thus the source repository) contains underscores "_"
+instead of hyphens, e.g.: `my_microservice`
+However, an abbreviated version is prefered as package name.
+
+### Adapt to your service
+This is just a template and needs some adaption to your specific use case.
+
+Please search for **"please adapt"** comments. They will indicate all locations
+that need modification. Once the adaptions are in place, please remove these #
+comments.
+
+Finally, follow the instructions to generate the README.md described in
+[`./readme_generation.md`](./readme_generation.md). Please also adapt this markdown file
+by providing an overview of the feature of the package.

.design.md

@@ -0,0 +1,7 @@
+<!-- Please provide an overview of the architecture and design of the code base.
+Mention anything that deviates from the standard triple hexagonal architecture and
+the corresponding structure. -->
+
+This is a Python-based service following the Triple Hexagonal Architecture pattern.
+It uses protocol/provider pairs and dependency injection mechanisms provided by the
+[hexkit](https://github.com/ghga-de/hexkit) library.

.github/workflows/check_readme.yaml

@@ -0,0 +1,16 @@
+name: Check if the readme is up to date.
+
+on: push
+
+jobs:
+  static-code-analysis:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v3
+
+      - id: common
+        uses: ghga-de/gh-action-common@v2
+
+      - name: Check readme
+        run: |
+          ./scripts/update_readme.py --check

.gitignore

@@ -50,6 +50,7 @@ coverage.xml
 *.py,cover
 .hypothesis/
 .pytest_cache/
+prof/
 
 # Translations
 *.mo

.mandatory_files

@@ -10,11 +10,6 @@
 .devcontainer/docker-compose.yml
 .devcontainer/Dockerfile
 
-
-docs/README.md
-
-example_data
-
 tests/__init__.py
 tests/fixtures/__init__.py
 
@@ -28,3 +23,6 @@ README.md
 setup.py
 setup.cfg
 requirements-dev.txt
+
+.description.md
+.design.md

.readme_template.md

@@ -0,0 +1,109 @@
+
+[![tests](https://github.com/ghga-de/$name/actions/workflows/unit_and_int_tests.yaml/badge.svg)](https://github.com/ghga-de/$name/actions/workflows/unit_and_int_tests.yaml)
+[![Coverage Status](https://coveralls.io/repos/github/ghga-de/$name/badge.svg?branch=main)](https://coveralls.io/github/ghga-de/$name?branch=main)
+
+# $title
+
+$summary
+
+## Description
+
+$description
+
+## Installation
+We recommend using the provided Docker container.
+
+A pre-build version is available at [docker hub](https://hub.docker.com/repository/docker/ghga/$name):
+```bash
+docker pull ghga/$name:$version
+```
+
+Or you can build the container yourself from the [`./Dockerfile`](./Dockerfile):
+```bash
+# Execute in the repo's root dir:
+docker build -t ghga/$name:$version .
+```
+
+For production-ready deployment, we recommend using Kubernetes, however,
+for simple use cases, you could execute the service using docker
+on a single server:
+```bash
+# The entrypoint is preconfigured:
+docker run -p 8080:8080 ghga/$name:$version --help
+```
+
+If you prefer not to use containers, you may install the service from source:
+```bash
+# Execute in the repo's root dir:
+pip install .
+
+# To run the service:
+$shortname --help
+```
+
+## Configuration
+### Parameters
+
+The service requires the following configuration parameters:
+$config_description
+
+### Usage:
+
+A template YAML for configurating the service can be found at
+[`./example-config.yaml`](./example-config.yaml).
+Please adapt it, rename it to `.$shortname.yaml`, and place it into one of the following locations:
+- in the current working directory were you are execute the service (on unix: `./.$shortname.yaml`)
+- in your home directory (on unix: `~/.$shortname.yaml`)
+
+The config yaml will be automatically parsed by the service.
+
+**Important: If you are using containers, the locations refer to paths within the container.**
+
+All parameters mentioned in the [`./example-config.yaml`](./example-config.yaml)
+could also be set using environment variables or file secrets.
+
+For naming the environment variables, just prefix the parameter name with `${shortname}_`,
+e.g. for the `host` set an environment variable named `${shortname}_host`
+(you may use both upper or lower cases, however, it is standard to define all env
+variables in upper cases).
+
+To using file secrets please refer to the
+[corresponding section](https://pydantic-docs.helpmanual.io/usage/settings/#secret-support)
+of the pydantic documentation.
+
+$openapi_doc
+
+## Architecture and Design:
+$design_description
+
+## Development
+For setting up the development environment, we rely on the
+[devcontainer feature](https://code.visualstudio.com/docs/remote/containers) of vscode
+in combination with Docker Compose.
+
+To use it, you have to have Docker Compose as well as vscode with its "Remote - Containers"
+extension (`ms-vscode-remote.remote-containers`) installed.
+Then open this repository in vscode and run the command
+`Remote-Containers: Reopen in Container` from the vscode "Command Palette".
+
+This will give you a full-fledged, pre-configured development environment including:
+- infrastructural dependencies of the service (databases, etc.)
+- all relevant vscode extensions pre-installed
+- pre-configured linting and auto-formating
+- a pre-configured debugger
+- automatic license-header insertion
+
+Moreover, inside the devcontainer, a convenience commands `dev_install` is available.
+It installs the service with all development dependencies, installs pre-commit.
+
+The installation is performed automatically when you build the devcontainer. However,
+if you update dependencies in the [`./setup.cfg`](./setup.cfg) or the
+[`./requirements-dev.txt`](./requirements-dev.txt), please run it again.
+
+## License
+This repository is free to use and modify according to the
+[Apache 2.0 License](./LICENSE).
+
+## Readme Generation
+This readme is autogenerate, please see [`readme_generation.md`](./readme_generation.md)
+for details.

.static_files

@@ -15,18 +15,22 @@
 scripts/script_utils/__init__.py
 scripts/script_utils/cli.py
 
+scripts/__init__.py
+scripts/update_all.py
 scripts/license_checker.py
 scripts/get_package_name.py
 scripts/update_config_docs.py
 scripts/update_template_files.py
 scripts/update_openapi_docs.py
+scripts/update_readme.py
 scripts/README.md
 
 .github/workflows/check_config_docs.yaml
 .github/workflows/check_template_files.yaml
 .github/workflows/static_code_analysis.yaml
 .github/workflows/unit_and_int_tests.yaml
 .github/workflows/check_openapi_spec.yaml
+.github/workflows/check_readme.yaml
 .github/workflows/cd.yaml
 
 example_data/README.md
@@ -44,3 +48,6 @@ LICENSE
 requirements.txt
 requirements-dev-common.txt
 setup.py
+
+.readme_template.md
+readme_generation.md

readme_generation.md

@@ -0,0 +1,47 @@
+<!--
+ Copyright 2021 - 2023 Universität Tübingen, DKFZ, EMBL, and Universität zu Köln
+ for the German Human Genome-Phenome Archive (GHGA)
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+     http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+
+-->
+
+# Readme Generation
+
+The README file is generated by collecting information from different sources as
+outlined in the following.
+
+- name: The full name of the package is derived from the remote origin Git repository.
+- title: A title case representation of the name.
+- shortname: An abbreviation of the full name. This is derived from the name mentioned
+  in the [`./setup.cfg`](`./setup.cfg).
+- summary: A short 1-2 sentence summary derived from the description in the
+  [`./setup.cfg`](`./setup.cfg).
+- version: The package version derived from the version specified in the
+  [`./setup.cfg`](`./setup.cfg).
+- description: A markdown-formatted description of the features and use cases of this
+  service or package. Obtained from the [`./.description.md`](./.description.md).
+- design_description: A markdown-formatted description of the overall architecture and
+  design of the package. Obtained from the [`./.design.md`](./.design.md).
+- config_description: A markdown-formatted description of all config parameters.
+  This is autogenerated from the [`./config_schema.json`](./config_schema.json).
+- openapi_doc: A markdown-formatted description of the HTTP API. This is autogenerated
+  and links to the [`./openapi.yaml`](./openapi.yaml). If the openapi.yaml is not
+  this documentation is empty.
+
+The [`./.readme_template.md`](./.readme_template.md) serves as a template where the
+above variable can be filled in using Pythons `string.Template` utility from the
+standard library.
+
+The [`./scripts/update_readme.py`] script can be used to collect all information and
+fill it into the template to generate the README file.

scripts/__init__.py

@@ -0,0 +1,17 @@
+# Copyright 2021 - 2023 Universität Tübingen, DKFZ, EMBL, and Universität zu Köln
+# for the German Human Genome-Phenome Archive (GHGA)
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+"""Scripts and utils used during development or in CI pipelines."""

scripts/update_all.py

@@ -0,0 +1,51 @@
+#!/usr/bin/env python3
+
+# Copyright 2021 - 2023 Universität Tübingen, DKFZ, EMBL, and Universität zu Köln
+# for the German Human Genome-Phenome Archive (GHGA)
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+"""Run all update scripts that are present in the repository in the correct order"""
+
+try:
+    from scripts.update_template_files import main as update_template
+except ImportError:
+    pass
+else:
+    print("Pulling in updates from template repository")
+    update_template()
+
+try:
+    from scripts.update_config_docs import main as update_config
+except ImportError:
+    pass
+else:
+    print("Updating config docs")
+    update_config()
+
+try:
+    from scripts.update_openapi_docs import main as update_openapi
+except ImportError:
+    pass
+else:
+    print("Updating OpenAPI docs")
+    update_openapi()
+
+try:
+    from scripts.update_readme import main as update_readme
+except ImportError:
+    pass
+else:
+    print("Updating README")
+    update_readme()