Add Synapse backup s3-integration (#139)

* Add Synapse backup s3-integration

* Add missing src-docs

* Add integration documentation

* Try integration tests for backup with localstack

* Run integration tests in self hosted runners

* Some small refactoring and renaming

* github actions corrections

* Use different name for fixture function and fixture name

* Fix typo

Co-authored-by: Christopher Bartz <[email protected]>

* Fix typo

Co-authored-by: Christopher Bartz <[email protected]>

* rename s3-backup integrations and improve docs

* back up -> backup for name

* Rename SynapseBackup to BackupObserver and split logic in two files

* Split test_backup in two files

* Swap test files. Improve config change

* Rename integration from s3-backup to backup

* Refactor for integration tests to use fixtures

* Join three tests using pytests parametrize

* Fix bug and add tests for it in validate region or endpoint

* No check for status in backup observer before setting to active

* Remove unused constant

---------

Co-authored-by: Christopher Bartz <[email protected]>

.github/workflows/integration_test.yaml

@@ -11,6 +11,8 @@ jobs:
       chaos-app-label: app.kubernetes.io/name=synapse
       chaos-enabled: false
       chaos-experiments: pod-delete
+      extra-arguments: --localstack-address 172.17.0.1
+      pre-run-script: localstack-installation.sh
       trivy-image-config: "trivy.yaml"
       juju-channel: 3.1/stable
       channel: 1.28-strict/stable

docs/how-to/contribute.md

@@ -80,12 +80,10 @@ and synapse-nginx images are required in the microk8s registry. To enable it:
 The following commands import the images in the Docker daemon and push them into
 the registry:
 
-    cd [project_dir]/synapse_rock && rockcraft pack rockcraft.yaml
-    skopeo --insecure-policy copy oci-archive:synapse_1.0_amd64.rock docker-daemon:localhost:32000/synapse:latest
-    docker push localhost:32000/synapse:latest
-    cd [project_dir]/nginx_rock && rockcraft pack rockcraft.yaml
-    skopeo --insecure-policy copy oci-archive:synapse-nginx_1.0_amd64.rock docker-daemon:localhost:32000/synapse-nginx:latest
-    docker push localhost:32000/synapse-nginx:latest
+    cd [project_dir]/synapse_rock && rockcraft pack
+    skopeo --insecure-policy copy --dest-tls-verify=false oci-archive:synapse_1.0_amd64.rock docker://localhost:32000/synapse:latest
+    cd [project_dir]/nginx_rock && rockcraft pack
+    skopeo --insecure-policy copy --dest-tls-verify=false oci-archive:synapse-nginx_1.0_amd64.rock docker://localhost:32000/synapse-nginx:latest
 
 ### Deploy
 
@@ -95,7 +93,7 @@ juju add-model synapse-dev
 # Enable DEBUG logging
 juju model-config logging-config="<root>=INFO;unit=DEBUG"
 # Deploy the charm (assuming you're on amd64)
-juju deploy ./synapse_ubuntu-20.04-amd64.charm \
+juju deploy ./synapse_ubuntu-22.04-amd64.charm \
   --resource synapse-image=localhost:32000/synapse:latest \
   --resource synapse-nginx-image=localhost:32000/synapse-nginx:latest
 ```

docs/reference/integrations.md

@@ -94,3 +94,17 @@ For the smtp-integrator, insecure configurations with `transport_security=none`
 authenticated connections with `auth_type=none` are not supported.
 
 See more information in [Charm Architecture](https://charmhub.io/synapse/docs/explanation-charm-architecture).
+
+
+### backup
+
+_Interface_: s3
+_Supported charms_: [s3-integrator](https://charmhub.io/s3-integrator/)
+
+In order to perform backups, Synapse has to be integrated with the s3-integrator charm using the
+endpoint backup. Backups will be stored, listed and recovered from the location
+indicated in the S3 compatible object storage provider configuration provided by the integration.
+The Synapse charm will back up the media files, signing keys and sqlite database file if applicable.
+If Synapse database integration is used, the Synapse charm will not back up the related database.
+
+Example backup integrate command: `juju integrate synapse:backup s3-integrator`