Create backup action

[javierdelapuente]

Applicable spec: ISD095

Overview

This PR creates the action create-backup to back up Synapse, using the s3 integration to get
S3 credentials and the backup_passphrase config variable to use as the encryption
key.

To create a backup, a tar file is created with all the files that are required for the
backup, then it is encrypted and sent to a S3 compatible object storage. All these
operations are done in the Synapse container using container.exec with a shell
command using pipes, like tar -c <paths> | gpg --symmetric <options> | aws s3 cp <options>. For
that command to work, the gpg passphrase to use was previously copied to the
container filesystem as a file and the aws client configured with the necessary options and
environment variables.

Besides, the aws s3 command has an option to give the expected size of the object to upload (--expected-size ). This is required so the aws s3 cp can calculate the correct size for the multipart uploads. This is done in this PR using a bare du command with all the paths to back up.

Rationale

Synapse data and config directories should be backed up for disaster and recovery purposes.

Juju Events Changes

Added create-backup action.

Module Changes

The entry point for the logic is the backup_observer, which register a handler for the create-backup action.

All the rest of the logic is inside the backup module, that performs several actions in the workload to
stream the backup, as explained in the overview section.

Library Changes

Checklist

• The charm style guide was applied
• The contributing guide was applied
• The changes are compliant with ISD054 - Manging Charm Complexity
• The documentation is generated using src-docs
• The documentation for charmhub is updated.
• The PR is tagged with appropriate label (urgent, trivial, complex)

[javierdelapuente]

Not sure this is the right place. Any idea?

[arturo-seijas]

Do you mean the key location? This would imply running synapse as root user, which contravenes the least privilege principle

[javierdelapuente]

That file is to store the passphrase key to use when encrypting (using the gpg command).

I put it in a file so I do not have it in a shell command (not run something like tar | gpg -password plainpassword | aws s3 ....

But it is an interesting point, running the backup as another user, not the root, that is the default when running container.exec (I am going to check if it works). where do you think would be a good place to put the file?

[arturo-seijas]

Perhaps under the application directory (/srv/synapse/... I guess) or under $HOME (wherever that is)?

[javierdelapuente]

I have updated the location of the passphrase file to Synapse config dir. I haven't changed to run it as the Synapse user as it is managed by the startup script and it is not fully configured (has no home dir for example).

WDYT?

[javierdelapuente]

try catch

[github-actions]

Test coverage for 9060821

Name                            Stmts   Miss Branch BrPart  Cover   Missing
---------------------------------------------------------------------------
src/actions/__init__.py             3      0      0      0   100%
src/actions/register_user.py       22      0      2      0   100%
src/actions/reset_instance.py      21      0      2      0   100%
src/backup.py                     130      6     20      3    94%   216-217, 261-262, 278->280, 281, 337
src/backup_observer.py             58      0      4      0   100%
src/charm.py                      222      8     50      4    96%   165-166, 227-228, 247-248, 289, 303
src/charm_state.py                 63      1     10      1    97%   127
src/charm_types.py                 23      0      6      0   100%
src/database_client.py             53      1     10      3    94%   35, 47->exit, 69->exit
src/database_observer.py           48      4      4      0    92%   69-71, 87
src/exceptions.py                   4      1      0      0    75%   22
src/mjolnir.py                     76      6     20      1    91%   60-64, 73
src/observability.py                9      0      0      0   100%
src/pebble.py                      97     12     16      6    84%   92-93, 95-96, 104, 106, 108, 112, 133-134, 149-150
src/saml_observer.py               45      1      8      0    98%   64
src/smtp_observer.py               70      3     14      1    95%   70-74, 96->101
src/synapse/__init__.py             3      0      0      0   100%
src/synapse/api.py                161      2     22      2    98%   207, 376
src/synapse/workload.py           270      8     42      6    96%   406->exit, 435-436, 488-489, 525-526, 542, 591->594, 640, 648->650, 650->652
src/user.py                        24      0      4      0   100%
---------------------------------------------------------------------------
TOTAL                            1402     53    234     27    95%

Static code analysis report

Run started:2024-02-06 08:07:57.344881

Test results:
  No issues identified.

Code scanned:
  Total lines of code: 7527
  Total lines skipped (#nosec): 4
  Total potential issues skipped due to specifically being disabled (e.g., #nosec BXXX): 0

Run metrics:
  Total issues (by severity):
  	Undefined: 0
  	Low: 0
  	Medium: 0
  	High: 0
  Total issues (by confidence):
  	Undefined: 0
  	Low: 0
  	Medium: 0
  	High: 0
Files skipped (0):

[javierdelapuente]

closed as it will be done in another branch