Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

always use pg_dump for DB backups #893

Merged
merged 5 commits into from
Jul 19, 2024
Merged

always use pg_dump for DB backups #893

merged 5 commits into from
Jul 19, 2024

Conversation

evgeni
Copy link
Member

@evgeni evgeni commented Jun 26, 2024

  • always use pg_dump for DB backups
  • drop --include-db-dumps parameter, we always include them now
  • unify DB backup steps

includes #891 for the checks

for_feature :candlepin_database
preparation_steps { Checks::Candlepin::DBUp.new unless feature(:candlepin_database).local? }
param :backup_dir, 'Directory where to backup to', :required => true
param :tar_volume_size, 'Size of tar volume (indicates splitting)'
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's funny this was defined, but never passed to the procedure (heck, even the CLI param is called --split-pulp-tar), so yay less unused complexity?

@evgeni
Copy link
Member Author

evgeni commented Jun 26, 2024

# foreman-maintain backup offline --incremental /var/off-size/katello-backup-2024-06-26-10-03-42/ /var/lol
Starting backup: 2024-06-26 10:32:22 +0000
Running preparation steps required to run the next scenarios
================================================================================
Make sure Foreman DB is up: 
/ Checking connection to the Foreman DB                               [OK]      
--------------------------------------------------------------------------------
Make sure Candlepin DB is up: 
- Checking connection to the Candlepin DB                             [OK]      
--------------------------------------------------------------------------------
Make sure Pulpcore DB is up: 
\ Checking connection to the Pulpcore DB                              [OK]      
--------------------------------------------------------------------------------


Running Backup
================================================================================
Check if the incremental backup has the right type:                   [FAIL]
The existing backup has PostgreSQL as a tarball, but the new one will have a dump.
--------------------------------------------------------------------------------
Scenario [Backup] failed.

The following steps ended up in failing state:

  [backup-incremental-parent-type]

Resolve the failed steps and rerun the command.
In case the failures are false positives, use
--whitelist="backup-incremental-parent-type"



Running Failed backup cleanup
================================================================================
Start applicable services: 

Starting the following service(s):
redis, postgresql, pulpcore-api, pulpcore-content, [email protected], [email protected], tomcat, dynflow-sidekiq@orchestrator, foreman, httpd, dynflow-sidekiq@worker-1, dynflow-sidekiq@worker-hosts-queue-1, foreman-proxy
- All services started                                                [OK]      
--------------------------------------------------------------------------------
Clean up backup directory:                                            [OK]
--------------------------------------------------------------------------------

Done with backup: 2024-06-26 10:32:23 +0000
Backup didn't finish. Incomplete backup was removed.

Gotta find out how to move that check to the prep steps.

@ehelms
Copy link
Member

ehelms commented Jul 18, 2024

@evgeni Are you holding on merge to perform testing first?

@evgeni
Copy link
Member Author

evgeni commented Jul 18, 2024

Yeah, we were trying to run tests and they ended up red, so … waiting

evgeni added 3 commits July 19, 2024 09:26
when restoring incremental backups, you can run in the situation where
the on-disk representation of the database changes between the restore
of the base and the increment, leading to a broken database.

by using logical DB dumps, we ensure that the restore always work at the
cost that increments are slightly bigger as they now always include a
full DB dump

this also allows simplification of the code as we now only have one type
of DB backup to support
@evgeni
Copy link
Member Author

evgeni commented Jul 19, 2024

ah, nice. aaac56d did actually break things for this PR \o/

@evgeni evgeni merged commit 0478459 into master Jul 19, 2024
8 checks passed
@evgeni evgeni deleted the always-dump branch July 19, 2024 12:21
evgeni added a commit to evgeni/forklift that referenced this pull request Jul 24, 2024
ekohl pushed a commit to theforeman/forklift that referenced this pull request Jul 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants