duplicity-utils

Tooling around duplicity to streamline backup creation and lifecycle.

development

Project development makes extensive use of emacs, racket, nix and direnv.

Run tests within emacs using racket-mode via C-c C-t. Run all tests from cli via raco test . in the project root.

To create a standalone executable, run raco exe --gui duplicity-fib-discard.rkt (using the old 3m, ‘BC’ runtime)

For coverage analysis, execute raco cover duplicity-fib-discard.rkt and visit file:coverage/index.html

Create documentation by executing e.g. raco scribble literate-p.rkt.

usage

execute ./dublicity-fib-discard.rkt --help

changelog / todos

• [X] configuration + backup of hard-links in ‘docbase’ (see https://github.com/gunther-bachmann/emacs-doc-base) solution: script saves docbase links into deeply nested folder structure in separate text file
• [X] write properties that are checked before deletion is carried out
• [X] complete moving to temporary folder
• [X] cleanup local cache of duplicity meta data (in ~~/.cache/duplicity~) after (re)move of obsolete backup files obsolete: simply cleanup whole cache folder once in a while.
• [X] implement property based testing for these properties obsolete: proof needs no additional testing
• [ ] provide podman based image for a backend that can be queried via rest services
• [ ] write react redux based frontend against that backend

IMPLEMENT new strategy

• [X] algorithm for candidate removal
  • given b_i-1, b_i, b_i+1 (age ordered index into the backups available), b being the age of this backup in months from now
  • further given, two of those three backups cover the same interval iv_n, with the interval length len(iv_n) = { fib(n) for n > 0
  • and given that b_i+1 - b_i-1 < len(iv_n) = fib(n)
  • then: b_i can be safely removed without the risk that it will now or ever add to the interval coverage!
• [X] prove that this algorithm does not worsen interval coverage on the next step (org-roam)
• [X] implement this algorithm
• [ ] translate this proof to agda

OBSOLETE properties of backups to keep

• no backup of the four youngest (available) generations is deleted!
• the oldest backup is never deleted
• a backup that is deleted must satisfy the following (besides the above):
  • given its age is $t_1$, the next younger backup (age $t_0$), the next older backup (age $t_2$)
  • don’t punch holes that leave large black spots! what are large black spots?
  • further back, larger back spots are tolerated
  • given a certain number of backups $n_b$, regardless of their age
  • given the following interval [][-][-][-][--][---][-----][--------][-------------] I want a backup within each interval a backup can be deleted if I can still guarantee, that each interval is populated, even if time goes on (and no further backups are made) if only some intervals are populated to begin with, the situation must not become worse (not even in the future) [o][o][o][o][-o][--o][----o][-------o][------------o] ;; each fib number has a backup generation

          0  1  2  3   5    8     13        21             34

                                1111  11111122  2222222233333
          0  1  2  3  45  678  90123  45678901  2345678901234
         [o][o][o][o][-o][--o][o----][-------o][------------o] ;; now ok, next gen not ok,
      1  [-][o][o][o][o-][o--][oo---][--------][o------------] ;; nok
      2  [-][-][o][o][oo][-o-][-oo--][--------][-o-----------] ;; nok
      3  [-][-][-][o][oo][o-o][--oo-][--------][--o----------] ;; nok
      4  [-][-][-][-][o#][oo-][o--oo][--------][---o---------] ;; nok
      5  [-][-][-][-][-o][##o][-#--o][o-------][----o--------] ;; ok
      6  [-][-][-][-][--][o##][o-#--][o#------][-----o-------] ;; ok
      7  [-][-][-][-][--][-o#][#o-#-][-o#-----][------o------]
      8  [-][-][-][-][--][--o][##o-#][--o#----][-------o-----]
      9  [-][-][-][-][--][---][o###-][#--o#---][--------o----]
     10  [-][-][-][-][--][---][-o###][-#--o#--][---------o---]
     11  [-][-][-][-][--][---][--o##][#-#--o#-][----------o--]
     12  [-][-][-][-][--][---][---o#][##-#--o#][-----------o-]
     13  [-][-][-][-][--][---][----o][###-#--o][#-----------o] ;; next ideal
     14  [-][-][-][-][--][---][-----][oooo-o--][oo-----------]
     15  [-][-][-][-][--][---][-----][-oooo-o-][-oo----------]
     16  [-][-][-][-][--][---][-----][--oooo-o][--oo---------]
     17  [-][-][-][-][--][---][-----][---oooo-][o--oo--------]
     18  [-][-][-][-][--][---][-----][----oooo][-o--oo-------]
     19  [-][-][-][-][--][---][-----][-----ooo][o-o--oo------]
     20  [-][-][-][-][--][---][-----][------oo][oo-o--oo-----]
     21  [-][-][-][-][--][---][-----][-------o][ooo-o--oo----]
     22  [-][-][-][-][--][---][-----][--------][oooo-o--oo---]
     23  [-][-][-][-][--][---][-----][--------][-oooo-o--oo--]
     24  [-][-][-][-][--][---][-----][--------][--oooo-o--oo-]

• rule: the distance between two backups must not be > than the interval width the older one is in
• reformulated: given an interval within which a backup exists, the next older available backup must not be farther away than the interval length (of the older/newer one)
• and! there may not arise this situation in the future => interval borders are of interest here