[State Sync] Experiment with using pebble as the execution datastore db

[UlyanaAndrukhiv]

Close #6017

Note: this PR includes:

• changes to go-ds-pebble which are detailed in PR Add WithPebbleDB configuration option to Datastore implementation ipfs/go-ds-pebble#36
• pebble version of ConsumerProgress from PoC PR ([CI Version] Refactor Protocol State with Pebble-based Storage #6206)

Context

In this pull request:

• Added a CLI flag (executionDataDBMode) that controls which db type to use.
• Added DatastoreManager interface that defines the methods for managing a datastore used for handling execution data and badger (BadgerDatastoreManager) and pebble (PebbleDatastoreManager) implementations .
• Added integration test for pebble execution data .

IPFS PR: ipfs/go-ds-pebble#36

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 8.60215% with 170 lines in your changes missing coverage. Please review.

Project coverage is 41.42%. Comparing base (60e14fd) to head (406a64a).

Files	Patch %	Lines
cmd/access/node_builder/access_node_builder.go	0.00%	38 Missing ⚠️
cmd/observer/node_builder/observer_builder.go	0.00%	37 Missing ⚠️
...cutiondatasync/storage/pebble_datastore_manager.go	0.00%	23 Missing ⚠️
storage/pebble/consumer_progress.go	0.00%	20 Missing ⚠️
...executiondatasync/execution_data/execution_data.go	0.00%	16 Missing ⚠️
...cutiondatasync/storage/badger_datastore_manager.go	0.00%	13 Missing ⚠️
storage/pebble/operation/prefix.go	57.14%	12 Missing ⚠️
storage/pebble/operation/jobs.go	0.00%	6 Missing ⚠️
cmd/execution_builder.go	0.00%	5 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #6180      +/-   ##
==========================================
- Coverage   41.49%   41.42%   -0.07%     
==========================================
  Files        2003     2008       +5     
  Lines      142757   142903     +146     
==========================================
- Hits        59232    59196      -36     
- Misses      77352    77532     +180     
- Partials     6173     6175       +2     

Flag	Coverage Δ
unittests	41.42% <8.60%> (-0.07%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[peterargue]

Added a few comments. I like the direction you're taking, but I think we should separate the tracker updates into a separate PR. That way we could begin to experiment with using pebble as the datastore sooner.

For the tracker changes, I think it would make sense to break it down into 2-3 PRs:

1. Split out db code similar to how we've done it for all of the other storage interfaces (e.g. storage.Header)
2. Introduce a new common interface similar to what you have that the badger storage implementation should implement. At this point, there should be no badger specific code left in the tracker/pruner.
3. Add the pebble version of the storage object.
   I think this will help reduce the coupling of the storage and even simplify the interface

[peterargue]

added a few small comments, but I think this is ready to start testing with. we'll need to get the ipfs change merged first (or we can create a fork in the onflow repo if it's going to take a long time)

[peterargue]

I created a fork of ipfs/go-ds-pebble with your changes here: https://github.com/onflow/go-ds-pebble. please update the replace to this repo, then I'll approve

[UlyanaAndrukhiv]

I created a fork of ipfs/go-ds-pebble with your changes here: https://github.com/onflow/go-ds-pebble. please update the replace to this repo, then I'll approve

Thank you, @peter, for creating the forked repo. I have updated the replace.

[Guitarheroua]

Nicely done!