Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(scanner): Add a new zebra-scanner binary #8608

Merged
merged 7 commits into from
Jul 9, 2024
Merged

Conversation

oxarbitrage
Copy link
Contributor

@oxarbitrage oxarbitrage commented Jun 12, 2024

Motivation

We want to run the blockchain scanner in a new binary and process.

Close #8581
Close #8633

PR Author Checklist

Check before marking the PR as ready for review:

  • Will the PR name make sense to users?
  • Does the PR have a priority label?
  • Have you added or updated tests?
  • Is the documentation up to date?

Solution

Use zebra_state::init_read_only to run zebra_scan::spawn_init from a new binary.

Testing

Arguments:

     Running `target/debug/zebra-scanner --help`
zebra-scan 0.1.0-alpha.6
zebra-scanner arguments

USAGE:
    zebra-scanner [OPTIONS] --zebra-rpc-listen-addr <zebra-rpc-listen-addr>

FLAGS:
    -h, --help       Prints help information
    -V, --version    Prints version information

OPTIONS:
        --listen-addr <listen-addr>                         IP address and port for the gRPC server
        --network <network>                                 The Zcash network [default: Mainnet]
        --sapling-keys-to-scan <sapling-keys-to-scan>...    The sapling keys to scan for
        --scanning-cache-dir <scanning-cache-dir>           Path to scanning state [default: /home/alfredo/.cache/zebra]
        --zebra-rpc-listen-addr <zebra-rpc-listen-addr>
            The listen address of Zebra's RPC server used by the syncer to check for chain tip changes and get blocks in
            Zebra's non-finalized state
        --zebrad-cache-dir <zebrad-cache-dir>               Path to zebrad state [default: /home/alfredo/.cache/zebra]

I am using the following command to run the scanner with the ZECPAGES key, open the grpc port and see the logs:

RUST_LOG=info cargo run --bin zebra-scanner --features="shielded-scan" -- --sapling-keys-to-scan {\"key\":\"zxviews1q0duytgcqqqqpqre26wkl45gvwwwd706xw608hucmvfalr759ejwf7qshjf5r9aa7323zulvz6plhttp5mltqcgs9t039cx2d09mgq05ts63n8u35hyv6h9nc9ctqqtue2u7cer2mqegunuulq2luhq3ywjcz35yyljewa4mgkgjzyfwh6fr6jd0dzd44ghk0nxdv2hnv4j5nxfwv24rwdmgllhe0p8568sgqt9ckt02v2kxf5ahtql6s0ltjpkckw8gtymxtxuu9gcr0swvz\"} --zebrad-cache-dir "/media/alfredo/stuff/chain/zebra" --listen-addr "127.0.0.1:8231" --zebra-rpc-listen-addr "127.0.0.1:8232"

Output:

     Running `target/debug/zebra-scanner --sapling-keys-to-scan '{"key":"zxviews1q0duytgcqqqqpqre26wkl45gvwwwd706xw608hucmvfalr759ejwf7qshjf5r9aa7323zulvz6plhttp5mltqcgs9t039cx2d09mgq05ts63n8u35hyv6h9nc9ctqqtue2u7cer2mqegunuulq2luhq3ywjcz35yyljewa4mgkgjzyfwh6fr6jd0dzd44ghk0nxdv2hnv4j5nxfwv24rwdmgllhe0p8568sgqt9ckt02v2kxf5ahtql6s0ltjpkckw8gtymxtxuu9gcr0swvz"}' --zebrad-cache-dir /media/alfredo/stuff/chain/zebra --listen-addr '127.0.0.1:8231' --zebra-rpc-listen-addr '127.0.0.1:8232'`
2024-06-24T18:36:42.193678Z  INFO zebra_state::service::finalized_state::disk_format::upgrade: trying to open current database format running_version=25.3.0
2024-06-24T18:36:42.193959Z  INFO zebra_state::service::finalized_state::disk_db: the open file limit is high enough for Zebra current_limit=1024 min_limit=512 ideal_limit=1024
2024-06-24T18:36:42.226420Z  INFO zebra_state::service::finalized_state::disk_db: creating secondary db directory create_dir_result=Ok(())
2024-06-24T18:36:46.533748Z  INFO zebra_state::service::finalized_state::disk_db: Opened Zebra state cache at /media/alfredo/stuff/chain/zebra/state/v25/mainnet
2024-06-24T18:36:46.534008Z  INFO zebra_state::service::finalized_state: loaded Zebra state cache tip=Some((Height(2547784), block::Hash("0000000000d74176f15e0f742a43b70723899a518d708c079d69b46662ca6a5d")))
2024-06-24T18:36:46.534745Z  INFO zebra_scan::init: starting scan service config=Config { sapling_keys_to_scan: 1, db_config: Config { cache_dir: "/home/alfredo/.cache/zebra", ephemeral: false, delete_old_database: true, debug_stop_at_height: None, debug_validity_check_interval: None } }
2024-06-24T18:36:46.534980Z  INFO zebra_state::service::finalized_state::disk_format::upgrade: creating new database with the current format running_version=1.0.0
2024-06-24T18:36:46.535066Z  INFO zebra_state::service::finalized_state::disk_db: the open file limit is high enough for Zebra current_limit=1024 min_limit=512 ideal_limit=1024
2024-06-24T18:36:46.552968Z  INFO zebra_state::service::finalized_state::disk_db: Opened Zebra state cache at /home/alfredo/.cache/zebra/private-scan/v1/mainnet
2024-06-24T18:36:46.553081Z  INFO zebra_scan::storage::db: loaded Zebra scanner cache
2024-06-24T18:36:46.553416Z  INFO zebra_scan::init: starting scan gRPC server listen_addr=127.0.0.1:8231
2024-06-24T18:36:46.553464Z  INFO zebra_scan::service::scan_task::scan: starting scan task network=Mainnet
2024-06-24T18:36:46.600463Z  INFO zebra_scan::service::scan_task::scan: scanner finished waiting for Sapling activation. Current tip: 2547784, Sapling activation: 419200
2024-06-24T18:36:46.600681Z  INFO zebra_scan::service::scan_task::scan: got min scan height start_height=Height(419199)
2024-06-24T18:37:08.772641Z  INFO zebra_scan::service::scan_task::scan: Scanning the blockchain for key 0, started at block 419200, now at block 420000, current tip 2547784
2024-06-24T18:38:40.466124Z  INFO zebra_scan::service::scan_task::scan: Scanning the blockchain for key 0, started at block 419200, now at block 430000, current tip 2547784
2024-06-24T18:40:08.461023Z  INFO zebra_scan::service::scan_task::scan: Scanning the blockchain for key 0, started at block 419200, now at block 440000, current tip 2547784
2024-06-24T18:41:38.833909Z  INFO zebra_scan::service::scan_task::scan: Scanning the blockchain for key 0, started at block 419200, now at block 450000, current tip 2547784

...

Review

Reviewer Checklist

Check before approving the PR:

  • Does the PR scope match the ticket?
  • Are there enough tests to make sure it works? Do the tests cover the PR motivation?
  • Are all the PR blockers dealt with?
    PR blockers can be dealt with in new tickets or PRs.

And check the PR Author checklist is complete.

Follow Up Work

@oxarbitrage oxarbitrage added A-blockchain-scanner Area: Blockchain scanner of shielded transactions P-Medium ⚡ labels Jun 12, 2024
@github-actions github-actions bot added the C-feature Category: New features label Jun 12, 2024
@oxarbitrage
Copy link
Contributor Author

  • Need changelog entry.

Copy link
Contributor

@arya2 arya2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good, it may need to select the env-filter feature on tracing-subscriber, but otherwise I see no blockers.

Thank you!

zebra-scan/src/bin/scanner/main.rs Outdated Show resolved Hide resolved
zebra-scan/Cargo.toml Outdated Show resolved Hide resolved
zebra-scan/Cargo.toml Outdated Show resolved Hide resolved
@oxarbitrage
Copy link
Contributor Author

oxarbitrage commented Jun 17, 2024

This should be merged only after the next release. #8321

@oxarbitrage oxarbitrage added the do-not-merge Tells Mergify not to merge this PR label Jun 17, 2024
@oxarbitrage oxarbitrage changed the base branch from main to non-final-read-state June 21, 2024 19:09
@oxarbitrage oxarbitrage force-pushed the zebra-scanner-binary branch from 1f056c6 to 3bb5a9e Compare June 21, 2024 23:00
@oxarbitrage oxarbitrage marked this pull request as ready for review June 24, 2024 18:46
@oxarbitrage oxarbitrage requested a review from a team as a code owner June 24, 2024 18:46
@oxarbitrage oxarbitrage requested review from arya2 and removed request for a team June 24, 2024 18:46
arya2
arya2 previously approved these changes Jun 27, 2024
Copy link
Contributor

@arya2 arya2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good!

@mpguerra mpguerra linked an issue Jul 1, 2024 that may be closed by this pull request
2 tasks
@arya2 arya2 removed the do-not-merge Tells Mergify not to merge this PR label Jul 2, 2024
Base automatically changed from non-final-read-state to main July 9, 2024 14:15
@mergify mergify bot dismissed arya2’s stale review July 9, 2024 14:15

The base branch was changed.

@mergify mergify bot requested a review from a team as a code owner July 9, 2024 14:15
@mergify mergify bot requested review from arya2 and removed request for a team July 9, 2024 14:15
@oxarbitrage oxarbitrage force-pushed the zebra-scanner-binary branch from 01fdf97 to 23f72e9 Compare July 9, 2024 15:14
@oxarbitrage oxarbitrage self-assigned this Jul 9, 2024
@mergify mergify bot merged commit a94b2be into main Jul 9, 2024
132 checks passed
@mergify mergify bot deleted the zebra-scanner-binary branch July 9, 2024 18:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-blockchain-scanner Area: Blockchain scanner of shielded transactions C-feature Category: New features P-Medium ⚡
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Use sync::init_read_state_with_syncer() in the zebra-scanner Create a new binary zebra-scanner
2 participants