Resetting Testnet: Bitcoin Explorers, Indexers & APIs

[CharlieC3]

Today we discussed the pros/cons of running a Stacks Testnet against a private Bitcoin Regtest network. One of the challenges we're looking to understand is the difficulties and feature-gaps of running a Bitcoin explorer, indexer, and API which other entities and groups in the Stacks ecosystem may rely upon, as many of these services are available on the internet for Bitcoin mainnet (e.g. https://mempool.space/).

Some folks at Hiro, such as myself, @zone117x and @saralab, researched a bunch of different OSS in this field in early 2023. Our research led us to understand that the mempool frontend & backend projects, along with the mempool electrs indexer/API had the best feature-set and alleged performance and reliability at the time.
This is what we're using to power https://mempool.bitcoin.regtest.hiro.so/

To better understand the challenges in running these services and their feature-gaps, I thought it would be useful to provide the research we conducted in early 2023 to facilitate a discussion around this matter in hopes that it helps us make a more informed decision on what the future of the Stacks Testnet network may look like.

It should be noted that:

• Many of the different projects in the Bitcoin explorer/api/indexer space are often forks of each other with usually minor differences between them. So if there are incompatibility issues found in one of our selections, replacing the component with a different project may not resolve any compatibility issues as they may be based on the same code, or may introduce other challenges that the original project accounts for, such as performance or reliability concerns.
• As this research was conducted in early 2023, take it with a grain of salt and when in doubt verify it again as the status for some of these projects may have changed.
• The mempool electrs indexer/API that we chose is not mentioned in the list below, as it was created between the time we finished this research and implemented it. But since it was based on romanz/electrs and Blockstream/electrs which our research favored, it checked all the boxes and made the most sense to use with the mempool frontend & backend projects.

Bitcoin Indexer Comparison (early 2023)

Project	Protocol	Prod-ready	Deployment Scope	Scalability	Metrics support	Community Support	Verdict
romanz/electrs	⚠️ Electrum	❌ Note that this implementation of Electrum server is optimized for personal/small-scale (family/friends) usage. It's a bad idea to run it publicly as it'd expose you to DoS and maybe also other attacks.	⚠️ Hiro would need to build and maintain a Docker image Requires an empty disk to store database locally	❌ Does not scale well; scaling up would require a separate database. This could be done with snapshots potentially, but is time-consuming and increases overhead. Load balancing may return differing data with session persistence.	✅ (Prometheus)	✅ > 500 stars Latest official release: 1/5/23	❌ 1 Intended for small-scale operations
blockstream/electrs	✅ Electrum + Esplora	✅ Used as the backend for the Esplora block explorer powering blockstream.info.	⚠️ Hiro would need to build and maintain a Docker image Requires an empty disk to store database locally	❌ Does not scale well; scaling up would require a separate database. This could be done with snapshots potentially, but is time-consuming and increases overhead. Load balancing may return differing data with session persistence.	✅ (Prometheus)	⚠️ ~200 stars Latest official release: 10/14/18 Latest commit to default branch: 6/22/22 The GH issues page has been hidden; no clear way to submit new issues to track current ones	⚠️ 3 Flexible due to combination of protocols May need to change the storage engine to improve scalability.
coinbase/rosetta-bitcoin	✅ Rosetta	❌ ROSETTA-BITCOIN IS CONSIDERED ALPHA SOFTWARE.	⚠️ Hiro would need to build and maintain a Docker image Requires an empty disk to store database locally	❌ Does not scale well; scaling up would require a separate database. This could be done with snapshots potentially, but is time-consuming and increases overhead. Load balancing may return differing data with session persistence.	❌	⚠️ ~90 stars Latest official release: 2/8/22 Latest commit to default branch: 11/1/22	❌ 1 Coinbase is not running this and has not maintained it for a while
dpc/rust-bitcoin-indexer	✅ Custom (Postgres)	⚠️ An experiment in creating a perfect Bitcoin Indexer ... This started as an educational experiment, but is quite advanced already. ... rust-bitcoin-indexer (at least right now) is just a neat experiment that gave good results and seems to be working quite well.	✅ Hiro would need to build and maintain a Docker image Requires a Postgres database deployed	⚠️ Uses a Postgres db, so the app can theoretically scale horizontally. Since this project is not prod-ready, we’d still need to verify how well the app performs under heavy load.	❌	❌ ~90 stars No official releases/tags Latest commit to default branch: 5/17/21	❌ 1 Recognized as an “experiment”, low dev activity
cculianu/Fulcrum	⚠️ Electrum (Fulcrum fork)	❔ unclear	⚠️ Hiro would need to build and maintain a Docker image Requires an empty disk to store database locally	❌ Does not scale well; scaling up would require a separate database. This could be done with snapshots potentially, but is time-consuming and increases overhead. Load balancing may return differing data with session persistence.	❌	✅ ~160 stars Latest official release: 12/20/22 Active development, issues being resolved regularly	❌ 2 Poor scaling capabilities, Electrum protocol has limitations
spesmilo/electrumx	⚠️ Electrum	❔ unclear	⚠️ Hiro would need to build and maintain a Docker image Requires an empty disk to store database locally	❌ Does not scale well; scaling up would require a separate database. This could be done with snapshots potentially, but is time-consuming and increases overhead. Load balancing may return differing data with session persistence.	❌	⚠️ ~300 stars Latest official release: 12/10/20 Latest commit to default branch: 11/10/22 Despite no official release/tag in a while, there’s still plenty of active development on this project	❌ 2 Poor scaling capabilities, Electrum protocol has limitations
block-core/blockcore-indexer	✅ Blockcore	✅ Used as the backend for the Blockcore.net explorer	✅ Official Docker image exists Requires a MongoDB database deployed	⚠️ Uses MongoDB, so the app can theoretically scale horizontally better than an app using a local directory can. Since this project is used by Blockcore.net, chances are it has some capability to scale. It seems blockcore.net is a personal project or doesn’t receive high traffic. Scaling exploration would need to be carried out	❌	⚠️ 17 stars Latest official release: 12/16/22 Latest commit to default branch: 12/21/22 Seems like a small team actively working on this project; issues getting resolved at a slow pace. If Hiro needs a new feature added to bug fixed, we might need to contribute. Written in C#; Hiro doesn’t have many C# devs besides Matt 🙂	⚠️ 2.5 Low community usage Hiro isn’t historically a C# shop Low confidence in its usage for a low traffic site
bitauth/chaingraph	✅ Custom (GraphQL + Postgres)	❌ Limited BTC support (was designed for BCH)	✅ Official Docker images exist Requires a Postgres database deployed	⚠️ Uses a Postgres db, so the app can theoretically scale horizontally. Since this project is not prod-ready, we’d still need to verify how well the app performs under heavy load. Also has a K8s Helm chart, making is simple to deploy internally.	⚠️ Really limited and custom metrics. Might be able to find a way to stream this into Prometheus	⚠️ 24 stars Latest official release: 5/14/22 Latest commit to default branch: 1/6/23 Despite some active development, issues are not getting resolved at all	❌ 1 Limited/questionable BTC functionality BTC support is unofficial
bitpay/bitcore	✅ Custom (REST)	✅ Used as the backend for the Bitpay.com Explorer	⚠️ Hiro would need to build and maintain a Docker image Requires a MongoDB database deployed Some endpoints may need to be blocked.	✅ Uses MongoDB, and supports specification of the db connection details in the config, so the app can theoretically scale horizontally. Since this project is used by Bitpay.com, chances are it has some capability to scale.	⚠️ Really limited and custom metrics. Might be able to find a way to stream this into Prometheus	✅ 4.5k stars Latest official release: 10/16/22 (npm) 1/2/23 (gh tag) Active development, issues being resolved regularly	⚠️ 3.5 Good dev activity and community support Used in high(ish) traffic Explorer Scales with remote database
mempool/mempool	✅ Custom (REST) Integrates with a backend Electrum or Esplora server to increase functionality.	✅ Used as the backend for the mempool.space	✅ Official Docker images exist May require deployment of an Electrum server for increased functionality. Seems to be compatible with romanz/electrs or cculianu/Fulcrum and Blockstream/electrs. Lots of configuration suggestions for deploying to production.	✅ Uses MariaDB, and supports specification of the db connection details in the config, so the app can theoretically scale horizontally. Since this project is used by mempool.space, chances are it has some capability to scale.	❌	✅ 1.2k stars Latest official release/tag: 7/25/22 Despite no new release in 6 months, there’s active development almost every day against master, issues being resolved regularly	✅ Charlie: 4 Matt: 4 Sarala : 4

Protocol usability

The Electrum protocol has a limitation that makes fetching txs for a given block pretty expensive (see blockchain.transaction.id_from_pos).

This may be a considered a downside for the following projects: romanz/electrs, cculianu/Fulcrum, spesmilo/electrumx.

---