Skip to content
This repository has been archived by the owner on Jul 19, 2024. It is now read-only.

Fix slow response #26

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

longhoangwkm
Copy link

Problem:

curl -XGET localhost:8432/a/1AuGvfDfNhbeVNUDzXHjLmEnFnWv3pkjjG/utxos

Response time: ~60 seconds or time out.

Debug:
To get utxos, we searchRange all blocks by addressToScriptID:
If latest block is ~1M6. Split all blocks to n time to hit db by chunkSize.
Example, latest block is ~1M6. / chunkSize(10) = 160k times.
i found that chunkSize too small, hit to leveldb too much.
Reduce n to help reduce response time

Solution:
I tried to increase chunkSize value to 100, 1000, 10_000 and 100_000.
Benchmark show that with chunkSize = 10_000 return good response Time
< 1 seconds on mainnet

$ time curl localhost:8432/a/1AuGvfDfNhbeVNUDzXHjLmEnFnWv3pkjjG/utxos
real	0m0.638s
user	0m0.005s
sys	0m0.015s

$ time curl localhost:8432/a/1Afb76aMKPTpSppV9nZrf3Nfmg4KXvikeA/utxos
real	0m0.169s
user	0m0.006s
sys	0m0.005s

$ time curl localhost:8432/a/19s5TAWJNdsRxzoHN51TPsWnCieD2h1M1y/utxos
real	0m0.090s
user	0m0.006s
sys	0m0.004s

chunkSize too small(10).
Scan all block take n*M times.
- M is average time to hit to leveldb
- n = highestBlock / chunkSize
Reduce n to help reduce response time
@jdogresorg
Copy link

jdogresorg commented May 30, 2020

I just noticed this pull requests and applied the suggested fix to the indexd instances on my servers and it seems to be running MUCH faster. Thank you very much @longhoangwkm for submitting this fix.

@chiguireitor ^^

@longhoangwkm
Copy link
Author

You're welcome. Btw, we sometimes have problems with exceeding the max limit for the number of node object errors. I think we should keep developing this CIP: https://counterpartytalk.org/t/cip22-change-indexd-to-addrindexrs/5870. We need a big change, I'm eager to contribute to a new project.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants