-
Notifications
You must be signed in to change notification settings - Fork 845
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Streaming to postgresql is Extremely slow. #457
Comments
This could be useful https://github.com/blockchain-etl/ethereum-etl-postgres |
Thank you, I have another question: I am trying to download the data locally, is that a possibility? I do not wish to use Cloud SQL |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I noticed that streaming is very very slow.
I'm running an Erigon Archive node locally
with this command
ethereumetl stream -e block,transaction,token_transfer,token,log --provider-uri http://127.0.0.1:8545 --output=postgresql+pg8000://user:pw@127.0.0.1:5432/db--period-seconds 0 --max-workers 1000
It took 2 hours to index 15963 blocks. (recorded the time)
this is indexing old blocks starting from block 10,000,000
I am also running another one that's indexing the blockchain from the last block onwards (to keep database up to date)
Any advice on how to speed it up?
The text was updated successfully, but these errors were encountered: