Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

about the env variables #1

Open
Gusfzzm opened this issue Oct 8, 2023 · 6 comments
Open

about the env variables #1

Gusfzzm opened this issue Oct 8, 2023 · 6 comments

Comments

@Gusfzzm
Copy link

Gusfzzm commented Oct 8, 2023

62e5ce6f0ce11828829068973ec1e12 I'm more interested in knowing how to fill in the values for the variables in the red circle?
@Exca-DK
Copy link
Owner

Exca-DK commented Oct 8, 2023

You can fill them with whatever values you wish to. For testing I would define NODE_UTIL_BROKER_CRAWLER_PASSWORD to crawler_secret and NODE_UTIL_BROKER_SCANNER_PASSWORD to scanner_secret otherwise you will need to also update definitions.json, eg. on Linux:
export NODE_UTIL_SCANNER_REDIS_PASSWORD="foo"
export NODE_UTIL_DB_USERNAME="bar"
export NODE_UTIL_DB_PASSWORD="baz
export NODE_UTIL_BROKER_CRAWLER_PASSWORD="crawler_secret "
export NODE_UTIL_BROKER_SCANNER_PASSWORD="scanner_secret "

@Gusfzzm
Copy link
Author

Gusfzzm commented Oct 9, 2023

Thank you very much. Also, I would like to ask if this code can crawl the nodes in the ethereum network

@Exca-DK
Copy link
Owner

Exca-DK commented Oct 9, 2023

It should be able to crawl any eth-like network.

@Gusfzzm
Copy link
Author

Gusfzzm commented Oct 10, 2023

How do I set this up if I only want to analyze the network for Bitcoin or Ether? Also will this crawler be limited by the region where the server is located, I keep getting stuck here.
1696923486766
Am I missing something in the configuration of the environment or something else?

@Gusfzzm
Copy link
Author

Gusfzzm commented Oct 11, 2023

I managed to run through your code, but where do I go to find the results of the run?
e74a012226f12c9c72fb0437d8915ee

@Exca-DK
Copy link
Owner

Exca-DK commented Oct 11, 2023

How do I set this up if I only want to analyze the network for Bitcoin or Ether?

The example docker-compose at /infra/ should work. You just need to setup the bootstrap nodes and maybe some other flags which are basically defined at readme.

Also will this crawler be limited by the region where the server is located

I am not sure If I blocked crawling private networks so that might get you flagged on cloud, everything except that should be fine since it does come with some basic ratelimits etc.

I keep getting stuck here.

It's docker building image, shouldn't take a lot of time on reasonable vps.

I managed to run through your code, but where do I go to find the results of the run?

The results from crawling and scanning are saved to DB and published to event queue when intrumented. You can access db with nice interface at 127.0.0.1:5058, all of the definitions are in crawler/docker-compose.yaml.

Sorry for the late response, was busy with some stuff.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants