Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CLI Captures "COOKIE BANNERS" and/or does not work the same as the Web Extension #143

Open
QueenieArtemisia opened this issue Nov 29, 2024 · 0 comments

Comments

@QueenieArtemisia
Copy link

QueenieArtemisia commented Nov 29, 2024

I was trying to download this page with the CLI: https://additional-creatures.fandom.com/wiki/Additional_Creatures:_Aquaria

  • it captures the cookie banner, you can test it.

Secondly, I tried downloading this page the same way: https://game-maps.com/MEA/Mass-Effect-Andromeda-Walkthrough.asp

  • This one downloads a dimmed page that does not have even 20% of the information it should.

Both cases are 100% reproducible for me - even in VMs.


Both pages download just fine with the web extension.
I am trying to use the CLI for these because I want to crawl... but if I do, every single page downloads the same way as the aforementioned - cookie banners on the first site, dimmed and missing info on the second.


Here's my settings code (when crawling):

"C:\Program Files\Google\Chrome\Application\single-file.exe" https://additional-creatures.fandom.com/wiki/Additional_Creatures:_Aquaria --filename-template="{page-title}.{filename-extension}" --crawl-links=true --crawl-inner-links-only=false --crawl-max-depth=1 --crawl-replace-urls=true --crawl-rewrite-rule="^(.)\?.$ $1"


  • Running as admin
  • Antivirus exception

What am I doing wrong?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant