Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Functionality to JUST update existing CSS / XPath Selectors #36

Closed
srhinos opened this issue Mar 27, 2023 · 1 comment
Closed

Functionality to JUST update existing CSS / XPath Selectors #36

srhinos opened this issue Mar 27, 2023 · 1 comment

Comments

@srhinos
Copy link

srhinos commented Mar 27, 2023

I love this as a concept and would love to implement something like this in my project https://github.com/srhinos/primelooter, but my biggest holdup is overall cost.

One alternative I'd really support would be the ability to have the scraper be told to, on top of returning the jsonified data, also return selectors to fetch that data WITHOUT the library using traditional libraries (that are free to use).

This would give me all the benefits of AI to continuously and automotously update my selectors somewhere in code, saving me tons of time, while also minimizing the drawbacks of a ton of cost.

Not sure if this necessarily fits the scope of this project but IMO, would make this much safer to implement in a lot more projects (especially mine <3). Would be more than happy to work on something like this when I catch time over the next few weekends and contribute it back if out of scope for your current short term plans.

@jamesturk
Copy link
Owner

Hi, There's a question in the FAQ about this with my current thoughts: https://jamesturk.github.io/scrapeghost/faq/#why-not-ask-the-scraper-to-write-css-xpath-selectors

It's much harder and would be a different mode of execution since this bypasses the need for selectors and does better. I've been experimenting with this and will start posting updates in #7 soon.

If you really need this I'd love to see what you come up with, but I think it'd likely be best as a different project that depends on this to verify the selectors work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants