Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix xbanxia #2579

Open
ecl95 opened this issue Feb 27, 2025 · 1 comment
Open

Fix xbanxia #2579

ecl95 opened this issue Feb 27, 2025 · 1 comment

Comments

@ecl95
Copy link

ecl95 commented Feb 27, 2025

Let us know

Novel URL: https://www.xbanxia.com/books/241190.html
App Location: pip
App Version: 3.7.5

Describe this issue

No chapters found

Error message

Retrieving novel info...
Exception in thread Thread-1 (read_novel_info):
Traceback (most recent call last):
File "/Users/user_1/opt/anaconda3/lib/python3.12/threading.py", line 1075, in _bootstrap_inner
self.run()
File "/Users/user_1/opt/anaconda3/lib/python3.12/threading.py", line 1012, in run
self._target(*self._args, **self._kwargs)
File "/Users/user_1/opt/anaconda3/lib/python3.12/site-packages/sources/zh/xbanxia.py", line 43, in read_novel_info
soup = self.get_soup(self.novel_url)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/user_1/opt/anaconda3/lib/python3.12/site-packages/lncrawl/core/scraper.py", line 307, in get_soup
response = self.get_response(url, headers=headers, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/user_1/opt/anaconda3/lib/python3.12/site-packages/lncrawl/core/scraper.py", line 202, in get_response
return self.__process_request(
^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/user_1/opt/anaconda3/lib/python3.12/site-packages/lncrawl/core/scraper.py", line 131, in __process_request
raise e
File "/Users/user_1/opt/anaconda3/lib/python3.12/site-packages/lncrawl/core/scraper.py", line 124, in __process_request
response.raise_for_status()
File "/Users/user_1/opt/anaconda3/lib/python3.12/site-packages/requests/models.py", line 1024, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://www.xbanxia.com/books/241190.html

❗ Error: No chapters found
<class 'Exception'>
File "/Users/user_1/opt/anaconda3/lib/python3.12/site-packages/lncrawl/bots/console/integration.py", line 109, in start
raise e
File "/Users/user_1/opt/anaconda3/lib/python3.12/site-packages/lncrawl/bots/console/integration.py", line 103, in start
_download_novel()
File "/Users/user_1/opt/anaconda3/lib/python3.12/site-packages/lncrawl/bots/console/integration.py", line 87, in _download_novel
self.app.get_novel_info()
File "/Users/user_1/opt/anaconda3/lib/python3.12/site-packages/lncrawl/core/app.py", line 138, in get_novel_info
raise Exception("No chapters found")

@ecl95
Copy link
Author

ecl95 commented Feb 28, 2025

I was playing around and for some reason this works if line 210 of scraper.py is reverted to
def post_response(self, url, data={}, retry=1, **kwargs) -> Response:
from
' def post_response(self, url, data={}, retry=0, **kwargs) -> Response:'
, the source works again.

Is there a reason this was changed? I can't seem to figure out why this makes such a difference.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant