Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Main task never ends in case of there is no result in google map. #188

Open
iRajul opened this issue Sep 13, 2024 · 0 comments
Open

Main task never ends in case of there is no result in google map. #188

iRajul opened this issue Sep 13, 2024 · 0 comments

Comments

@iRajul
Copy link

iRajul commented Sep 13, 2024

how to reproduce

Search for string "botototootototototototototootototototoot"

it will give following error :

Traceback (most recent call last):
  File "C:\Python312\Lib\site-packages\botasaurus_server\task_executor.py", line 192, in run_task
    self.update_parent_task(task_id, result)
  File "C:\Python312\Lib\site-packages\botasaurus_server\retry_on_db_error.py", line 37, in wrapper
    raise e
  File "C:\Python312\Lib\site-packages\botasaurus_server\retry_on_db_error.py", line 26, in wrapper
    created_result = func(*args, **kwargs)
                     ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python312\Lib\site-packages\botasaurus_server\retry_on_db_error.py", line 77, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python312\Lib\site-packages\botasaurus_server\task_executor.py", line 205, in update_parent_task
    self.complete_parent_task_if_possible(parent_id, Server.get_remove_duplicates_by(scraper_name), result)
  File "C:\Python312\Lib\site-packages\botasaurus_server\task_executor.py", line 230, in complete_parent_task_if_possible
    fn()
  File "C:\Python312\Lib\site-packages\botasaurus_server\task_executor.py", line 227, in <lambda>
    fn = lambda: TaskHelper.read_clean_save_task(parent_id, remove_duplicates_by, status)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python312\Lib\site-packages\botasaurus_server\task_helper.py", line 190, in read_clean_save_task
    rs = normalize_dicts_by_fieldnames(TaskResults.get_all_task(parent_id))
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python312\Lib\site-packages\botasaurus_server\cleaners.py", line 35, in normalize_dicts_by_fieldnames
    for item in data:
TypeError: 'NoneType' object is not iterable
Error in run_task  'NoneType' object is not iterable

Potential Fix:
Return an empty list from function normalize_dicts_by_fieldnames

def normalize_dicts_by_fieldnames(data):
    if data is None or len(data) == 0:
        return []
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant