Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TypeError: Object of type Element is not JSON serializable - Error when scraping a website #203

Open
vvkishere opened this issue Nov 8, 2024 · 0 comments

Comments

@vvkishere
Copy link

My code running in a windows 10 instance:

from botasaurus.browser import browser, Driver

@browser (reuse_driver=True)
def scrape_data(driver: Driver, link):
driver.get(link)
return driver.get_all_elements_with_exact_text("Equipment Install/Dismantle History")

url = "https://help.sap.com/docs/SAP_S4HANA_ON-PREMISE/ee6ff9b281d8448f96b4fe6c89f2bdc8/a18e976c8a244478a79e3b4177d525dc.html"
output = scrape_data(url)
print(output)

I get a json error:

output = scrape_data(url)
         ^^^^^^^^^^^^^^^^

File "C:\Users\XXX\AppData\Roaming\Python\Python312\site-packages\botasaurus\browser_decorator.py", line 295, in wrapper_browser
write_output(
File "C:\Users\XXX\AppData\Roaming\Python\Python312\site-packages\botasaurus\decorators_common.py", line 202, in write_output
write_json(result, filename, False)
File "C:\Users\XXX\AppData\Roaming\Python\Python312\site-packages\botasaurus\output.py", line 90, in write_json
write_json(data, filename)
File "C:\Users\XXX\AppData\Roaming\Python\Python312\site-packages\botasaurus\utils.py", line 52, in write_json
json.dump(data, fp, indent=indent)
File "C:\Python312\Lib\json_init
.py", line 179, in dump
for chunk in iterable:
^^^^^^^^
File "C:\Python312\Lib\json\encoder.py", line 430, in _iterencode
yield from _iterencode_list(o, _current_indent_level)
File "C:\Python312\Lib\json\encoder.py", line 326, in _iterencode_list
yield from chunks
File "C:\Python312\Lib\json\encoder.py", line 439, in _iterencode
o = _default(o)
^^^^^^^^^^^
File "C:\Python312\Lib\json\encoder.py", line 180, in default
raise TypeError(f'Object of type {o.class.name} '
TypeError: Object of type Element is not JSON serializable

the JSON library is working so I am not sure what the issue is.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant