MacOS - Message: disconnected: not connected to DevTools (failed to check if window was closed: disconnected: not connected to DevTools) #357

miklyx · 2023-04-06T18:23:51Z

HI!
Today I've got fresh repo update and latest chrome update.
And every 20 mins getting bot fail just after immoscout24 link
Traceback (most recent call last): File "/Users/kristina/Documents/python/flathunter_bot/flathunter/flathunt.py", line 118, in <module> main() File "/Users/kristina/Documents/python/flathunter_bot/flathunter/flathunt.py", line 114, in main launch_flat_hunt(config, heartbeat) File "/Users/kristina/Documents/python/flathunter_bot/flathunter/flathunt.py", line 45, in launch_flat_hunt hunter.hunt_flats() File "/Users/kristina/Documents/python/flathunter_bot/flathunter/flathunter/hunter.py", line 56, in hunt_flats for expose in processor_chain.process(self.crawl_for_exposes(max_pages)): File "/Users/kristina/Documents/python/flathunter_bot/flathunter/flathunter/hunter.py", line 35, in crawl_for_exposes return chain(*[try_crawl(searcher, url, max_pages) File "/Users/kristina/Documents/python/flathunter_bot/flathunter/flathunter/hunter.py", line 35, in <listcomp> return chain(*[try_crawl(searcher, url, max_pages) File "/Users/kristina/Documents/python/flathunter_bot/flathunter/flathunter/hunter.py", line 27, in try_crawl return searcher.crawl(url, max_pages) File "/Users/kristina/Documents/python/flathunter_bot/flathunter/flathunter/abstract_crawler.py", line 150, in crawl return self.get_results(url, max_pages) File "/Users/kristina/Documents/python/flathunter_bot/flathunter/flathunter/crawler/immobilienscout.py", line 90, in get_results soup = self.get_page(search_url, self.get_driver(), page_no) File "/Users/kristina/Documents/python/flathunter_bot/flathunter/flathunter/crawler/immobilienscout.py", line 175, in get_page return self.get_soup_from_url( File "/Users/kristina/.local/share/virtualenvs/flathunter-Ytmhlxpi/lib/python3.9/site-packages/backoff/_sync.py", line 105, in retry ret = target(*args, **kwargs) File "/Users/kristina/Documents/python/flathunter_bot/flathunter/flathunter/abstract_crawler.py", line 80, in get_soup_from_url driver.get(url) File "/Users/kristina/.local/share/virtualenvs/flathunter-Ytmhlxpi/lib/python3.9/site-packages/undetected_chromedriver/__init__.py", line 476, in get_wrapped if self.execute_script("return navigator.webdriver"): File "/Users/kristina/.local/share/virtualenvs/flathunter-Ytmhlxpi/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py", line 500, in execute_script return self.execute(command, {"script": script, "args": converted_args})["value"] File "/Users/kristina/.local/share/virtualenvs/flathunter-Ytmhlxpi/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py", line 440, in execute self.error_handler.check_response(response) File "/Users/kristina/.local/share/virtualenvs/flathunter-Ytmhlxpi/lib/python3.9/site-packages/selenium/webdriver/remote/errorhandler.py", line 245, in check_response raise exception_class(message, screen, stacktrace) selenium.common.exceptions.WebDriverException: Message: disconnected: not connected to DevTools (failed to check if window was closed: disconnected: not connected to DevTools) (Session info: chrome=112.0.5615.49)

The text was updated successfully, but these errors were encountered:

miklyx · 2023-04-07T13:15:37Z

Updated all packages and chrome - and error still presents

codders · 2023-04-11T16:45:52Z

Hi @miklyx ,

What are you using as the driver_arguments for chromedriver in your config? Are you running inside of docker or directly from a terminal?

miklyx · 2023-04-11T20:53:53Z

Hi!
I use
driver_arguments: - "--headless" - "--no-sandbox" - "--disable-gpu" - "--remote-debugging-port=9222" - "--disable-dev-shm-usage" - "window-size=1024,768"

also, I run it directly from terminal.

codders · 2023-04-15T20:36:52Z

Hi @miklyx ,

You can try changing "window-size=1024,768" for "--window-size=1024,768" - that might help. Also, if you're running from the console you can drop the "--headless" argument and see what happens in the browser.

miklyx · 2023-04-16T12:28:31Z

Hi! Done it. Result is - it starts to open search page at immoscout24. After some cycles it throws error again

Traceback (most recent call last): File "/Users/kristina/Documents/python/flathunter_bot/flathunter/flathunt.py", line 118, in <module> main() File "/Users/kristina/Documents/python/flathunter_bot/flathunter/flathunt.py", line 114, in main launch_flat_hunt(config, heartbeat) File "/Users/kristina/Documents/python/flathunter_bot/flathunter/flathunt.py", line 45, in launch_flat_hunt hunter.hunt_flats() File "/Users/kristina/Documents/python/flathunter_bot/flathunter/flathunter/hunter.py", line 56, in hunt_flats for expose in processor_chain.process(self.crawl_for_exposes(max_pages)): File "/Users/kristina/Documents/python/flathunter_bot/flathunter/flathunter/hunter.py", line 35, in crawl_for_exposes return chain(*[try_crawl(searcher, url, max_pages) File "/Users/kristina/Documents/python/flathunter_bot/flathunter/flathunter/hunter.py", line 35, in <listcomp> return chain(*[try_crawl(searcher, url, max_pages) File "/Users/kristina/Documents/python/flathunter_bot/flathunter/flathunter/hunter.py", line 27, in try_crawl return searcher.crawl(url, max_pages) File "/Users/kristina/Documents/python/flathunter_bot/flathunter/flathunter/abstract_crawler.py", line 150, in crawl return self.get_results(url, max_pages) File "/Users/kristina/Documents/python/flathunter_bot/flathunter/flathunter/crawler/immobilienscout.py", line 90, in get_results soup = self.get_page(search_url, self.get_driver(), page_no) File "/Users/kristina/Documents/python/flathunter_bot/flathunter/flathunter/crawler/immobilienscout.py", line 175, in get_page return self.get_soup_from_url( File "/Users/kristina/Library/Python/3.9/lib/python/site-packages/backoff/_sync.py", line 105, in retry ret = target(*args, **kwargs) File "/Users/kristina/Documents/python/flathunter_bot/flathunter/flathunter/abstract_crawler.py", line 80, in get_soup_from_url driver.get(url) File "/Users/kristina/Library/Python/3.9/lib/python/site-packages/undetected_chromedriver/__init__.py", line 645, in get return super().get(url) File "/Users/kristina/Library/Python/3.9/lib/python/site-packages/selenium/webdriver/remote/webdriver.py", line 449, in get self.execute(Command.GET, {"url": url}) File "/Users/kristina/Library/Python/3.9/lib/python/site-packages/selenium/webdriver/remote/webdriver.py", line 440, in execute self.error_handler.check_response(response) File "/Users/kristina/Library/Python/3.9/lib/python/site-packages/selenium/webdriver/remote/errorhandler.py", line 245, in check_response raise exception_class(message, screen, stacktrace) selenium.common.exceptions.WebDriverException: Message: disconnected: not connected to DevTools (failed to check if window was closed: disconnected: not connected to DevTools) (Session info: chrome=112.0.5615.49)

silasburger · 2023-05-16T15:21:32Z

I'm experiencing the same

taraskatsora1998 · 2023-05-23T07:48:12Z

I have the same issue too

codders · 2023-05-24T08:27:18Z

@silasburger @taraskatsora1998 can you share more about your setup? What platform are you running on? Are you running inside docker? How much RAM is available to the processes? What browser (Chrome or Chromium) are you using and at what version? Do you have the latest code?

cibanez-teladoc · 2023-06-05T13:16:33Z

Same issue here.
Platform: macOS Ventura 13.3.1 (a) (22E772610a)
Chrome driver: mac-arm64/114.0.5735.90
Chrome browser version: 114.0.5735.90 (Official Build) (arm64)
RAM: 16 GB
Running on my local machine.

arpitpaliwaldh · 2023-06-14T14:37:34Z

Facing same issue. Is there solution for this?

codders · 2023-06-14T14:55:03Z

@arpitpaliwaldh the solution is collecting more data. Can you confirm what platform you're using?

pembertonrw · 2023-06-15T09:58:56Z

I'm facing the same issue and I've got a similar setup to @cibanez-teladoc. I encounter this issue as soon as it's trying to interact with the page i.e. it goes to the webpage OK but as soon as it tryings to type something it, it fails.

Platform: macOS Ventura 13.4 (22F66)
Chrome driver: mac-arm64/114.0.5735.90
Chrome browser version: 114.0.5735.106 (Official Build) (arm64)
RAM: 16 GB
Java: 11.0.11
Running on my local machine.

arpitpaliwaldh · 2023-06-15T10:08:44Z

Yes i think the issue is with the latest chrome version and chrome driver not compatible. Downgraded chrome and chromedriver version on my machine now test is working.

Platform: macOS Ventura 13.3
Chrome driver: 114.0.5735.90
Chrome browser version: 114.0.5735.106

miklyx · 2023-06-15T10:19:49Z

Hi! Actually - It's floating error. Sometimes it presents, sometimes process is working well for a couple of days.
Also - after some fallings it may start a phantom "google chrome helper renderer" process with >100% cpu usage.

codders · 2023-06-15T14:57:04Z

@miklyx Thanks for the info. Are you also running on Mac with Chrome?

TomerLankri · 2023-06-15T15:03:32Z

Following having same error :(

codders · 2023-06-15T15:05:41Z

@TomerLankri Please share your platform details to help us with the debugging

codders · 2023-06-15T15:05:56Z

@TomerLankri Please share your platform details to help us with the debugging

TomerLankri · 2023-06-15T15:13:58Z

Platform: macOS Ventura 13.3.1
Chrome driver: mac-arm64/114.0.5735.90
Chrome browser version: 114.0.5735.90
RAM: 16 GB
Running on my local machine.

miklyx · 2023-06-15T16:29:46Z

@miklyx Thanks for the info. Are you also running on Mac with Chrome?

I'm running latest mac OS with latest chrome

hubx · 2023-06-25T08:37:46Z

Just wanted to confirm that the Stacktrace is the same as above but was cut of and continues as the referenced issue #52

selenium.common.exceptions. (failed to check if (Session info: chrome=114.0 Stacktrace:
0 undetected_chromedriver 1 undetected_chromedriver 2 undetected_chromedriver 3 undetected_chromedriver 4 undetected_chromedriver 5 undetected_chromedriver 6 undetected_chromedriver 7 undetected_chromedriver 8 undetected_chromedriver 9 undetected_chromedriver 10 undetected_chromedriver 11 undetected_chromedriver 12 undetected_chromedriver 13 undetected_chromedriver 14 undetected_chromedriver 15 undetected_chromedriver 16 undetected_chromedriver 17 undetected_chromedriver 18 libsystem_pthread.dylib 19 libsystem_pthread.dylib WebDriverException: Message: disconnected: not connected to DevTools
window was closed: disconnected: not connected to DevTools)
.5735.133)
0x00000001050e56b8 undetected_chromedriver + 4937400
0x00000001050dcb73 undetected_chromedriver + 4901747
0x0000000104c9a616 undetected_chromedriver + 435734
0x0000000104c80bc1 undetected_chromedriver + 330689
0x0000000104c80923 undetected_chromedriver + 330019
0x0000000104c9c262 undetected_chromedriver + 442978
0x0000000104d1c56e undetected_chromedriver + 968046
0x0000000104d01de3 undetected_chromedriver + 859619
0x0000000104ccfd7f undetected_chromedriver + 654719
0x0000000104cd10de undetected_chromedriver + 659678
0x00000001050a12ad undetected_chromedriver + 4657837
0x00000001050a6130 undetected_chromedriver + 4677936
0x00000001050acdef undetected_chromedriver + 4705775
0x00000001050a705a undetected_chromedriver + 4681818
0x000000010507992c undetected_chromedriver + 4495660
0x00000001050c4838 undetected_chromedriver + 4802616
0x00000001050c49b7 undetected_chromedriver + 4802999
0x00000001050d599f undetected_chromedriver + 4872607
0x00007ff814af31d3 _pthread_start + 125
0x00007ff814aeebd3 thread_start + 15

For now what works for me is a good old bash loop that automatically retries while true; do python flathunt.py || sleep 5; done

miklyx · 2023-06-26T21:17:52Z

At fresh repo version and updated selenium and IMDBTraktSyncer I've got

[2023/06/26 23:16:17|patcher.py |INFO ]: patching driver executable /Users/mrashkovskiy/Library/Application Support/undetected_chromedriver/undetected_chromedriver
Traceback (most recent call last):
File "/Users/mrashkovskiy/Documents/py/flathunt/flathunt_2_0/flathunter/flathunt.py", line 99, in
main()
File "/Users/mrashkovskiy/Documents/py/flathunt/flathunt_2_0/flathunter/flathunt.py", line 95, in main
launch_flat_hunt(config, heartbeat)
File "/Users/mrashkovskiy/Documents/py/flathunt/flathunt_2_0/flathunter/flathunt.py", line 35, in launch_flat_hunt
hunter.hunt_flats()
File "/Users/mrashkovskiy/Documents/py/flathunt/flathunt_2_0/flathunter/flathunter/hunter.py", line 56, in hunt_flats
for expose in processor_chain.process(self.crawl_for_exposes(max_pages)):
File "/Users/mrashkovskiy/Documents/py/flathunt/flathunt_2_0/flathunter/flathunter/hunter.py", line 35, in crawl_for_exposes
return chain([try_crawl(searcher, url, max_pages)
File "/Users/mrashkovskiy/Documents/py/flathunt/flathunt_2_0/flathunter/flathunter/hunter.py", line 35, in
return chain([try_crawl(searcher, url, max_pages)
File "/Users/mrashkovskiy/Documents/py/flathunt/flathunt_2_0/flathunter/flathunter/hunter.py", line 27, in try_crawl
return searcher.crawl(url, max_pages)
File "/Users/mrashkovskiy/Documents/py/flathunt/flathunt_2_0/flathunter/flathunter/abstract_crawler.py", line 148, in crawl
return self.get_results(url, max_pages)
File "/Users/mrashkovskiy/Documents/py/flathunt/flathunt_2_0/flathunter/flathunter/crawler/immobilienscout.py", line 90, in get_results
soup = self.get_page(search_url, self.get_driver(), page_no)
File "/Users/mrashkovskiy/Documents/py/flathunt/flathunt_2_0/flathunter/flathunter/crawler/immobilienscout.py", line 65, in get_driver
self.driver = get_chrome_driver(driver_arguments)
File "/Users/mrashkovskiy/Documents/py/flathunt/flathunt_2_0/flathunter/flathunter/chrome_wrapper.py", line 47, in get_chrome_driver
driver = uc.Chrome(version_main=chrome_version, options=chrome_options) # pylint: disable=no-member
File "/Users/mrashkovskiy/opt/anaconda3/lib/python3.9/site-packages/undetected_chromedriver/init.py", line 441, in init
super(Chrome, self).init(
TypeError: init() got an unexpected keyword argument 'executable_path'

miklyx · 2023-06-26T21:32:51Z

This is my pip list
Package Version

aiohttp 3.8.3
aiosignal 1.3.1
alabaster 0.7.12
anaconda-client 1.9.0
anaconda-navigator 2.1.1
anaconda-project 0.10.1
antiorm 1.2.1
anyio 2.2.0
appdirs 1.4.4
applaunchservices 0.2.1
appnope 0.1.2
apprise 1.2.1
appscript 1.1.2
argh 0.26.2
argon2-cffi 20.1.0
arrow 0.13.1
asn1crypto 1.4.0
astroid 2.6.6
astropy 4.3.1
async-generator 1.10
async-timeout 4.0.2
atomicwrites 1.4.0
attrs 21.2.0
autopep8 1.5.7
Babel 2.9.1
backcall 0.2.0
backoff 2.2.1
backports.functools-lru-cache 1.6.4
backports.shutil-get-terminal-size 1.0.0
backports.tempfile 1.0
backports.weakref 1.0.post1
beautifulsoup4 4.10.0
binaryornot 0.4.4
bitarray 2.3.0
bkcharts 0.2
black 19.10b0
bleach 4.0.0
bokeh 2.4.1
boto 2.49.0
Bottleneck 1.3.2
brotlipy 0.7.0
cached-property 1.5.2
certifi 2021.10.8
cffi 1.14.6
chardet 4.0.0
charset-normalizer 2.0.4
click 8.0.3
cloudpickle 2.0.0
clyent 1.2.2
colorama 0.4.4
conda 4.10.3
conda-build 3.21.5
conda-content-trust 0+unknown
conda-pack 0.6.0
conda-package-handling 1.7.3
conda-repo-cli 1.0.4
conda-token 0.3.0
conda-verify 3.4.2
contextlib2 0.6.0.post1
cookiecutter 1.7.2
cryptography 3.4.8
cycler 0.10.0
Cython 0.29.24
cytoolz 0.11.0
daal4py 2021.3.0
dask 2021.10.0
db 0.1.1
debugpy 1.4.1
decorator 5.1.0
defusedxml 0.7.1
diff-match-patch 20200713
distributed 2021.10.0
docutils 0.17.1
entrypoints 0.3
et-xmlfile 1.1.0
exceptiongroup 1.1.0
fastcache 1.1.0
feedparser 6.0.10
filelock 3.3.1
flake8 3.9.2
Flask 1.1.2
fonttools 4.25.0
frozenlist 1.3.3
fsspec 2021.8.1
future 0.18.2
gevent 21.8.0
glob2 0.7
gmpy2 2.0.8
greenlet 1.1.1
h11 0.14.0
h5py 3.2.1
HeapDict 1.0.1
html5lib 1.1
idna 3.2
imagecodecs 2021.8.26
imageio 2.9.0
imagesize 1.2.0
IMDBTraktSyncer 1.5.2
importlib-metadata 4.8.1
inflection 0.5.1
iniconfig 1.1.1
instaloader 4.9.6
intervaltree 3.1.0
ipykernel 6.4.1
ipython 7.29.0
ipython-genutils 0.2.0
ipywidgets 7.6.5
isort 5.9.3
itsdangerous 2.0.1
jdcal 1.4.1
jedi 0.18.0
Jinja2 2.11.3
jinja2-time 0.2.0
joblib 1.1.0
json5 0.9.6
jsonpath-ng 1.5.3
jsonschema 3.2.0
jupyter 1.0.0
jupyter-client 6.1.12
jupyter-console 6.4.0
jupyter-core 4.8.1
jupyter-server 1.4.1
jupyterlab 3.2.1
jupyterlab-pygments 0.1.2
jupyterlab-server 2.8.2
jupyterlab-widgets 1.0.0
keyring 23.1.0
kiwisolver 1.3.1
lazy-object-proxy 1.6.0
libarchive-c 2.9
llvmlite 0.37.0
locket 0.2.1
lxml 4.6.3
Markdown 3.4.1
MarkupSafe 1.1.1
matplotlib 3.4.3
matplotlib-inline 0.1.2
mccabe 0.6.1
mistune 0.8.4
mkl-fft 1.3.1
mkl-random 1.2.2
mkl-service 2.4.0
mock 4.0.3
more-itertools 8.10.0
mpmath 1.2.1
msgpack 1.0.2
multidict 6.0.2
multipledispatch 0.6.0
munkres 1.1.4
mypy-extensions 0.4.3
navigator-updater 0.2.1
nbclassic 0.2.6
nbclient 0.5.3
nbconvert 6.1.0
nbformat 5.1.3
nest-asyncio 1.5.1
networkx 2.6.3
nltk 3.6.5
nose 1.3.7
notebook 6.4.5
numba 0.54.1
numexpr 2.7.3
numpy 1.20.3
numpydoc 1.1.0
oauthlib 3.2.2
olefile 0.46
openpyxl 3.0.9
outcome 1.2.0
packaging 21.0
pandas 1.3.4
pandocfilters 1.4.3
parso 0.8.2
partd 1.2.0
path 16.0.0
pathlib2 2.3.6
pathspec 0.7.0
patsy 0.5.2
pep8 1.7.1
pexpect 4.8.0
pickleshare 0.7.5
Pillow 8.4.0
pip 23.1.2
pkginfo 1.7.1
pluggy 0.13.1
ply 3.11
poyo 0.5.0
prometheus-client 0.11.0
prompt-toolkit 3.0.20
psutil 5.8.0
psycopg2-binary 2.9.5
ptyprocess 0.7.0
py 1.10.0
pycodestyle 2.7.0
pycosat 0.6.3
pycparser 2.20
pycurl 7.44.1
pydocstyle 6.1.1
pyerfa 2.0.0
pyflakes 2.3.1
Pygments 2.10.0
PyJWT 2.1.0
pylint 2.9.6
pyls-spyder 0.4.0
pyodbc 4.0.0-unsupported
pyOpenSSL 21.0.0
pyparsing 3.0.4
pyrsistent 0.18.0
PySocks 1.7.1
pytest 6.2.4
python-dateutil 2.8.2
python-dotenv 0.21.1
python-lsp-black 1.0.0
python-lsp-jsonrpc 1.0.0
python-lsp-server 1.2.4
python-slugify 5.0.2
pytz 2021.3
PyWavelets 1.1.1
PyYAML 6.0
pyzmq 22.2.1
QDarkStyle 3.0.2
qstylizer 0.1.10
QtAwesome 1.0.2
qtconsole 5.1.1
QtPy 1.10.0
random-user-agent 1.0.1
regex 2021.8.3
requests 2.26.0
requests-oauthlib 1.3.1
requests-random-user-agent 2023.2.15
rope 0.19.0
Rtree 0.9.7
ruamel-yaml-conda 0.15.100
scikit-image 0.18.3
scikit-learn 0.24.2
scikit-learn-intelex 2021.20210714.100439
scipy 1.7.1
seaborn 0.11.2
selenium 4.10.0
Send2Trash 1.8.0
setuptools 58.0.4
sgmllib3k 1.0.0
simplegeneric 0.8.1
singledispatch 3.7.0
sip 4.19.13
six 1.16.0
sniffio 1.2.0
snowballstemmer 2.1.0
sortedcollections 2.1.0
sortedcontainers 2.4.0
soupsieve 2.2.1
spacexpy 2.0.2
Sphinx 4.2.0
sphinxcontrib-applehelp 1.0.2
sphinxcontrib-devhelp 1.0.2
sphinxcontrib-htmlhelp 2.0.0
sphinxcontrib-jsmath 1.0.1
sphinxcontrib-qthelp 1.0.3
sphinxcontrib-serializinghtml 1.1.5
sphinxcontrib-websupport 1.2.4
spyder 5.1.5
spyder-kernels 2.1.3
SQLAlchemy 1.4.22
statsmodels 0.12.2
sympy 1.9
tables 3.6.1
TBB 0.2
tblib 1.7.0
terminado 0.9.4
testpath 0.5.0
text-unidecode 1.3
textdistance 4.2.1
threadpoolctl 2.2.0
three-merge 0.1.1
tifffile 2021.7.2
tinycss 0.4
toml 0.10.2
toolz 0.11.1
tornado 6.1
tqdm 4.62.3
traitlets 5.1.0
trio 0.22.0
trio-websocket 0.9.2
typed-ast 1.4.3
typing-extensions 3.10.0.2
ujson 4.0.2
undetected-chromedriver 3.2.1
unicodecsv 0.14.1
Unidecode 1.2.0
urllib3 1.26.7
watchdog 2.1.3
wcwidth 0.2.5
webencodings 0.5.1
websockets 10.4
Werkzeug 2.0.2
wheel 0.37.0
whichcraft 0.6.1
widgetsnbextension 3.5.1
wrapt 1.12.1
wsproto 1.2.0
wurlitzer 2.1.1
xlrd 2.0.1
XlsxWriter 3.0.1
xlwings 0.24.9
xlwt 1.3.0
xmltodict 0.12.0
yapf 0.31.0
yarl 1.8.1
zict 2.0.0
zipp 3.6.0
zope.event 4.5.0
zope.interface 5.4.0

codders · 2023-06-27T07:54:18Z

Hi @miklyx ,

Not sure what IMDBTraktSyncer has to do with it - that's not in the pipfile. Are you using pipenv to install the dependencies?

codders · 2023-06-27T07:57:10Z

If you're not using pipenv to run the project, there can be unexpected interactions with other libraries installed. I would strongly recommend using pipenv install to install dependencies and pipenv shell to launch the project, to ensure you're using the same versions of all the libraries as other users of the project.

miklyx · 2023-06-27T08:53:04Z

Sorry, with pipenv it still works. I'll write if it'll be an errors

miklyx · 2023-06-28T08:36:58Z

With pipenv - It worked yesterday all day and dropped today - kinda record! no falls in 24 hours. MacOs Ventura 13.4.1, Python 3.9.7, all other from pipenv.
trace looks the same

`[2023/06/28 10:30:29|immobilienscout.py |DEBUG ]: Got search URL https://www.immobilienscout24.de/Suche/de/berlin/berlin/wohnung-mieten?haspromotion=false&price=-1100.0&livingspace=50.0-&exclusioncriteria=projectlisting,swapflat&pricetype=rentpermonth&geocodes=110000000703,110000000101,110000000201,110000000102,110000000301,110000000202,110000000401,110000000105,110000000701&sorting=2&enteredFrom=result_list&pagenumber={0}
Traceback (most recent call last):
File "/Users/mrashkovskiy/Documents/py/flathunt/flathunt_2_0/flathunter/flathunt.py", line 99, in
main()
File "/Users/mrashkovskiy/Documents/py/flathunt/flathunt_2_0/flathunter/flathunt.py", line 95, in main
launch_flat_hunt(config, heartbeat)
File "/Users/mrashkovskiy/Documents/py/flathunt/flathunt_2_0/flathunter/flathunt.py", line 44, in launch_flat_hunt
hunter.hunt_flats()
File "/Users/mrashkovskiy/Documents/py/flathunt/flathunt_2_0/flathunter/flathunter/hunter.py", line 56, in hunt_flats
for expose in processor_chain.process(self.crawl_for_exposes(max_pages)):
File "/Users/mrashkovskiy/Documents/py/flathunt/flathunt_2_0/flathunter/flathunter/hunter.py", line 35, in crawl_for_exposes
return chain([try_crawl(searcher, url, max_pages)
File "/Users/mrashkovskiy/Documents/py/flathunt/flathunt_2_0/flathunter/flathunter/hunter.py", line 35, in
return chain([try_crawl(searcher, url, max_pages)
File "/Users/mrashkovskiy/Documents/py/flathunt/flathunt_2_0/flathunter/flathunter/hunter.py", line 27, in try_crawl
return searcher.crawl(url, max_pages)
File "/Users/mrashkovskiy/Documents/py/flathunt/flathunt_2_0/flathunter/flathunter/abstract_crawler.py", line 148, in crawl
return self.get_results(url, max_pages)
File "/Users/mrashkovskiy/Documents/py/flathunt/flathunt_2_0/flathunter/flathunter/crawler/immobilienscout.py", line 90, in get_results
soup = self.get_page(search_url, self.get_driver(), page_no)
File "/Users/mrashkovskiy/Documents/py/flathunt/flathunt_2_0/flathunter/flathunter/crawler/immobilienscout.py", line 175, in get_page
return self.get_soup_from_url(
File "/Users/mrashkovskiy/.local/share/virtualenvs/flathunter-N5-Pi6pW/lib/python3.9/site-packages/backoff/_sync.py", line 105, in retry
ret = target(*args, **kwargs)
File "/Users/mrashkovskiy/Documents/py/flathunt/flathunt_2_0/flathunter/flathunter/abstract_crawler.py", line 70, in get_soup_from_url
driver.get(url)
File "/Users/mrashkovskiy/.local/share/virtualenvs/flathunter-N5-Pi6pW/lib/python3.9/site-packages/undetected_chromedriver/init.py", line 473, in get_wrapped
if self.execute_script("return navigator.webdriver"):
File "/Users/mrashkovskiy/.local/share/virtualenvs/flathunter-N5-Pi6pW/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py", line 406, in execute_script
return self.execute(command, {"script": script, "args": converted_args})["value"]
File "/Users/mrashkovskiy/.local/share/virtualenvs/flathunter-N5-Pi6pW/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py", line 346, in execute
self.error_handler.check_response(response)
File "/Users/mrashkovskiy/.local/share/virtualenvs/flathunter-N5-Pi6pW/lib/python3.9/site-packages/selenium/webdriver/remote/errorhandler.py", line 245, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: disconnected: not connected to DevTools
(failed to check if window was closed: disconnected: not connected to DevTools)
(Session info: chrome=114.0.5735.133)
Stacktrace:
0 undetected_chromedriver 0x000000010536f6b8 undetected_chromedriver + 4937400
1 undetected_chromedriver 0x0000000105366b73 undetected_chromedriver + 4901747
2 undetected_chromedriver 0x0000000104f24616 undetected_chromedriver + 435734
3 undetected_chromedriver 0x0000000104f0abc1 undetected_chromedriver + 330689
4 undetected_chromedriver 0x0000000104f0a923 undetected_chromedriver + 330019
5 undetected_chromedriver 0x0000000104f26262 undetected_chromedriver + 442978
6 undetected_chromedriver 0x0000000104fa656e undetected_chromedriver + 968046
7 undetected_chromedriver 0x0000000104f8bde3 undetected_chromedriver + 859619
8 undetected_chromedriver 0x0000000104f59d7f undetected_chromedriver + 654719
9 undetected_chromedriver 0x0000000104f5b0de undetected_chromedriver + 659678
10 undetected_chromedriver 0x000000010532b2ad undetected_chromedriver + 4657837
11 undetected_chromedriver 0x0000000105330130 undetected_chromedriver + 4677936
12 undetected_chromedriver 0x0000000105336def undetected_chromedriver + 4705775
13 undetected_chromedriver 0x000000010533105a undetected_chromedriver + 4681818
14 undetected_chromedriver 0x000000010530392c undetected_chromedriver + 4495660
15 undetected_chromedriver 0x000000010534e838 undetected_chromedriver + 4802616
16 undetected_chromedriver 0x000000010534e9b7 undetected_chromedriver + 4802999
17 undetected_chromedriver 0x000000010535f99f undetected_chromedriver + 4872607
18 libsystem_pthread.dylib 0x00007ff8019531d3 _pthread_start + 125
19 libsystem_pthread.dylib 0x00007ff80194ebd3 thread_start + 15

[2023/06/28 10:30:29|init.py |INFO ]: ensuring close`

Lamiamrad · 2023-07-11T00:27:14Z

Hello Guys,

I am facing the same issue as yours, I am working under
MacBook pro
MacOs 13.4.1.

did u get any chance to fix this issue ?

Thanks :)

rjborba · 2023-07-28T17:17:04Z

Hey guys. Same issue for M1 Pro Ventura 13.4.1

Johnny0405 · 2023-07-31T06:09:17Z

Hey Guys,
i am facing the same Problem.
MacBook pro MacOs 13.4.1.
When I remove "--remote-debugging-port=9222" it works.

miklyx · 2023-07-31T17:51:25Z

Actually that works! No fails all the day

miklyx · 2023-08-01T17:24:04Z

Early I was delighted - it fell. But it worked for almost 2 days

ManiRenukuntla · 2023-08-10T13:39:24Z

Hi Guys,

After chrome browser updated to v115. I have faced same issue.

my system config is:
MAC M1 chip
selenium == 4.11.0

I'm able to execute it on my PC using "locust -f ./locust.py -u 1 -r 1 --headless"
But I couldn't execute it in the Docker container on my machine.

I'm facing Devtools error.
ERROR/builderUI_Test: BuilderUI_loginFlowMessage: unknown error: Chrome failed to start: crashed.
performance-slave-1 | (unknown error: DevToolsActivePort file doesn't exist)
performance-slave-1 | (The process started from chrome location /usr/bin/chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
performance-slave-1 | Stacktrace:
performance-slave-1 | #0 0x0040006a7613
performance-slave-1 | #1 0x0040003d3537
performance-slave-1 | #2 0x0040003fe588
performance-slave-1 | #3 0x0040003fb5d9
performance-slave-1 | #4 0x00400043b643
performance-slave-1 | #5 0x00400043ad8f
performance-slave-1 | #6 0x004000432803
performance-slave-1 | #7 0x00400040668b
performance-slave-1 | #8 0x00400040742e
performance-slave-1 | #9 0x004000668c28
performance-slave-1 | #10 0x00400066caf7
performance-slave-1 | #11 0x00400067701c
performance-slave-1 | #12 0x00400066d723
performance-slave-1 | #13 0x00400063c457
performance-slave-1 | #14 0x004000691628
performance-slave-1 | #15 0x0040006917b7
performance-slave-1 | #16 0x0040006a0483
performance-slave-1 | #17 0x004002550ea7 start_thread
performance-slave-1 |

filipopo · 2023-08-21T16:12:28Z

Just ran into this issue, it seems that the argument
--disable-gpu
Was causing this DevTools error for me, removing it made Chrome work for me

codders · 2023-08-22T13:59:02Z

@filipopo - Thanks for the report! Does that work for any of the other Mac OS users? (I only have Linux to test on here).

A1exMiranda · 2023-09-04T19:02:02Z

I'm facing the same problem on a Linux server. It seems to be working when restarting the server but after a few cycles the problem reappears, I'll try removing the argument "--disable-gpu" and tell you if it works.

A1exMiranda · 2023-09-11T15:39:07Z

Error still persists after a few hours of running right, "--disable-gpu" didn't seem to fix the issue for me.

The error I'm getting is:
selenium.common.exceptions.WebDriverException: Message: disconnected: not connected to DevTools
(failed to check if window was closed: disconnected: not connected to DevTools)
(Session info: headless chrome=114.0.5735.90)

These are my actual arguments:
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('--window-size=1920,1080')
chrome_options.add_argument('--headless')
chrome_options.add_argument('--disable-dev-shm-usage')
chrome_options.add_argument('--remote-debugging-port=9222')
driver = webdriver.Chrome("/usr/local/bin/chromedriver",options=chrome_options)

anees1221 · 2023-10-15T17:36:53Z

I have the same problem
I am using windows 10
I started experiencing the same problem when updated chrome browser to 115. All the later updates have the same problem.
Today I have installed chrome 118. I though it would be fixed but the same problem persists.

A1exMiranda · 2023-10-24T22:15:59Z

I've found a version that seems to work, I've been testing it for a couple of weeks.

Try with Google Chrome for Testing 117.0.5938.62 and ChromeDriver 117.0.5938.62.

These are my arguments:
chrome_options = webdriver.ChromeOptions()
service = Service('/usr/local/bin/chromedriver')
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('--headless')
chrome_options.add_argument('--remote-debugging-port=9222')
chrome_options.add_argument('--disable-gpu')
driver = webdriver.Chrome(service=service,options=chrome_options)

anees1221 · 2023-11-01T14:20:27Z

@A1exMiranda
Thank you for your reply, it definitely it works with any version of Google Chrome for Testing

anees1221 · 2023-11-14T14:45:05Z

I've found a version that seems to work, I've been testing it for a couple of weeks.

Try with Google Chrome for Testing 117.0.5938.62 and ChromeDriver 117.0.5938.62.

These are my arguments: chrome_options = webdriver.ChromeOptions() service = Service('/usr/local/bin/chromedriver') chrome_options.add_argument('--no-sandbox') chrome_options.add_argument('--headless') chrome_options.add_argument('--remote-debugging-port=9222') chrome_options.add_argument('--disable-gpu') driver = webdriver.Chrome(service=service,options=chrome_options)

@A1exMiranda again facing the same issue with Google Chrome for Testing.

kothaaridarshi · 2023-12-18T19:14:57Z

@anees1221 @A1exMiranda I am facing the same issue did you able to fix it?

anees1221 · 2024-01-14T03:46:42Z

@kothaaridarshi Unfortunately no.

codders · 2024-01-17T19:21:32Z

Mandatory XKCD: https://xkcd.com/2881

A1exMiranda · 2024-01-26T22:27:06Z

Try updating selenium too. The version I have is 4.12.0.
I'm getting new updates of chrome and chromedriver and I've been running with no issues for the past 3 months.

ChromeDriver 119.0.6045.105
Google Chrome for Testing 119.0.6045.105

I really hope upgrading fixes your issues.

nienienienie · 2024-04-05T11:09:59Z

same here

tsbxmw · 2024-04-25T07:36:56Z

same in win10 & chrome 124 & java 11 & selenium 4.19.1
same .jar file, at agent A, it always failed. but another agent works well.

(failed to check if window was closed: disconnected: not connected to DevTools)
  (Session info: chrome=124.0.6367.78)
Build info: version: '4.19.1', revision: 'abe0ee07dc'
System info: os.name: 'Windows 10', os.arch: 'amd64', os.version: '10.0', java.version: '11.0.16'
Driver info: org.openqa.selenium.chrome.ChromeDriver
Command: [c65f8d45f6d213834ce8053b8c32abe8, get {url=https://www.google.com}]
Capabilities {acceptInsecureCerts: false, browserName: chrome, browserVersion: 124.0.6367.78, chrome: {chromedriverVersion: 124.0.6367.78 (a087f2dd364d..., userDataDir: C:\Users\TP-CESHI\AppData\L...}, fedcm:accounts: true, goog:chromeOptions: {debuggerAddress: localhost:61009}, networkConnectionEnabled: false, pageLoadStrategy: normal, platformName: windows, proxy: Proxy(), se:cdp: ws://localhost:61009/devtoo..., se:cdpVersion: 124.0.6367.78, setWindowRect: true, strictFileInteractability: false, timeouts: {implicit: 0, pageLoad: 300000, script: 30000}, unhandledPromptBehavior: dismiss and notify, webauthn:extension:credBlob: true, webauthn:extension:largeBlob: true, webauthn:extension:minPinLength: true, webauthn:extension:prf: true, webauthn:virtualAuthenticators: true}
Session ID: c65f8d45f6d213834ce8053b8c32abe8
	at jdk.internal.reflect.GeneratedConstructorAccessor63.newInstance(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
	at org.openqa.selenium.remote.ErrorCodec.decode(ErrorCodec.java:167)
	at org.openqa.selenium.remote.codec.w3c.W3CHttpResponseCodec.decode(W3CHttpResponseCodec.java:138)
	at org.openqa.selenium.remote.codec.w3c.W3CHttpResponseCodec.decode(W3CHttpResponseCodec.java:50)
	at org.openqa.selenium.remote.HttpCommandExecutor.execute(HttpCommandExecutor.java:190)
	at org.openqa.selenium.remote.service.DriverCommandExecutor.invokeExecute(DriverCommandExecutor.java:216)
	at org.openqa.selenium.remote.service.DriverCommandExecutor.execute(DriverCommandExecutor.java:174)
	at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:518)
	at org.openqa.selenium.remote.RemoteWebDriver.get(RemoteWebDriver.java:300)
	at com.taobao.uiauto.service.selenium.action.AbstractAction.pageGoto(AbstractAction.java:523)
	at com.taobao.uiauto.service.selenium.action.system.PageGoto.run(PageGoto.java:70)
	at com.taobao.uiauto.service.selenium.ActionsRunner.run(ActionsRunner.java:54)
	at com.taobao.uiauto.service.service.impl.BrowserImpl.executeCommandAsync(BrowserImpl.java:138)
	at jdk.internal.reflect.GeneratedMethodAccessor49.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
	at org.springframework.aop.interceptor.AsyncExecutionInterceptor.lambda$invoke$0(AsyncExecutionInterceptor.java:115)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)

steelliberty · 2024-05-04T02:39:57Z

Help- I need this not to happen.

MacOS : Monterey M1 version 12.6
selenium==4.20.0

options.headless = True
from selenium import webdriver
from selenium.webdriver.chrome.service import Service as ChromeService
from webdriver_manager.chrome import ChromeDriverManager
driver = webdriver.Chrome(service=ChromeService(ChromeDriverManager().install()))

Error fetching page content: Message: disconnected: not connected to DevTools
(failed to check if window was closed: disconnected: not connected to DevTools)
(Session info: chrome=124.0.6367.119)

Stacktrace:
0 chromedriver 0x00000001024fda18 chromedriver + 5216792
1 chromedriver 0x00000001024f5653 chromedriver + 5183059
2 chromedriver 0x0000000102067527 chromedriver + 406823
3 chromedriver 0x000000010204d59e chromedriver + 300446
4 chromedriver 0x000000010204d493 chromedriver + 300179
5 chromedriver 0x00000001020698b2 chromedriver + 415922
6 chromedriver 0x00000001020f405b chromedriver + 983131
7 chromedriver 0x00000001020d54a3 chromedriver + 857251
8 chromedriver 0x00000001020a5fe3 chromedriver + 663523
9 chromedriver 0x00000001020a692e chromedriver + 665902
10 chromedriver 0x00000001024bb930 chromedriver + 4946224
11 chromedriver 0x00000001024c19e4 chromedriver + 4970980
12 chromedriver 0x000000010249c42e chromedriver + 4817966
13 chromedriver 0x00000001024c24f9 chromedriver + 4973817
14 chromedriver 0x000000010248c6b4 chromedriver + 4753076
15 chromedriver 0x00000001024e4ba8 chromedriver + 5114792
16 chromedriver 0x00000001024e4d67 chromedriver + 5115239
17 chromedriver 0x00000001024f5273 chromedriver + 5182067
18 libsystem_pthread.dylib 0x00007ff80e6814e1 _pthread_start + 125
19 libsystem_pthread.dylib 0x00007ff80e67cf6b thread_start + 15

RileyXX mentioned this issue Jun 24, 2023

Issue with Driver call in v1.5.1 (selenium.common.exceptions.WebDriverException: Message: disconnected: not connected to DevTools) RileyXX/IMDB-Trakt-Syncer#52

Closed

1 task

soulgalore mentioned this issue Jun 29, 2023

ERROR: BrowserError: Could not start chrome with 3 tries sitespeedio/sitespeed.io#3895

Closed

MacOS - Message: disconnected: not connected to DevTools (failed to check if window was closed: disconnected: not connected to DevTools) #357

MacOS - Message: disconnected: not connected to DevTools (failed to check if window was closed: disconnected: not connected to DevTools) #357

Comments

miklyx commented Apr 6, 2023

miklyx commented Apr 7, 2023

codders commented Apr 11, 2023

miklyx commented Apr 11, 2023

codders commented Apr 15, 2023

miklyx commented Apr 16, 2023

silasburger commented May 16, 2023

taraskatsora1998 commented May 23, 2023

codders commented May 24, 2023

cibanez-teladoc commented Jun 5, 2023

arpitpaliwaldh commented Jun 14, 2023

codders commented Jun 14, 2023

pembertonrw commented Jun 15, 2023

arpitpaliwaldh commented Jun 15, 2023

miklyx commented Jun 15, 2023

codders commented Jun 15, 2023

TomerLankri commented Jun 15, 2023

codders commented Jun 15, 2023

codders commented Jun 15, 2023

TomerLankri commented Jun 15, 2023

miklyx commented Jun 15, 2023

hubx commented Jun 25, 2023 • edited Loading

miklyx commented Jun 26, 2023

miklyx commented Jun 26, 2023 • edited Loading

codders commented Jun 27, 2023

codders commented Jun 27, 2023

miklyx commented Jun 27, 2023

miklyx commented Jun 28, 2023

Lamiamrad commented Jul 11, 2023

rjborba commented Jul 28, 2023

Johnny0405 commented Jul 31, 2023

miklyx commented Jul 31, 2023

miklyx commented Aug 1, 2023

ManiRenukuntla commented Aug 10, 2023

filipopo commented Aug 21, 2023

codders commented Aug 22, 2023

A1exMiranda commented Sep 4, 2023

A1exMiranda commented Sep 11, 2023 • edited Loading

anees1221 commented Oct 15, 2023

A1exMiranda commented Oct 24, 2023 • edited Loading

anees1221 commented Nov 1, 2023

anees1221 commented Nov 14, 2023

kothaaridarshi commented Dec 18, 2023 • edited Loading

anees1221 commented Jan 14, 2024

codders commented Jan 17, 2024

A1exMiranda commented Jan 26, 2024 • edited Loading

nienienienie commented Apr 5, 2024

tsbxmw commented Apr 25, 2024

steelliberty commented May 4, 2024 • edited Loading

hubx commented Jun 25, 2023 •

edited

Loading

miklyx commented Jun 26, 2023 •

edited

Loading

A1exMiranda commented Sep 11, 2023 •

edited

Loading

A1exMiranda commented Oct 24, 2023 •

edited

Loading

kothaaridarshi commented Dec 18, 2023 •

edited

Loading

A1exMiranda commented Jan 26, 2024 •

edited

Loading

steelliberty commented May 4, 2024 •

edited

Loading