Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

selenium_driverless causing multiple errors #4

Open
34j opened this issue Sep 11, 2024 · 2 comments
Open

selenium_driverless causing multiple errors #4

34j opened this issue Sep 11, 2024 · 2 comments

Comments

@34j
Copy link

34j commented Sep 11, 2024 •
edited
Loading

First this happened:

> python /main.py
2024-09-11 18:09:15,798 asyncio DEBUG Using proactor: IocpProactor
2024-09-11 18:09:15,805 aiohttp_chromium.client DEBUG ClientSession: init
2024-09-11 18:09:15,807 aiohttp_chromium.client DEBUG ClientSession: using tempdir AppData\Local\Temp\aiohttp_chromium.ujsh0v6_
2024-09-11 18:09:15,808 aiohttp_chromium.client DEBUG ClientSession: starting chromium
2024-09-11 18:09:15,809 aiohttp_chromium.client DEBUG ClientSession: using chromium user-data-dir AppData\Local\Temp\aiohttp_chromium.ujsh0v6_/chromium-user-data
2024-09-11 18:09:15,825 Ublock DEBUG init done: path: AppData\Local\Temp\aiohttp_chromium.ujsh0v6_/extensions/ublock id: fopomajdelnjnmegngdicnlhnngapano
2024-09-11 18:09:16,302 Ublock DEBUG patch: patching /js/background.js
2024-09-11 18:09:16,303 Ublock DEBUG async_init done
2024-09-11 18:09:16,304 NoDownloadShelf DEBUG init done: path: AppData\Local\Temp\aiohttp_chromium.ujsh0v6_/extensions/no_download_shelf id: eliahhpaikhblhffngfbbgibhmnjplnk
2024-09-11 18:09:16,309 NoDownloadShelf DEBUG async_init done
2024-09-11 18:09:16,310 ClearDownloads DEBUG init done: path: AppData\Local\Temp\aiohttp_chromium.ujsh0v6_/extensions/clear_downloads id: odjhdehjficfhidfnhdgiomhjlfdpddi
2024-09-11 18:09:16,319 ClearDownloads DEBUG async_init done
2024-09-11 18:09:16,319 aiohttp_chromium.client DEBUG writing chromium config file: AppData\Local\Temp\aiohttp_chromium.ujsh0v6_/chromium-user-data/Default/Preferences
2024-09-11 18:09:16,320 aiohttp_chromium.client DEBUG writing chromium state file: AppData\Local\Temp\aiohttp_chromium.ujsh0v6_/chromium-user-data/Local State
2024-09-11 18:09:16,322 aiohttp_chromium.client DEBUG _start_chromium: selenium_webdriver.Chrome
Traceback (most recent call last):
  File "main.py", line 523, in <module>
    asyncio.run(main())
  File "AppData\Local\Programs\Python\Python311\Lib\asyncio\runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "AppData\Local\Programs\Python\Python311\Lib\asyncio\runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "AppData\Local\Programs\Python\Python311\Lib\asyncio\base_events.py", line 653, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "main.py", line 509, in main
    async with YahooShopping(
  File "main.py", line 166, in __aenter__
    await self.session.__aenter__()
  File "venv\Lib\site-packages\aiohttp_chromium\client.py", line 940, in __aenter__
    return await self
           ^^^^^^^^^^
  File "venv\Lib\site-packages\aiohttp_chromium\client.py", line 950, in _async_init
    await self._start_chromium()
  File "venv\Lib\site-packages\aiohttp_chromium\client.py", line 1206, in _start_chromium
    driver = await selenium_webdriver.Chrome(**chromium_args)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "venv\Lib\site-packages\selenium_driverless\webdriver.py", line 114, in __init__
    if not options.binary_location:
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "venv\Lib\site-packages\selenium_driverless\types\options.py", line 285, in binary_location
    self._binary_location = find_chrome_executable()
                            ^^^^^^^^^^^^^^^^^^^^^^^^
  File "venv\Lib\site-packages\selenium_driverless\utils\utils.py", line 67, in find_chrome_executable
    raise FileNotFoundError("Couldn't find installed Chrome or Chromium executable")
FileNotFoundError: Couldn't find installed Chrome or Chromium executable

Therefore I changed the code that calls find_chrome_executable() and specified the path manually. Then:

> python /main.py
2024-09-11 18:04:58,841 asyncio DEBUG Using proactor: IocpProactor
2024-09-11 18:04:58,850 aiohttp_chromium.client DEBUG ClientSession: init
2024-09-11 18:04:58,852 aiohttp_chromium.client DEBUG ClientSession: using tempdir AppData\Local\Temp\aiohttp_chromium.2bugab3j
2024-09-11 18:04:58,854 aiohttp_chromium.client DEBUG ClientSession: starting chromium
2024-09-11 18:04:58,855 aiohttp_chromium.client DEBUG ClientSession: using chromium user-data-dir AppData\Local\Temp\aiohttp_chromium.2bugab3j/chromium-user-data
2024-09-11 18:04:58,875 Ublock DEBUG init done: path: AppData\Local\Temp\aiohttp_chromium.2bugab3j/extensions/ublock id: eobhiineohjdbcoilbgglmjngkiidalo
downloading https://github.com/gorhill/uBlock/releases/download/1.55.0/uBlock0_1.55.0.chromium.zip
200
2024-09-11 18:05:00,498 Ublock DEBUG patch: patching /js/background.js
2024-09-11 18:05:00,499 Ublock DEBUG async_init done
2024-09-11 18:05:00,499 NoDownloadShelf DEBUG init done: path: AppData\Local\Temp\aiohttp_chromium.2bugab3j/extensions/no_download_shelf id: nlomidnifpnncoganambdflkifgbabmi
downloading https://github.com/aoirint/NoDownloadShelf/archive/342bbc883e276cdbcb0dee305fcb7c0bc7cb56b5.zip
200
2024-09-11 18:05:01,182 NoDownloadShelf DEBUG async_init done
2024-09-11 18:05:01,183 ClearDownloads DEBUG init done: path: AppData\Local\Temp\aiohttp_chromium.2bugab3j/extensions/clear_downloads id: fegandgcgdakafpfflbleamnpcnkjlgb
downloading https://github.com/sindresorhus/clear-downloads/archive/35c58beb63256eb6f8fbe55cd4a11ebe51fb044c.zip
200
2024-09-11 18:05:01,910 ClearDownloads DEBUG async_init done
2024-09-11 18:05:01,910 aiohttp_chromium.client DEBUG writing chromium config file: AppData\Local\Temp\aiohttp_chromium.2bugab3j/chromium-user-data/Default/Preferences
2024-09-11 18:05:01,912 aiohttp_chromium.client DEBUG writing chromium state file: AppData\Local\Temp\aiohttp_chromium.2bugab3j/chromium-user-data/Local State
2024-09-11 18:05:01,913 aiohttp_chromium.client DEBUG _start_chromium: selenium_webdriver.Chrome
Traceback (most recent call last):
  File "venv\Lib\site-packages\aiohttp\connector.py", line 1073, in _wrap_create_connection
    sock = await aiohappyeyeballs.start_connection(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "venv\Lib\site-packages\aiohappyeyeballs\impl.py", line 104, in start_connection    
    raise first_exception
  File "venv\Lib\site-packages\aiohappyeyeballs\impl.py", line 81, in start_connection     
    sock = await _connect_sock(
           ^^^^^^^^^^^^^^^^^^^^
  File "venv\Lib\site-packages\aiohappyeyeballs\impl.py", line 166, in _connect_sock       
    await loop.sock_connect(sock, address)
  File "AppData\Local\Programs\Python\Python311\Lib\asyncio\proactor_events.py", line 726, in sock_connect
    return await self._proactor.connect(sock, address)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "AppData\Local\Programs\Python\Python311\Lib\asyncio\windows_events.py", line 846, in _poll
    value = callback(transferred, key, ov)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "AppData\Local\Programs\Python\Python311\Lib\asyncio\windows_events.py", line 633, in finish_connect
    ov.getresult()
ConnectionRefusedError: [WinError 1225] リモート コンピューターによりネットワーク接続が拒否されました。

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "venv\Lib\site-packages\selenium_driverless\types\base_target.py", line 76, in _init
    res = await session.get(url, timeout=10)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "venv\Lib\site-packages\aiohttp\client.py", line 657, in _request
    conn = await self._connector.connect(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "venv\Lib\site-packages\aiohttp\connector.py", line 564, in connect
    proto = await self._create_connection(req, traces, timeout)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "venv\Lib\site-packages\aiohttp\connector.py", line 975, in _create_connection
    _, proto = await self._create_direct_connection(req, traces, timeout)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "venv\Lib\site-packages\aiohttp\connector.py", line 1350, in _create_direct_connection
    raise last_exc
  File "venv\Lib\site-packages\aiohttp\connector.py", line 1319, in _create_direct_connection
    transp, proto = await self._wrap_create_connection(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "venv\Lib\site-packages\aiohttp\connector.py", line 1088, in _wrap_create_connection
    raise client_error(req.connection_key, exc) from exc
aiohttp.client_exceptions.ClientConnectorError: Cannot connect to host 127.0.0.1:54112 ssl:default [リモート コンピューターによりネットワーク接続が拒否されました。]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "main.py", line 523, in <module>
    asyncio.run(main())
  File "AppData\Local\Programs\Python\Python311\Lib\asyncio\runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "AppData\Local\Programs\Python\Python311\Lib\asyncio\runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "AppData\Local\Programs\Python\Python311\Lib\asyncio\base_events.py", line 653, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "main.py", line 509, in main
    async with YahooShopping(
  File "main.py", line 166, in __aenter__
    await self.session.__aenter__()
  File "venv\Lib\site-packages\aiohttp_chromium\client.py", line 940, in __aenter__
    return await self
           ^^^^^^^^^^
  File "venv\Lib\site-packages\aiohttp_chromium\client.py", line 950, in _async_init
    await self._start_chromium()
  File "venv\Lib\site-packages\aiohttp_chromium\client.py", line 1206, in _start_chromium
    driver = await selenium_webdriver.Chrome(**chromium_args)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "venv\Lib\site-packages\selenium_driverless\webdriver.py", line 260, in start_session
    self._base_target = await BaseTarget(host=self._host, is_remote=self._is_remote,
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "venv\Lib\site-packages\selenium_driverless\types\base_target.py", line 81, in _init
    raise asyncio.TimeoutError(
TimeoutError: Couldn't connect to chrome within 30 seconds

This is odd because the following tutorial provided by the selenium_driverless docs worked fine at least until line 11 where TimeoutError is raised.

from selenium_driverless import webdriver
from selenium_driverless.types.by import By
import asyncio


async def main():
    options = webdriver.ChromeOptions()
    async with webdriver.Chrome(options=options) as driver:
        await driver.get('http://nowsecure.nl#relax', wait_load=True)
        await driver.sleep(0.5)
        await driver.wait_for_cdp("Page.domContentEventFired", timeout=15)
        
        # wait 10s for elem to exist
        elem = await driver.find_element(By.XPATH, '/html/body/div[2]/div/main/p[2]/a', timeout=10)
        await elem.click(move_to=True)

        alert = await driver.switch_to.alert
        print(alert.text)
        await alert.accept()

        print(await driver.title)


asyncio.run(main())

OS: Windows 10, Python: 3.11
Thanks in advance
@milahu
Copy link
Owner

milahu commented Sep 11, 2024 •
edited
Loading 

  File "venv\Lib\site-packages\aiohttp_chromium\client.py", line 1206, in _start_chromium
    driver = await selenium_webdriver.Chrome(**chromium_args)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

...

TimeoutError: Couldn't connect to chrome within 30 seconds

seems like some of chromium_args make chrome start too slow

try to raise the timeout to 99999 seconds
try to remove some of chromium_args

see also: upstream issue kaliiiiiiiiii/Selenium-Driverless#214

@34j
Copy link
Author

34j commented Sep 11, 2024 •
edited
Loading

I removed **chromium_args on line 1206 in client.py, but still the same problem persists. I'm sorry, but I'm running out of time, so I give up this.

image

Instantiating Chrome before using this library (and after importing this library) does not cause the error, so it is likely that some global settings change after the instantiation of ClientSession to be the cause.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@milahu @34j