Selenium timed out waiting for chart containers to draw at url

[jinlinyouyou]

2024-07-28 23:07:50 [2024-07-28 15:07:50,787: ERROR/ForkPoolWorker-7] Selenium timed out waiting for chart containers to draw at url http://superset:8088/superset/dashboard/1/?standalone=3
2024-07-28 23:07:50 Traceback (most recent call last):
2024-07-28 23:07:50 File "/app/superset/utils/webdriver.py", line 378, in get_screenshot
2024-07-28 23:07:50 WebDriverWait(driver, self._screenshot_locate_wait).until(
2024-07-28 23:07:50 File "/usr/local/lib/python3.10/site-packages/selenium/webdriver/support/wait.py", line 80, in until
2024-07-28 23:07:50 raise TimeoutException(message, screen, stacktrace)
2024-07-28 23:07:50 selenium.common.exceptions.TimeoutException: Message:
2024-07-28 23:07:50
2024-07-28 23:07:52 [2024-07-28 15:07:52,259: INFO/ForkPoolWorker-7] Task cache_dashboard_thumbnail[9d3c2f15-c6b7-43f3-a68d-021c01025b6b] succeeded in 116.81248734199835s: None

[dosubot[bot]]

Hey there, @jinlinyouyou! 👋 I'm here to assist you with any bugs, questions, or contributions while we wait for a human maintainer. Let's squash those bugs together!

To resolve the timeout issue with Selenium while waiting for chart containers to draw, you can increase the wait time for the chart containers to render. This can be done by adjusting the SCREENSHOT_LOCATE_WAIT and SCREENSHOT_LOAD_WAIT configurations in your application.

Here is the relevant code snippet that shows how the waits are configured and used:

class WebDriverSelenium(WebDriverProxy):
    def get_screenshot(self, url: str, element_name: str, user: User) -> bytes | None:
        driver = self.auth(user)
        driver.set_window_size(*self._window)
        driver.get(url)
        img: bytes | None = None
        selenium_headstart = current_app.config["SCREENSHOT_SELENIUM_HEADSTART"]
        logger.debug("Sleeping for %i seconds", selenium_headstart)
        sleep(selenium_headstart)

        try:
            try:
                # page didn't load
                logger.debug(
                    "Wait for the presence of %s at url: %s", element_name, url
                )
                element = WebDriverWait(driver, self._screenshot_locate_wait).until(
                    EC.presence_of_element_located((By.CLASS_NAME, element_name))
                )
            except TimeoutException:
                logger.exception("Selenium timed out requesting url %s", url)
                raise

            try:
                # chart containers didn't render
                logger.debug("Wait for chart containers to draw at url: %s", url)
                WebDriverWait(driver, self._screenshot_locate_wait).until(
                    EC.visibility_of_all_elements_located(
                        (By.CLASS_NAME, "chart-container")
                    )
                )
            except TimeoutException:
                logger.exception(
                    "Selenium timed out waiting for chart containers to draw at url %s",
                    url,
                )
                raise

            try:
                # charts took too long to load
                logger.debug(
                    "Wait for loading element of charts to be gone at url: %s", url
                )
                WebDriverWait(driver, self._screenshot_load_wait).until_not(
                    EC.presence_of_all_elements_located((By.CLASS_NAME, "loading"))
                )
            except TimeoutException:
                logger.exception(
                    "Selenium timed out waiting for charts to load at url %s", url
                )
                raise

            selenium_animation_wait = current_app.config[
                "SCREENSHOT_SELENIUM_ANIMATION_WAIT"
            ]
            logger.debug("Wait %i seconds for chart animation", selenium_animation_wait)
            sleep(selenium_animation_wait)
            logger.debug(
                "Taking a PNG screenshot of url %s as user %s",
                url,
                user.username,
            )

To increase the wait times, you can modify the SCREENSHOT_LOCATE_WAIT and SCREENSHOT_LOAD_WAIT values in your application's configuration. For example:

current_app.config["SCREENSHOT_LOCATE_WAIT"] = 30  # Increase this value as needed
current_app.config["SCREENSHOT_LOAD_WAIT"] = 60    # Increase this value as needed

By increasing these wait times, you give Selenium more time to wait for the chart containers to render and for the charts to load, which should help resolve the timeout issues ^[1].

To continue talking to Dosu, mention @dosu.

[jinlinyouyou]

我已经设置过了，但是我想知道的是为什么会超时，不应该超时呀，因为我在容器上都能访问这个URL

[dosubot[bot]]

Selenium 超时的原因可能有多种，尽管你在容器上可以访问该 URL。以下是一些可能的原因和解决方法：

1. 页面加载时间过长：即使 URL 可以访问，页面上的元素可能需要更长时间才能完全加载。你可以进一步增加 SCREENSHOT_LOCATE_WAIT 和 SCREENSHOT_LOAD_WAIT 的值来给页面更多的加载时间。

2. 网络问题：网络延迟或不稳定可能导致页面加载时间变长，导致 Selenium 超时。

3. 页面结构变化：确保页面上的元素（如 chart-container）的类名没有变化。如果页面结构发生变化，Selenium 可能找不到预期的元素。

4. 资源限制：容器可能资源不足（如 CPU 或内存），导致页面加载变慢。可以尝试增加容器的资源限制。

5. Selenium 配置：检查 Selenium 的其他配置，如 SCREENSHOT_SELENIUM_HEADSTART 和 SCREENSHOT_SELENIUM_ANIMATION_WAIT，确保它们有足够的时间来处理页面加载和动画。

6. 重试机制：Superset 支持重试机制，可以增加 SCREENSHOT_SELENIUM_RETRIES 的值来增加重试次数。

以下是相关代码片段，展示了如何处理这些超时：

try:
    # 等待页面加载
    logger.debug("Wait for the presence of %s at url: %s", element_name, url)
    element = WebDriverWait(driver, self._screenshot_locate_wait).until(
        EC.presence_of_element_located((By.CLASS_NAME, element_name))
    )
except TimeoutException:
    logger.exception("Selenium timed out requesting url %s", url)
    raise

try:
    # 等待图表容器渲染
    logger.debug("Wait for chart containers to draw at url: %s", url)
    WebDriverWait(driver, self._screenshot_locate_wait).until(
        EC.visibility_of_all_elements_located((By.CLASS_NAME, "chart-container"))
    )
except TimeoutException:
    logger.exception("Selenium timed out waiting for chart containers to draw at url %s", url)
    raise

try:
    # 等待图表加载完成
    logger.debug("Wait for loading element of charts to be gone at url: %s", url)
    WebDriverWait(driver, self._screenshot_load_wait).until_not(
        EC.presence_of_all_elements_located((By.CLASS_NAME, "loading"))
    )
except TimeoutException:
    logger.exception("Selenium timed out waiting for charts to load at url %s", url)
    raise

确保你已经根据需要调整了 SCREENSHOT_LOCATE_WAIT 和 SCREENSHOT_LOAD_WAIT 的值 ^[1][2]。

To continue talking to Dosu, mention @dosu.

[Mz-scripter]

@jinlinyouyou You can also use the time module to wait while the chart renders, like a wait time 2-5 seconds
import time
time.sleep(4)

[persistanceTube]

@jinlinyouyou 解决了吗？碰到相同的问题