Seleniumbase gets detected by imperva

[SpielerNogard]

Hey since today imperva seems to detect seleniumbase in uc mode.
the site im crawling is https://join.pokemon.com
and since today a new script gets injected, which removes half os the site. But only when using Brave or when the browser is started using a script (both webbrowser or seleniumbase)

Does someone have any ideas ?
Blocking this script does not work

[mdmintz]

Follow the example from SeleniumBase/examples/raw_antibot_login.py, which demonstrates how to avoid detection on sites using real-time bot-scanning software (such as Brotector, Imperva, and DataDome):

from seleniumbase import SB

with SB(uc=True, test=True) as sb:
    url = "https://seleniumbase.io/antibot/login"
    sb.uc_open_with_disconnect(url, 2.15)
    sb.uc_gui_write("\t" + "demo_user")
    sb.uc_gui_write("\t" + "secret_pass")
    sb.uc_gui_press_keys("\t" + " ")  # For Single-char keys
    sb.sleep(1.5)
    sb.uc_gui_press_keys(["\t", "ENTER"])  # Multi-char keys
    sb.reconnect(1.8)
    sb.assert_text("Welcome!", "h1")
    sb.set_messenger_theme(location="bottom_center")
    sb.post_message("SeleniumBase wasn't detected!")

There, sb uses special PyAutoGUI calls to avoid detection while the driver is disconnected from the browser. Once it's safe to reconnect, do so (as shown), and then you can resume using Selenium calls. This is a bit of a hack right now, as you need to know how many times to press the Tab key before typing text or clicking a button.

Here's another example, SeleniumBase/examples/raw_brotector_captcha.py:

from seleniumbase import SB

with SB(uc=True, test=True) as sb:
    url = "https://seleniumbase.io/apps/brotector"
    sb.uc_open_with_disconnect(url, 2.2)
    sb.uc_gui_press_key("\t")
    sb.uc_gui_press_key(" ")
    sb.reconnect(2.2)

Here's a slightly longer, more fun example: SeleniumBase/examples/raw_hobbit.py:

from seleniumbase import SB

with SB(uc=True, test=True) as sb:
    url = "https://seleniumbase.io/hobbit/login"
    sb.uc_open_with_disconnect(url, 2.2)
    sb.uc_gui_press_keys("\t ")
    sb.reconnect(1.5)
    sb.assert_text("Welcome to Middle Earth!", "h1")
    sb.set_messenger_theme(location="bottom_center")
    sb.post_message("SeleniumBase wasn't detected!")
    sb.click("img")
    sb.sleep(5.888)  # Cool animation happening now!

I'm currently working on a way to use selectors while disconnected so that tabbing through elements isn't needed anymore to avoid detection.

[SpielerNogard]

Thanks, didnt know uc_open_with_disconnect exists. Sadly this also does not work und gets detected.

This is how the site should look, when nothing gets detected

And here is, when it gets detected

Minimal code example:

from seleniumbase import Driver
import time

kwargs = {}
chromium_args = "--lang=en-us"
kwargs["chromium_arg"] = chromium_args
driver = Driver(uc=True, headless2=False, **kwargs)
driver.uc_open_with_disconnect("https://join.pokemon.com/", 2.2)
time.sleep(200)

Im open for any other ideas

[mdmintz]

This worked for me:

from seleniumbase import SB

with SB(uc=True, incognito=True) as sb:
    url = "https://join.pokemon.com/"
    sb.uc_open_with_disconnect(url)

    breakpoint()

[SpielerNogard]

Interresting, for me not

[SpielerNogard]

Ok i have

Google Chrome:
Version 128.0.6613.120 (Official Build) (arm64)

Python:
Python 3.11.5

selenium 4.24.0
seleniumbase 4.30.3

And MacOS 14.6.1 (23G93)

Do you have any more ideas ?

[mdmintz]

You can try https://github.com/ultrafunkamsterdam/nodriver

pip install nodriver

import nodriver

async def main():
    browser = await nodriver.start()
    page = await browser.get("https://join.pokemon.com/")

    breakpoint()

if __name__ == "__main__":
    nodriver.loop().run_until_complete(main())

See if that works for you. It's a completely different repo and API.

[SpielerNogard]

Same issue ...
Hm i dont think its ip bound, since if i use the chrome browser normaly i dont get the bot protection

[mdmintz]

Not sure then. Can't reproduce what you're seeing.

[SpielerNogard]

Yeah, but very big thanks for your help. I will report back, when i have found a solution

[estiven94ops]

hello @SpielerNogard did you find any solution?

[mdmintz]

See SeleniumBase/examples/cdp_mode/raw_pokemon.py for the new example.