Proxy issue

[sicholas]

Have tried many proxy providers but haven't been able to get a single one to work with TWScrape. I receive no errors, just the scripts never do anything and sit there.

I have been adding proxies as so (have tried socks5, HTTP, and HTTPS):

async def add_proxy_to_account(api, account, proxy):
    username, password, email, email_password, cookie = account.split(':')
    proxy_url = proxy
    # print(proxy)
    await api.pool.add_account(username, password, email, email_password, cookies=cookie, proxy=proxy_url)
    print(f"Added proxy to account: {username}")

Thanks!

[takabinance]

fwiw, i've been able to use a proxy server without a problem. have you tried to do it globally. something like:

tw_scrape_db = os.path.join(os_path, "accounts.db")
token_pool = AccountsPool(db_file=tw_scrape_db)
tw_scrape_api = API(pool=token_pool, proxy=proxy_url)

[sicholas]

fwiw, i've been able to use a proxy server without a problem. have you tried to do it globally. something like:

tw_scrape_db = os.path.join(os_path, "accounts.db") token_pool = AccountsPool(db_file=tw_scrape_db) tw_scrape_api = API(pool=token_pool, proxy=proxy_url)

Sorry for the delayed response. What provider do you use and what type of proxy are you using (HTTP, HTTPS, SOCKS5)? Thanks again!

[takabinance]

I've used royal and stormproxies. I settled on stormproxies residential as it's cheap, unlimited bandwidth and twitter doesn't seem to ban it so quickly. Http

[sicholas]

@takabinance thank you so much and storm does rotating residential? So every request is randomized or are you just buying one singular static resi?

[takabinance]

I think it rotates every 5 mins.
They have a data center service that rotates on every request but felt like that was getting banned too often.

[linbuxiao]

proxy must be static.

[vladkens]

Continue here please: https://github.com/vladkens/twscrape/discussions/237