ServerError: 503 Server Error: Service Temporarily Unavailable #56

orel35 · 2021-06-02T11:56:10Z

Traceback (most recent call last):
File "C:\perso\tools\docto\doctoshotgun.py", line 506, in
sys.exit(Application().main())
File "C:\perso\tools\docto\doctoshotgun.py", line 433, in main
if not docto.do_login():
File "C:\perso\tools\docto\doctoshotgun.py", line 192, in do_login
self.open('https://www.doctolib.fr/sessions/new')
File "C:\Users\aurel\AppData\Local\Programs\Python\Python39\lib\site-packages\woob\browser\browsers.py", line 898, in open
return super(PagesBrowser, self).open(callback=internal_callback, *args, **kwargs)
File "C:\Users\aurel\AppData\Local\Programs\Python\Python39\lib\site-packages\woob\browser\browsers.py", line 790, in open
return super(DomainBrowser, self).open(req, *args, **kwargs)
File "C:\Users\aurel\AppData\Local\Programs\Python\Python39\lib\site-packages\woob\browser\browsers.py", line 531, in open
response = self.session.send(preq,
File "C:\perso\tools\docto\doctoshotgun.py", line 57, in send
return callback(self, resp)
File "C:\Users\aurel\AppData\Local\Programs\Python\Python39\lib\site-packages\woob\browser\browsers.py", line 527, in inner_callback
self.raise_for_status(response)
File "C:\Users\aurel\AppData\Local\Programs\Python\Python39\lib\site-packages\woob\browser\browsers.py", line 563, in raise_for_status
raise ServerError(http_error_msg, response=response)
woob.browser.exceptions.ServerError: 503 Server Error: Service Temporarily Unavailable

nicolaus-hee · 2021-06-02T16:29:27Z

Do you get this consistently? Usually, trying again will 'resolve' it.

Possibly related to #31 , #21

rbignon · 2021-06-03T06:31:31Z

I can't reproduce.

sgoendoer · 2021-06-04T08:41:34Z

Hi,
I started to get this also. First from time to time, then permanently. My assumption is that the guys at Doctolib figured that someone is using a (this) script and started blocking it. Not sure if this generally detects this script and blocks it or if this is an IP-based blocking.

porlk · 2021-06-07T16:32:22Z

I can confirm to have had the same issue. In my case, the casing of the city name was the cause (eg. you should write paris or grenoble instead of Paris and Grenoble).

gitolicious · 2021-06-12T13:04:29Z

I tried to learn a bit more about this error. For me it's CloudFlare blocking my requests.

    def do_login(self):
        while True:
            retryAfterLoginError = False
            try:
                self.open(self.BASEURL + '/sessions/new')
            except ServerError as e:
                if e.response.status_code in [503]:
                    if 'text/html' in e.response.headers['Content-Type'] and 'cloudflare' in e.response.text:
                        log('Request blocked by CloudFlare', color='red')
                    log('Login failed. Retrying in 10s...')
                    self.session.close()
                    retryAfterLoginError = True
                    sleep(10)
                else:
                    raise
            if retryAfterLoginError == False:
                break

With this change, the login is retried though I didn't have any success even after multiple retries.

$ ./doctoshotgun.py de koln [email protected] password --dry-run
Request blocked by CloudFlare
Login failed. Will retry in 10 sec
Request blocked by CloudFlare
Login failed. Will retry in 10 sec

Stopping and starting the script immediately after helps sometimes though. This is weird behavior.

$ ./doctoshotgun.py de koln [email protected] password --dry-run
Request blocked by CloudFlare
Login failed. Will retry in 10 sec
^CAbort.
$ ./doctoshotgun.py de koln [email protected] password --dry-run
Starting to look for vaccine slots for John Doe between 2021-06-12 and 2021-06-19...
Vaccines: Pfizer, Moderna, Janssen
Country: de
This may take a few minutes/hours, be patient!

rbignon · 2021-06-12T13:32:14Z

That's strange, I never experienced this… Do you use the latest version of cloudscraper, and do you run your script from a home ISP (not from a server)?

nicarl · 2021-06-12T13:37:19Z

I saw the problem only when running from a VPS and after some days (the IP address of the VPS never changes).

rbignon · 2021-06-12T13:42:37Z

Yes, I guess it is harder to bypass cloudflare from a server.

gitolicious · 2021-06-12T20:29:10Z

I was running the script inside a Docker container. When doing so on my laptop (VS Code Dev Container as well as Alpine base image) I got 503s from time to time (~2 out of 10), when doing so on my Odroid N2+ smart home central (Linux OS without a Desktop) it happens most of the time (~9 out of 10). Seems like cloudscaper doesn't play nice with Non-UI environments and that's something doctoshotgun is not to blame for.

gitolicious · 2021-06-17T21:14:15Z

After playing around more with the script, in my Docker environments it is a hit and miss. Once it is running, it works fine all the way - if it starts with a Cloudflare error this seems not to be recoverable with waiting loops. I will add the more detailed error message but not the retries.

lurkaround · 2021-06-23T08:32:59Z

I never get past this error running the script on my local machine. Per #21, I don't guess this has been fixed yet?

gitolicious · 2021-06-23T11:04:48Z

Are you blocked by the Cloudflare check or what response are you getting?

Lvrks · 2021-06-23T11:13:08Z

Whoops, probably should have included this initially. Here's the log:
Traceback (most recent call last): File "C:\Users\tools\doctoshotgun\doctoshotgun.py", line 869, in <module> sys.exit(Application().main()) File "C:\Users\tools\doctoshotgun\doctoshotgun.py", line 698, in main if not docto.do_login(): File "C:\Users\tools\doctoshotgun\doctoshotgun.py", line 216, in do_login self.open(self.BASEURL + "/sessions/new") File "C:\Python39\lib\site-packages\woob\browser\browsers.py", line 898, in open return super(PagesBrowser, self).open(callback=internal_callback, *args, **kwargs) File "C:\Python39\lib\site-packages\woob\browser\browsers.py", line 790, in open return super(DomainBrowser, self).open(req, *args, **kwargs) File "C:\Python39\lib\site-packages\woob\browser\browsers.py", line 531, in open response = self.session.send(preq, File "C:\Users\tools\doctoshotgun\doctoshotgun.py", line 64, in send return callback(self, resp) File "C:\Python39\lib\site-packages\woob\browser\browsers.py", line 527, in inner_callback self.raise_for_status(response) File "C:\Python39\lib\site-packages\woob\browser\browsers.py", line 563, in raise_for_status raise ServerError(http_error_msg, response=response) woob.browser.exceptions.ServerError: 503 Server Error: Service Temporarily Unavailable

gitolicious · 2021-06-23T14:59:27Z

I was able to reproduce this in a Docker environment. This is what I see in the HTML response:

Checking your browser before accessing doctolib.de.
This process is automatic. Your browser will redirect to your requested content shortly.
Please allow up to 5 seconds
Redirecting

So this is a Cloudflare check that blocks the request which is not identified by the changes in #74. (More info on this "Dynamic Analysis" can be found here)

I am unsure if this means cloudscraper has ultimately failed to circumvent the Cloudflare checks, or it will handle it once we raise the error.

gitolicious · 2021-07-05T20:03:58Z

We now have explanatory error messages and nothing else we can do other than retrying or hoping for cloudscraper to improve.

tobiasBora · 2021-07-18T15:01:57Z

I can also confirm this, with the enhanced cloudflare log message PR I get a similar error Request blocked by CloudFlare. This was run locally.

Request blocked by CloudFlare
Traceback (most recent call last):
  File "/tmp/nix-shell.DpTZUZ/tmp.IIan1yyEqe/bin/doctoshotgun.py", line 7, in <module>
    exec(compile(f.read(), __file__, 'exec'))
  File "/home/me/doctoshotgun/doctoshotgun.py", line 863, in <module>
    sys.exit(Application().main())
  File "/home/me/doctoshotgun/doctoshotgun.py", line 686, in main
    if not docto.do_login(args.code):
  File "/home/me/doctoshotgun/doctoshotgun.py", line 259, in do_login
    self.open(self.BASEURL + '/sessions/new')
  File "/nix/store/vjz7gf1v4jicn88v2m6a7knyshzqbv25-python3.8-woob-3.0/lib/python3.8/site-packages/woob/browser/browsers.py", line 898, in open
    return super(PagesBrowser, self).open(callback=internal_callback, *args, **kwargs)
  File "/nix/store/vjz7gf1v4jicn88v2m6a7knyshzqbv25-python3.8-woob-3.0/lib/python3.8/site-packages/woob/browser/browsers.py", line 790, in open
    return super(DomainBrowser, self).open(req, *args, **kwargs)
  File "/nix/store/vjz7gf1v4jicn88v2m6a7knyshzqbv25-python3.8-woob-3.0/lib/python3.8/site-packages/woob/browser/browsers.py", line 531, in open
    response = self.session.send(preq,
  File "/home/me/doctoshotgun/doctoshotgun.py", line 74, in send
    return callback(self, resp)
  File "/nix/store/vjz7gf1v4jicn88v2m6a7knyshzqbv25-python3.8-woob-3.0/lib/python3.8/site-packages/woob/browser/browsers.py", line 527, in inner_callback
    self.raise_for_status(response)
  File "/nix/store/vjz7gf1v4jicn88v2m6a7knyshzqbv25-python3.8-woob-3.0/lib/python3.8/site-packages/woob/browser/browsers.py", line 563, in raise_for_status
    raise ServerError(http_error_msg, response=response)
woob.browser.exceptions.ServerError: 503 Server Error: Service Temporarily Unavailable

gitolicious · 2021-07-18T18:02:48Z

Did it work for you when retrying right away?

flrnt · 2021-11-30T13:41:06Z

same error here, tried using a proxy (germany, spain, argentina, belgium, whatever..) but none worked

Stephan972 · 2021-12-05T19:07:31Z

I get stuck with the very same error. I'm connecting from my home (ie no VPS). Any try is a complete failure (even with large pause between two attempts).

gitolicious mentioned this issue Jun 17, 2021

add Cloudflare-specific error message on login request #74

Merged

gitolicious mentioned this issue Jun 23, 2021

Enhanced Cloudflare log message #89

Merged

jensMF mentioned this issue Dec 1, 2021

CloudFlare blocking request macOS 10.13.3 #299

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ServerError: 503 Server Error: Service Temporarily Unavailable #56

ServerError: 503 Server Error: Service Temporarily Unavailable #56

orel35 commented Jun 2, 2021

nicolaus-hee commented Jun 2, 2021

rbignon commented Jun 3, 2021

sgoendoer commented Jun 4, 2021

porlk commented Jun 7, 2021

gitolicious commented Jun 12, 2021

rbignon commented Jun 12, 2021 •

edited

Loading

nicarl commented Jun 12, 2021

rbignon commented Jun 12, 2021

gitolicious commented Jun 12, 2021 •

edited

Loading

gitolicious commented Jun 17, 2021

lurkaround commented Jun 23, 2021

gitolicious commented Jun 23, 2021

Lvrks commented Jun 23, 2021 •

edited

Loading

gitolicious commented Jun 23, 2021

gitolicious commented Jul 5, 2021

tobiasBora commented Jul 18, 2021 •

edited

Loading

gitolicious commented Jul 18, 2021

flrnt commented Nov 30, 2021

Stephan972 commented Dec 5, 2021

ServerError: 503 Server Error: Service Temporarily Unavailable #56

ServerError: 503 Server Error: Service Temporarily Unavailable #56

Comments

orel35 commented Jun 2, 2021

nicolaus-hee commented Jun 2, 2021

rbignon commented Jun 3, 2021

sgoendoer commented Jun 4, 2021

porlk commented Jun 7, 2021

gitolicious commented Jun 12, 2021

rbignon commented Jun 12, 2021 • edited Loading

nicarl commented Jun 12, 2021

rbignon commented Jun 12, 2021

gitolicious commented Jun 12, 2021 • edited Loading

gitolicious commented Jun 17, 2021

lurkaround commented Jun 23, 2021

gitolicious commented Jun 23, 2021

Lvrks commented Jun 23, 2021 • edited Loading

gitolicious commented Jun 23, 2021

gitolicious commented Jul 5, 2021

tobiasBora commented Jul 18, 2021 • edited Loading

gitolicious commented Jul 18, 2021

flrnt commented Nov 30, 2021

Stephan972 commented Dec 5, 2021

rbignon commented Jun 12, 2021 •

edited

Loading

gitolicious commented Jun 12, 2021 •

edited

Loading

Lvrks commented Jun 23, 2021 •

edited

Loading

tobiasBora commented Jul 18, 2021 •

edited

Loading