[Fanbox] Cloudflare challenges #5791

biggestsonicfan · 2024-06-26T13:22:39Z

Looks like Fanbox recently updated to add cloudflare challenges? I'm using my cookies from browser with the same browser type and user agent.

EDIT: Only way I've gotten anything to work is by a Japanese VPN

drunk-moe · 2024-06-26T21:48:04Z

Refresh your cookies and double check your user agent. Just happened to me, got my UA from the requested headers in the network panel and rexported my cookies and it works fine now. A bit annoying but what can you do?

Also - not using any sort of VPN or anything. It worked fine for me once I grabbed my cookies again and used the same User Agent my browser was using.

biggestsonicfan · 2024-06-27T01:40:49Z

Strangely, it was the "browser": "firefox" key that was the culprit. Removing that resolved my issue.

mikf · 2024-06-27T13:24:10Z

Removing that resolved my issue.

browser overrides user-agent when present.

bugreporter2334 · 2024-06-27T22:11:54Z

drunk-moe's method is not quite working for me. spoofing my browser's user agent Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36 and using the same FANBOXSESSID from the cookies still gives me the error
[fanbox][warning] Cloudflare challenge
[fanbox][error] HttpError: '403 Forbidden' for 'https://api.fanbox.cc/post.listCreator?creatorId=[creator]&limit=10'
refreshing the page on my browser does not give a cloudfare challenge.

edit:
gallery-dl version for posterity (windows):

$ gallery-dl --version
1.27.1

relevant portions of my config:

    "extractor":
    {
        "base-directory": "./gallery-dl/",
        "parent-directory": false,
        "postprocessors": null,
        "archive": null,
        "cookies": null,
        "cookies-update": true,
        "proxy": null,
        "skip": true,

        "user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36",
        "retries": 4,
        "timeout": 30.0,
        "verify": true,
        "fallback": true,

        "sleep": 0,
        "sleep-request": 0,
        "sleep-extractor": 0,

        "path-restrict": "auto",
        "path-replace": "_",
        "path-remove": "\\u0000-\\u001f\\u007f",
        "path-strip": "auto",
        "extension-map": {
            "jpeg": "jpg",
            "jpe" : "jpg",
            "jfif": "jpg",
            "jif" : "jpg",
            "jfi" : "jpg"
        },

        "fanbox":
        {
            "cookies": 
            {
              "FANBOXSESSID": "<sessionid>"
            },
            "postprocessors": 
            [{
              "name": "metadata",
              "event": "post",
              "mode": "json",
              "filename": "{id}.json",
              "directory": "posts"
            }]
        },

ZenoArrows · 2024-06-28T20:42:53Z

I've also suddenly started encountering these none of the methods described so far resolve it unfortunately.

biggestsonicfan · 2024-06-28T22:25:05Z

"FANBOXSESSID": "<sessionid>"

You are missing both the cf_clearance and __cf_bm cloudflare cookies.

ZenoArrows · 2024-06-28T22:35:34Z

You are missing both the cf_clearance and __cf_bm cloudflare cookies.

I've tried that also with the user agent set to be the same as my browser, but it still seems to be able to distinguish between my browser and gallery-dl.

biggestsonicfan · 2024-06-29T00:03:01Z

My reply was for @bugreporter2334, I can't see your config file if you don't post it.

bugreporter2334 · 2024-06-29T00:13:04Z

"FANBOXSESSID": "<sessionid>"

You are missing both the cf_clearance and __cf_bm cloudflare cookies.

Yep, works now with these two cookies. Here's the syntax:

"fanbox":
        {
            "cookies": 
            {
              "FANBOXSESSID": "fanbox cookie",
              "cf_clearance": "cf cookie1",
              "__cf_bm": "cf cookie 2"
            },
            "postprocessors": 
            [{
              "name": "metadata",
              "event": "post",
              "mode": "json",
              "filename": "{id}.json",
              "directory": "posts"
            }]
        },

On Chrome you can extract them from the same Application tab that you normally get your Fanbox cookies from.

bossanovaorca · 2024-06-30T06:46:47Z

"fanbox": {
  "cookies": ["firefox", null, null, null, ".fanbox.cc"],
},

I currently retrieve my cookies from my browser. do I need to change this method for it to retrieve the cloudflare cookies properly or am I using it incorrectly?

mikf · 2024-06-30T13:33:25Z

You are using it correctly, but you need to use the user-agent of your browser as well.

"fanbox": {
  "cookies": ["firefox", null, null, null, ".fanbox.cc"],
  "user-agent": "..."
},

Extracting cookies from a browser profile can only get cookies that have been written to disk. Sometimes it takes a while for your browser to update its cookies database file.

tryoutuser · 2024-07-02T16:16:32Z

i've got this error instead :

[2024-07-02 23:09:03][warning] cookies: not enough values to unpack (expected 7, got 6)
[2024-07-02 23:09:03][warning] no 'FANBOXSESSID' cookie set

i'm using gallery-dl latest v1.27.1.
my config for fanbox :

        "fanbox":
        {
            "user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36",
            "cookies": "%APPDATA%\\gallery-dl\\fanbox.txt",
            "cookies-update": false,
            "base-directory": "I:\\Server\\Server 1\\home\\user\\gallery-dl\\kemonoparty\\",
            "verify": false,
            "archive": "%APPDATA%\\gallery-dl\\fanbox.sqlite3",
            "archive-format": "{user['userId']}_{id}_{num:02d}_{filename}",
            "embeds": false,
            "filename": "{id}_{title}_{num:02d}_{filename}.{extension}",
            "directory": ["fanbox", "{user['userId']}"]
        },

inside my cookies text file :

# Netscape HTTP Cookie File
# http://curl.haxx.se/rfc/cookie_spec.html
# This file was generated by EditThisCookie
.fanbox.cc	TRUE	/	TRUE	1719936563	__cf_bm <redacted>
.fanbox.cc        TRUE       /       TRUE        1751445568    cf_clearance <redacted>
.fanbox.cc	TRUE	/	TRUE	1722524401	agreement_master_terms_of_use_202408	1
.fanbox.cc	TRUE	/	TRUE	1722526763	FANBOXSESSID	<redacted>
.fanbox.cc	TRUE	/	TRUE	1725825006	p_ab_d_id	187055155
.fanbox.cc	TRUE	/	TRUE	1725825006	p_ab_id	5
.fanbox.cc	TRUE	/	TRUE	1725825006	p_ab_id_2	3
.fanbox.cc	TRUE	/	TRUE	1751481237	privacy_policy_agreement	7
.fanbox.cc	TRUE	/	TRUE	1754494763	privacy_policy_notification	0
www.fanbox.cc	FALSE	/	TRUE	0	d_prefs	<redacted>

and i'm connected to softether VPN volunteer servers that is not the public-vpn.*number*.ad.ne.jp ones.

mikf · 2024-07-02T17:21:07Z

@tryoutuser
Values in a cookies.txt file must be separated by tab characters and not spaces like it is the case for your cf_clearance line. Replace it with the following and it should load:

.fanbox.cc	TRUE	/	TRUE	1751445568	cf_clearance	<redacted>

tryoutuser · 2024-07-03T05:51:06Z

@mikf i did edit with tab after realizing that and it still says same error those 2 lines above.

updated cookie :

# Netscape HTTP Cookie File
# http://curl.haxx.se/rfc/cookie_spec.html
# This file was generated by EditThisCookie
.fanbox.cc	TRUE	/	TRUE	1719941452	__cf_bm	<redacted>
.fanbox.cc	TRUE	/	TRUE	1751475670	cf_clearance	<redacted>
.fanbox.cc	TRUE	/	TRUE	1722524401	agreement_master_terms_of_use_202408	1
.fanbox.cc	TRUE	/	TRUE	1722531668	FANBOXSESSID	<redacted>
.fanbox.cc	TRUE	/	TRUE	1725825006	p_ab_d_id	187055155
.fanbox.cc	TRUE	/	TRUE	1725825006	p_ab_id	5
.fanbox.cc	TRUE	/	TRUE	1725825006	p_ab_id_2	3
.fanbox.cc	TRUE	/	TRUE	1751481237	privacy_policy_agreement	7
.fanbox.cc	TRUE	/	TRUE	1754499668	privacy_policy_notification	0
www.fanbox.cc	FALSE	/	TRUE	0	d_prefs	<redacted>

Hrxn · 2024-07-04T14:41:57Z

You sure those are real tabs, and not soft tabs, elastic tabs etc. or something else?

Coolchilion · 2024-07-04T23:26:13Z

Where do you get cf_clearance from? I keep clearing my cookies and solving the verification but it never appears in my Get Cookies.txt

komoreshi · 2024-07-09T03:01:46Z

Where do you get cf_clearance from? I keep clearing my cookies and solving the verification but it never appears in my Get Cookies.txt

Same issue on my end, but I suspect it just takes a while for it to be updated. Worst case you just extract it manually using developer tools and add it to your config:

USBhost · 2024-07-11T04:43:24Z

Okay lads I need some help understanding what to do.
./gallery-dl --cookies-from-browser firefox --user-agent browser URL does not work
./gallery-dl --cookies cookies-fanbox-cc.txt --user-agent browser URL does not work.
cookie file gotten from the export cookie addon.
I keep getting 403 Forbidden

edit: ./gallery-dl --cookies-from-browser firefox --user-agent 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:128.0) Gecko/20100101 Firefox/128.0' URL

for the dumb lads like me make sure firefox is your primary or manually get your useragent.

Coolchilion · 2024-07-12T03:51:32Z

bugreporter2334

I tried following your config but I get

Extra data: line 1 column 16 (char 15)
and
no 'FANBOXSESSID' cookie set

TheCrypticOne · 2024-08-23T21:25:29Z

Having this issue as well. Using a cookies.txt file and getting

[fanbox][warning] Cloudflare challenge
[fanbox][error] HttpError: '403 Forbidden' for 'https://api.fanbox.cc/post.info?postId=8400185'

as the output

amackenzie · 2024-09-04T22:23:05Z

Same here, using cookies.txt method that worked for a while:

[fanbox][warning] Cloudflare challenge
[fanbox][error] HttpError: '403 Forbidden' for 'https://api.fanbox.cc/post.paginateCreator?creatorId=<id removed>'

JacobDreiling · 2024-09-11T22:03:31Z

I started getting the same error. Here are my fanbox configs. I'm not sure if I need to specify a user agent or a cookie file, because this used to work before I added them.

"user-agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.5 Safari/605.1.15",
"cookies": {
    "agreement_master_terms_of_use_202408": "1",
    "p_ab_id": "...",
    "p_ab_id_2": "...",
    "p_ab_d_id": "...",
    "FANBOXSESSID": "...",
    "cf_clearance": "...",
    "__cf_bm": "...",
    "privacy_policy_agreement": "...",
    "privacy_policy_notification": "..."
    },
"cookies-update": "test-fanbox-cookies.txt"

mikf · 2024-09-12T07:48:39Z

Visit the URL causing the 403 error in a browser
Let it solve the challenge
Export its cookies (cf_clearance is important)
Run gallery-dl with those cookies and the browser's User-Agent header

$ gallery-dl https://xub.fanbox.cc --cookies cookies-fanbox-cc.txt --user-agent "Mozilla/5.0 (X11; Linux x86_64; rv:128.0) Gecko/20100101 Firefox/128.0"
./fanbox/xub/7288162_0.jpg
./fanbox/xub/6566008_0.jpg
./fanbox/xub/6561218_0.jpg
...

(I can personally access fanbox without any of this)

JacobDreiling · 2024-09-12T23:45:34Z

Please excuse all my comments, but I'm getting an error when trying to export cookies. Here's what I have to automatically get cookies from safari.

"fanbox": {
    "user-agent": "browser",
    "cookies": ["safari"]
}

This gives me an error 1.

[fanbox][warning] cookies: [Errno 1] Operation not permitted: '/Users/quasarbooster/Library/Cookies/Cookies.binarycookies'
[fanbox][warning] no 'FANBOXSESSID' cookie set
[fanbox][warning] Cloudflare challenge

cloudywings2 · 2024-09-19T21:21:50Z

Hello! The process I have used is:

using the Get Cookies.txt LOCALLY, mentioned on the main GitHub page
config file contains the path to the cookies.txt file, and the user agent

When running Gallery-DL as an .exe file I get the "cloudflare challenge" error and "403 Forbidden." My cookies do not have the cf_clearance tag, even if I export all the cookies from the browser. I have exported the cookies from the Fanbox home page as well as an individual artist's page, being sure I am logged in, of course.

Syilumi · 2024-09-27T10:17:02Z

can anyone simply explain how to fix it because i dont know how, i dont even know how iv been using gallery-dl since iv just been using .bat files with the gallery-dl hyperlinks and hoping for the best

mikf added the cloudflare label Jun 27, 2024

matsumo0922 mentioned this issue Jul 1, 2024

ログインできなくなりました matsumo0922/PixiView-KMP#4

Closed

komoreshi mentioned this issue Jul 9, 2024

fanbox HttpError: '403 Forbidden' #5835

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Fanbox] Cloudflare challenges #5791

[Fanbox] Cloudflare challenges #5791

biggestsonicfan commented Jun 26, 2024 •

edited

Loading

drunk-moe commented Jun 26, 2024 •

edited

Loading

biggestsonicfan commented Jun 27, 2024

mikf commented Jun 27, 2024

bugreporter2334 commented Jun 27, 2024 •

edited

Loading

ZenoArrows commented Jun 28, 2024

biggestsonicfan commented Jun 28, 2024

ZenoArrows commented Jun 28, 2024

biggestsonicfan commented Jun 29, 2024

bugreporter2334 commented Jun 29, 2024

bossanovaorca commented Jun 30, 2024

mikf commented Jun 30, 2024 •

edited

Loading

tryoutuser commented Jul 2, 2024 •

edited

Loading

mikf commented Jul 2, 2024

tryoutuser commented Jul 3, 2024

Hrxn commented Jul 4, 2024

Coolchilion commented Jul 4, 2024

komoreshi commented Jul 9, 2024 •

edited

Loading

USBhost commented Jul 11, 2024 •

edited

Loading

Coolchilion commented Jul 12, 2024

TheCrypticOne commented Aug 23, 2024

amackenzie commented Sep 4, 2024

JacobDreiling commented Sep 11, 2024

mikf commented Sep 12, 2024

JacobDreiling commented Sep 12, 2024

cloudywings2 commented Sep 19, 2024

Syilumi commented Sep 27, 2024

[Fanbox] Cloudflare challenges #5791

[Fanbox] Cloudflare challenges #5791

Comments

biggestsonicfan commented Jun 26, 2024 • edited Loading

drunk-moe commented Jun 26, 2024 • edited Loading

biggestsonicfan commented Jun 27, 2024

mikf commented Jun 27, 2024

bugreporter2334 commented Jun 27, 2024 • edited Loading

ZenoArrows commented Jun 28, 2024

biggestsonicfan commented Jun 28, 2024

ZenoArrows commented Jun 28, 2024

biggestsonicfan commented Jun 29, 2024

bugreporter2334 commented Jun 29, 2024

bossanovaorca commented Jun 30, 2024

mikf commented Jun 30, 2024 • edited Loading

tryoutuser commented Jul 2, 2024 • edited Loading

mikf commented Jul 2, 2024

tryoutuser commented Jul 3, 2024

Hrxn commented Jul 4, 2024

Coolchilion commented Jul 4, 2024

komoreshi commented Jul 9, 2024 • edited Loading

USBhost commented Jul 11, 2024 • edited Loading

Coolchilion commented Jul 12, 2024

TheCrypticOne commented Aug 23, 2024

amackenzie commented Sep 4, 2024

JacobDreiling commented Sep 11, 2024

mikf commented Sep 12, 2024

JacobDreiling commented Sep 12, 2024

cloudywings2 commented Sep 19, 2024

Syilumi commented Sep 27, 2024

biggestsonicfan commented Jun 26, 2024 •

edited

Loading

drunk-moe commented Jun 26, 2024 •

edited

Loading

bugreporter2334 commented Jun 27, 2024 •

edited

Loading

mikf commented Jun 30, 2024 •

edited

Loading

tryoutuser commented Jul 2, 2024 •

edited

Loading

komoreshi commented Jul 9, 2024 •

edited

Loading

USBhost commented Jul 11, 2024 •

edited

Loading