Is there a way to properly skip links that throw 404 errors?

[etui-null]

This is a related question to another discussion post I made as it's happening in the same script/configuration (#4087), but I figured as it involves a different error it warranted its own answer.

When I attempt to download a FANBOX user's posts I get two links that throw 404 errors. These links do indeed not exist and I'm not sure how to either get the downloader to start reading the page properly and remove whatever cached data it's reading from, or just skip them entirely. Manually adding them to my archive file did nothing (I tried to add fanbox2308787_0 and fanbox2186696_0 as I have no idea how many images/files would have been extracted).

https://www.fanbox.cc/@camonome/posts/2308787
https://www.fanbox.cc/@camonome/posts/2186696

[mikf]

Should be fixed with 4fc9675, but I ran gallery-dl https://www.fanbox.cc/@camonome (without any cookies) and didn't get any errors even before this commit.

And no, there is no option to just ignore 404 errors. That has to be individually handled by the extractor code.

[mikf]

It is called "nightly" build in the README, but it is more of a git push build that happens immediately whenever I push new commits, meaning the files from https://github.com/mikf/gallery-dl/actions/runs/5056075959 already include all changes up to the latest commit on master, including 4fc9675.

[etui-null]

So I've updated to the newest version and I'm still seeing the 404 errors like I originally mentioned. I took a look at the code in that commit and I think I know why.
If I'm reading the change correctly, that exception handling block is only for catching errors when an item inside a post does not exist. My issue is that the entire post itself does not exist. The 404 is being thrown earlier in the code, so that warning is never reached.

I know you mentioned you were not able to reproduce this without cookies, but I'm definitely still seeing these ghost posts while using them.

[mikf]

The try-except block catches exceptions when loading data for a single post in _get_post_data().
The only other requests are for paginating over all post IDs and resolving embed URLs.

OK, maybe resolving embeds is what is causing 404 errors. I kind of overlooked this when writing 4fc9675.
(I'm not the one who wrote this extractor, but that's a bad excuse ...)

[mikf]

0281cc7