E621 - automatically grabbing pools in an artist tag?

[kendrakirai]

Anyone know how I might be able to go about this? Only way to see is by going to each image page and seeing if it's part of a pool, and for artists with hundreds or thousands of images that's just not feasible by hand. And with pools not having their own tags aside from what's put in the pools name you can't just search there, either.

[mikf]

Have you tried asking on the e621 forums if there is a recommended way or some sort of API endpoint that supports this?

You could theoretically do this with gallery-dl by going through all of an artist's posts and collecting unique pool IDs, but that's most likely more "expensive" than it needs to be.

[kendrakirai]

Expensive how? In disk space terms? I already have almost a terabyte of stuff there, and if I really need to I can do some deduplication like symlinks and the like.

I'll give that a shot though, thanks.

Edit: Looks like there's a few ways, like hitting up the db exports, but that all quickly went over my head. But this was found.

https://e621-pools.furry.cool/json?search[tags]=

Which outputs a json with all the pools that a certain tag contains. Now, I don't think that gallery-dl can do anything with this out of the box...there's no urls or anything to indicate where any of this is for one thing. But it's a step in the right direction.

[mikf]

To get all pool IDs of all posts in a tag with gallery-dl, you can use --print '\fF {chr(10).join(map(str, pools))}' (needs e2134b3 to interpret \f correctly)

gallery-dl --print-to-file '\fF {chr(10).join(map(str, pools))}' pools.txt --no-download --quiet https://e621.net/posts?tags=kurojojo

This doesn't filter out duplicates, but you can use sort and uniq to do that:

$ gallery-dl --print-to-file '\fF {chr(10).join(map(str, pools))}' pools.txt --no-download --quiet https://e621.net/posts?tags=kurojojo
$ sort pools.txt | uniq

# or

$ gallery-dl --print '\fF {chr(10).join(map(str, pools))}' --no-download --quiet https://e621.net/posts?tags=kurojojo | sort | uniq

[kendrakirai]

Looks like I'm gonna have to wait for the next version to try this, because of the below, but thank you!

gallery-dl --print-to-file '\fF {chr(10).join(map(str, pools))}' pools.txt --no-download --quiet https://e621.net/posts?tags=kurojojo
usage: gallery-dl [OPTION]... URL...
gallery-dl: error: unrecognized arguments: https://e621.net/posts?tags=kurojojo

Trying to use an input file caused further problems

gallery-dl --print-to-file '\fF {chr(10).join(map(str, pools))}' list-of-pools.txt --no-download --quiet -i pools.lst
[gallery-dl][error] Unsupported URL 'pools))}''
[gallery-dl][error] Unsupported URL 'list-of-pools.txt'
[postprocessor][error] 'metadata' initialization failed:  ValueError: expected '}' before end of string
[postprocessor][error] 'metadata' initialization failed:  ValueError: expected '}' before end of string

[mikf]

Oh, seems like you are using cmd.exe, which does not support single quotes '. In this case, you need to replace them with double quotes ".

gallery-dl --print-to-file "\fF {chr(10).join(map(str, pools))}" list-of-pools.txt --no-download --quiet -i pools.lst

---

wait for the next version

for pip installations

python3 -m pip install -U --force-reinstall --no-deps https://github.com/mikf/gallery-dl/archive/master.tar.gz

for standalone executables

gallery-dl --update-to dev

to install the latest code.

[kendrakirai]

Ahaaa, thank you. I'll spin up a dev version then!