(Deviantart) Questions on Filenames, folder stuff, etc #4554
Replies: 4 comments 25 replies
-
You could combine the conditionals "extension in ('jpg','png') and is_mature": "{date}-{title}-{index}-NSFW-{tags}-MediaPhoto.{extension}",
"extension in ('jpg','png')" : "{date}-{title}-{index}-Safe-{tags}-MediaPhoto.{extension}",
"": "etc"or use an f-string "\fF {date}-{title}-{index}-{'NSFW' if is_mature else 'Safe'}-{tags}-MediaPhoto.{extension}",
For a boolean value, you can just check with
You can set different options per subcategory. "directory": ["{author[username]}"],
"gallery": {
"directory": ["{author[username]}", "gallery"]
},
"scraps": {
"directory": ["{author[username]}", "scraps"]
},
That happens automatically. When there's no files to download, it won't create an empty directory.
Sounds like what the
You can use |
Beta Was this translation helpful? Give feedback.
-
|
Okay, working so far on these tips and have had some good success, and some more questions here is my current new config: The Filenaming is working very well, thank you for the tips here. The Folder stuff is giving me some oddities; it is only creating Here's what I got from the Scraps snippet: What's an eclipse interface? Isn't that just the sites dark mode? On posts, Posts with art by other users appears to generate a new Another important bit of information I saw was that some files were being skipped because their names would have been too long (I assume due to tagging). Is there a way to uncap the filename limitation, or if not, a way to trim the amount of tags a filename can grab? (I really don't want to exclude any tags if able) (I haven't tried anything relating to the Abort/Archive commands yet, I'll mess w/ those later) |
Beta Was this translation helpful? Give feedback.
-
Well there is an option for this to enable long paths support in the filesystem on Windows.. But if you do, you could simply run this in Powershell And you should see |
Beta Was this translation helpful? Give feedback.
-
|
Doesn't this only remove the 260 characters limit for total path length? |
Beta Was this translation helpful? Give feedback.
-
|
Not sure actually, but you have a point. Yes, this removes the total path length limitation (well, "remove", no, the limit is [Int16]::MaxValue, afaik), but there is still a per-segment limitation. |
Beta Was this translation helpful? Give feedback.
-
|
Not sure what exactly you're referring to with "this", but there shouldn't be a difference between files and directories regarding name length limits. With |
Beta Was this translation helpful? Give feedback.
-
|
I can't say for sure, because I'm usually hesitant in making absolute proclamations unless I've actually run a test for myself where I understand every step involved. I can only say for sure that this is usually one of the first things I change after a fresh installation, and that Windows, for me, can open files in e.g. its own "Movies & TV" app that fail to open in mpv, for example. But that is not surprising because mpv is not explicitly aware of LongPath support (yet). My assumption (and hence my "this" referring to path names specifically) was based on the distinction between path and file names, that exists in many places, for example the .NET API that has two different methods ( And it seems you're right here, based on this description from the docs
So this is a value or a property of the underlying file system, and
which indeed seems like it is referring to all path name segments, in general, so the path name / file name distinction does not exist here, like you originally suggested. So there is indeed a 255 "character" per segment limitation. Unless you maybe use ReFS instead of NTFS, but I don't know for sure. |
Beta Was this translation helpful? Give feedback.
-
Man, I really was just a smidge too late for the scraps. Bummer. Good to know
I set {tags} to I also edited my other things from "posts": {
"directory": ["{author[username]}", "posts"]
},to "journal": {
"directory": ["{author[username]}", "posts", "journals"]
},
"status": {
"directory": ["{author[username]}", "posts", "statuses"]
},and it works as desired. Ty again for clarifying. Anything I can do about it creating additional
I am using Linux. I apologize for not mentioning
New question that I have now, would it be possible to have different filename extractions depending on what I'm extracting? "favorites": {
"directory": ["{author[username]}", "favorites"]
"filename": {
"[filename stuff]"
}
},(Edit), "favorite": {
"directory": ["{username}", "favorites"]
"filename": {
"[filename stuff]"
}
},I still had My only question now I suppose only relate to the bizarre bit with posts with other artists art |
Beta Was this translation helpful? Give feedback.
-
Scraps should be working again with 8064663 and other recent DA code updates, although I haven't tested this with NSFW scraps yet.
The username from your input URL is available as Regarding subcategory names, you can check the output of |
Beta Was this translation helpful? Give feedback.
-
Hmm, I seem to be getting the same error as before. Do I have to update gallery-dl, or is it automatically updated? (I am using Version 1.26.0-dev).
Neat, I'll keep that in mind. It'll definitely be handy come configuring other site configs as well.
You're right. {username} seems to focus on the user you're extracting from, and {author[username]} on the artist of a thing. Very handy bits. I modified my directory extractions to use username instead for ease, and so far the directories made and data extracted are working. I noticed something possibly minor when extracting journals from my favorites, and that it seems to not inherit the subcategory of journal/status when extracting from favorites, since its filename was that of my "if none of these, then this" name (I only set htm/htlm files to differentiate by subcategory == journal or status, not neither). Probably something on DAs part, but I figure I'd mention it incase it was something that was able to be fixed I am having one issue though that has me confused, and it's in extracting favorites. One link I'm extracting from has 1,727 favorites, and once finished, I downloaded 1,682 of them, meaning 45 aren't being saved. In the error logs, I have 76 lines of error with something along the lines of Weirdly enough, I notice that despite this error, the images still download. Perhaps I'm misinterpreting the error log and maybe something about the original file being non-downloadable so instead it downloads a preview? What weirds me out then, is if this bit isn't erroneous, I still used to get 1,692 saved instead of 1,682. Now getting 1,692 was before setting up any configs, so it's possible that something I did caused 10 pieces to not be able to be downloaded, but the only errors I see when downloading said favorites is about the "deviation not downloadable" stuff, nothing about otherwise. I verified too by renaming I guess the tldr to my question is, does gallery-dl sometimes silently error out on some posts, making it hard to find when a post isn't being downloaded? What might be the cause of the 10 extra pieces to not be downloaded? |
Beta Was this translation helpful? Give feedback.
-
Don't need that part, as by Python conventions that's what Py looks for if not explicitly specified. |
Beta Was this translation helpful? Give feedback.
-
|
You're right... but there's a funny difference.. not sure if this is Windows specific, though. |
Beta Was this translation helpful? Give feedback.
-
|
Glad things are working out so far. |
Beta Was this translation helpful? Give feedback.
-
|
The "Deviation not downloadable." thing was a bug and should have benn fixed in the latest commit (bb39779).
No, the subcategory is defined by input URL, not by media type. |
Beta Was this translation helpful? Give feedback.
-
Tested, no more errors at all in the output log. Very nice.
So if a journal is extracted from a favorites page, it'll inherit the favorite subcategory over journal? |
Beta Was this translation helpful? Give feedback.
-
Hello. I have some questions on how I can configure certain things in my config.json for DeviantArt.
Here is my current config.json:
For filenaming:
How do I get "is_mature" to append the filename with NSFW if true, and Safe if false?
Example intention:
If an extension with png or jpg has an {is_mature} of true, the file would output "{date}-{title}-{index}-
NSFW-{tags}-MediaPhoto.{extension}",If false, "{date}-{title}-{index}-
Safe-{tags}-MediaPhoto.{extension}",Like this, while also working for all of the other extensions? Is it possible to set sub-arguments for this?
Example Idea:
I tried this, but sadly it didn't work. It gave me
[deviantart][error] FilenameFormatError: Applying filename format string failed (TypeError: unhashable type: 'dict')with me wrapping the stuff between "extension in..." in { }, andlist')if I tried [{ }],.Is there a way?
For Folders, I would like to learn how to set it to where if I
include: ""things, they get their own folder. IE/author-username/{gallery,posts,scraps,favorites}(/postsfor Journals and Statuses,/galleryfor the gallery and/or gallery/all, and/scrapsif I want to separate scraps), and if there's no data for a specific folder, to just not create the folder.(Content Edit) Additionally, if a user has multiple types of folders within a gallery (ie all, scraps, animated, etc), to put all of those under a gallery folder instead of all being in the same folder as /favorites, /posts, etc (ie
/authorname/{gallery{all,scraps,animated},favorites,posts})Etc question:
Apparently you are able to "update" what you've downloaded, IE if you downloaded from xyz link, if said link has more data later on and you run it again, it'll only extract the new data. That would be pretty cool, especially come downloading favorites or watchers content. How can I go about doing that?
Beta Was this translation helpful? Give feedback.
All reactions