Skip to content

Latest commit

 

History

History
2317 lines (2188 loc) · 165 KB

CHANGELOG.md

File metadata and controls

2317 lines (2188 loc) · 165 KB

Changelog

1.23.4 - 2022-10-29

Additions

  • [aibooru] add support for aibooru.online (#3075)
  • [instagram] add 'avatar' extractor (#929, #1097, #2992)
  • [instagram] support 'instagram.com/s/' highlight URLs (#3076)
  • [instagram] extract 'coauthors' metadata (#3107)
  • [mangasee] add support for 'mangalife' (#3086)
  • [mastodon] add 'bookmark' extractor (#3109)
  • [mastodon] support cross-instance user references and '/web/' URLs (#3109)
  • [moebooru] implement 'notes' extraction (#3094)
  • [pixiv] extend 'metadata' option (#3057)
  • [reactor] match 'best', 'new', 'all' URLs (#3073)
  • [smugloli] add 'smugloli' extractors (#3060)
  • [tumblr] add 'fallback-delay' and 'fallback-retries' options (#2957)
  • [vichan] add generic extractors for vichan imageboards

Fixes

  • [bcy] fix extraction (#3103)
  • [gelbooru] support alternate parameter order in post URLs (#2821)
  • [hentai2read] support minor versions in chapter URLs (#3089)
  • [hentaihere] support minor versions in chapter URLs
  • [kemonoparty] fix 'dms' extraction (#3106)
  • [kemonoparty] update pagination offset
  • [manganelo] update domain to 'chapmanganato.com' (#3097)
  • [pixiv] use 'exact_match_for_tags' as default search mode (#3092)
  • [redgifs] fix 'token' extraction (#3080, #3081)
  • [skeb] fix extraction (#3112)
  • improve compatibility of DownloadArchive (#3078)

1.23.3 - 2022-10-15

Additions

  • [2chen] Add 2chen.moe extractor (#2707)
  • [8chan] add thread and board extractors (#2938)
  • [deviantart] add group option (#3018)
  • [fanbox] add content metadata field (#3020)
  • [instagram] restore cursor functionality (#2991)
  • [instagram] restore warnings for private profiles (#3004, #3045)
  • [nana] add nana extractors (#2967)
  • [nijie] add feed and followed extractors (#3048)
  • [tumblr] support https://www.tumblr.com/BLOGNAME URLs (#3034)
  • [tumblr] add offset option
  • [vk] add tagged extractor (#2997)
  • add path-extended option (#3021)
  • emit debug logging messages before calling time.sleep() (#2982)

Changes

  • [postprocessor:metadata] assume "mode": "custom" when format is given

Fixes

  • [artstation] skip missing projects (#3016)
  • [danbooru] fix ugoira metadata extraction (#3056)
  • [deviantart] fix deviation extraction (#2981)
  • [hitomi] fall back to webp when selected format is not available (#3030)
  • [imagefap] fix and improve folder extraction and gallery pagination (#3013)
  • [instagram] fix login (#3011, #3015)
  • [nozomi] fix extraction (#3051)
  • [redgifs] fix extraction (#3037)
  • [tumblr] sleep between fallback retries (#2957)
  • [vk] unescape error messages
  • fix duplicated metadata bug with -j (#3033)
  • fix bug when processing input file comments (#2808)

1.23.2 - 2022-10-01

Additions

  • [artstation] support search filters (#2970)
  • [blogger] add label and query metadata fields (#2930)
  • [exhentai] add a slash to the end of gallery URLs (#2947)
  • [instagram] add count metadata field (#2979)
  • [instagram] add api option
  • [kemonoparty] add count metadata field (#2952)
  • [mastodon] warn about moved accounts (#2939)
  • [newgrounds] add games extractor (#2955)
  • [newgrounds] extract type metadata
  • [pixiv] add series extractor (#2964)
  • [sankaku] implement refresh option (#2958)
  • [skeb] add search extractor and filters option (#2945)

Fixes

  • [deviantart] fix extraction (#2981, #2983)
  • [fappic] fix extraction
  • [instagram] extract higher-resolution photos (#2666)
  • [instagram] fix username and fullname metadata for saved posts (#2911)
  • [instagram] update API headers
  • [kemonoparty] send Referer headers (#2989, #2990)
  • [kemonoparty] restore favorites API endpoints (#2994)
  • [myportfolio] use fallback when no images are found (#2959)
  • [plurk] fix extraction (#2977)
  • [sankaku] detect expired links (#2958)
  • [tumblr] retry extraction of failed higher-resolution images (#2957)

1.23.1 - 2022-09-18

Additions

  • [flickr] add support for secure.flickr.com URLs (#2910)
  • [hotleak] add hotleak extractors (#2890, #2909)
  • [instagram] add highlight_title and date metadata for highlight downloads (#2879)
  • [paheal] add support for videos (#2892)
  • [tumblr] fetch high-quality inline images (#2877)
  • [tumblr] implement ratelimit option (#2919)
  • [twitter] add general support for unified cards (#2875)
  • [twitter] implement cards-blacklist option (#2875)
  • [zerochan] add metadata option (#2861)
  • [postprocessor:zip] implement files option (#2872)

Fixes

  • [bunkr] fix extraction (#2903)
  • [bunkr] use media-files servers for m4v and mov downloads (#2925)
  • [exhentai] improve 509.gif detection (#2901)
  • [exhentai] guess extension for original files (#2842)
  • [poipiku] use img-org.poipiku.com as image domain (#2796)
  • [reddit] prevent exception with empty submission URLs (#2913)
  • [redgifs] fix download URLs (#2884)
  • [smugmug] update default API credentials (#2881)
  • [twitter] provide proper date for syndication results (#2920)
  • [twitter] fix new-style /card_img/ URLs
  • remove all whitespace before comments after input file URLs (#2808)

1.23.0 - 2022-08-28

Changes

  • [twitter] update user and author metdata fields
    • for URLs with a single username or ID like https://twitter.com/USER or a search with a single from: statement, user will now always refer to the user referenced in the URL.
    • for all other URLs like https://twitter.com/i/bookmarks, user and author refer to the same user
    • author will always refer to the original Tweet author
  • [twitter] update quote_id and quote_by metadata fields
    • quote_id is now non-zero for quoted Tweets and contains the Tweet ID of the quotng Tweet (was the other way round before)
    • quote_by is only defined for quoted Tweets like before, but now contains the screen name of the user quoting this Tweet
  • [skeb] improve archive IDs for thumbnails and article images

Additions

  • [artstation] add num and count metadata fields (#2764)
  • [catbox] add album extractor (#2410)
  • [blogger] emit metadata for posts without files (#2789)
  • [foolfuuka] update supported domains
  • [gelbooru] add support for api_key and user_id (#2767)
  • [gelbooru] implement pagination for pool results (#2853)
  • [instagram] add support for a user's saved collections (#2769)
  • [instagram] provide date for directory format strings (#2830)
  • [kemonoparty] add favorites option (#2826, #2831)
  • [oauth] add host config option (#2806)
  • [rule34] implement pagination for pool results (#2853)
  • [skeb] add option to download article images (#1031)
  • [tumblr] download higher-quality images (#2761)
  • [tumblr] add count metadata field (#2804)
  • [wallhaven] implement metadata option (#2803)
  • [zerochan] add tag and image extractors (#1434)
  • [zerochan] implement login with username & password (#1434)
  • [postprocessor:metadata] implement mode: modify and mode: delete (#2640)
  • [formatter] add g conversion for slugifying a string (#2410)
  • [formatter] apply :J only to lists (#2833)
  • implement path-metadata option (#2734)
  • allow comments after input file URLs (#2808)
  • add global warnings option to control urllib3 warning behavior (#2762)

Fixes

  • [bunkr] fix extraction (#2788)
  • [deviantart] use public access token for journals (#2702)
  • [e621] fix extraction of popular posts
  • [fanbox] download cover images in original size (#2784)
  • [mastodon] allow downloading without access token (#2782)
  • [hitomi] update cache expiry time (#2863)
  • [hitomi] fix error when number of tag results is a multiple of 25 (#2870)
  • [mangahere] fix page-reverse option (#2795)
  • [poipiku] fix posts with more than one image (#2796)
  • [poipiku] update filter for static images (#2796)
  • [slideshare] fix metadata extraction
  • [twitter] unescape + in search queries (#2226)
  • [twitter] fall back to unfiltered search (#2766)
  • [twitter] ignore invalid user entries (#2850)
  • [vk] prevent exceptions for broken/invalid photos (#2774)
  • [vsco] fix collection extraction
  • [weibo] prevent exception for missing playback_list (#2792)
  • [weibo] prevent errors when paginating over album entries (#2817)

1.22.4 - 2022-07-15

Additions

  • [instagram] add pinned metadata field (#2752)
  • [itaku] categorize sections by group (#1842)
  • [khinsider] extract platform metadata
  • [tumblr] support /blog/view URLs (#2760)
  • [twitter] implement strategy option (#2712)
  • [twitter] add count metadata field (#2741)
  • [formatter] implement O format specifier (#2736)
  • [postprocessor:mtime] add value option (#2739)
  • add --no-postprocessors command-line option (#2725)
  • implement format-separator option (#2737)

Changes

  • [pinterest] handle section pins with separate extractors (#2684)
  • [postprocessor:ugoira] enable mtime by default (#2714)

Fixes

  • [bunkr] fix extraction (#2732)
  • [hentaifoundry] fix metadata extraction
  • [itaku] fix user caching (#1842)
  • [itaku] fix date parsing
  • [kemonoparty] ensure all files have an extension (#2740)
  • [komikcast] update domain
  • [mangakakalot] update domain
  • [newgrounds] only attempt to login if necessary (#2715)
  • [newgrounds] prevent exception on empty results (#2727)
  • [nozomi] reduce memory consumption during searches (#2754)
  • [pixiv] fix default background filenames
  • [sankaku] rewrite file URLs to s.sankakucomplex.com (#2746)
  • [slideshare] fix description extraction
  • [twitter] ignore previously seen Tweets (#2712)
  • [twitter] unescape HTML entities in content (#2757)
  • [weibo] handle invalid or broken status objects
  • [postprocessor:zip] ensure target directory exists (#2758)
  • make brotli an optional dependency (#2716)
  • limit path length for --write-pages output on Windows (#2733)

Removals

  • [foolfuuka] remove archive.wakarimasen.moe

1.22.3 - 2022-06-28

Changes

  • [twitter] revert strategy changes for user URLs (#2712, #2710)
  • update default User-Agent headers

1.22.2 - 2022-06-27

Additions

  • [cyberdrop] add fallback URLs (#2668)
  • [horne] add support for horne.red (#2700)
  • [itaku] add gallery and image extractors (#1842)
  • [poipiku] add user and post extractors (#1602)
  • [skeb] add following extractor (#2698)
  • [twitter] implement expand option (#2665)
  • [twitter] implement csrf option (#2676)
  • [unsplash] add collection_title and collection_id metadata fields (#2670)
  • [weibo] support tabtype=video listings (#2601)
  • [formatter] implement slice operator as format specifier
  • support cygwin/BSD/etc for --cookies-from-browser

Fixes

  • [instagram] improve metadata generated by _parse_post_api() (#2695, #2660)
  • [instagram} fix tag extractor (#2659)
  • [instagram] automatically invalidate expired login sessions
  • [twitter] fix pagination for conversion tweets
  • [twitter] improve "replies": "self" (#2665)
  • [twitter] improve strategy for user URLs (#2665)
  • [vk] take URLs from *_src entries (#2535)
  • [weibo] fix URLs generated by user extractor (#2601)
  • [weibo] fix retweets (#2601)
  • [downloader:ytdl] update _set_outtmpl() (#2692)
  • [formatter] fix !j conversion for non-serializable types (#2624)
  • [snap] Fix missing libslang dependency (#2655)

1.22.1 - 2022-06-04

Additions

  • [gfycat] add support for collections (#2629)
  • [instagram] support specifying users by ID
  • [paheal] extract more metadata (#2641)
  • [reddit] add home extractor (#2614)
  • [weibo] support usernames in URLs (#1662)
  • [weibo] support livephoto and gif files (#2146)
  • [weibo] add support for several different tabtype listings (#686, #2601)
  • [postprocessor:metadata] write to stdout by setting filename to "-" (#2624)
  • implement output.ansi option (#2628)
  • support user-defined output.mode settings (#2529)

Changes

  • [readcomiconline] remove default browser setting (#2625)
  • [weibo] switch to desktop API (#2601)
  • fix command-line argument name of --cookies-from-browser (#1606, #2630)

Fixes

  • [bunkr] change domain to app.bunkr.is (#2634)
  • [deviantart] fix folder listings with "pagination": "manual" (#2488)
  • [gofile] fix 401 Unauthorized errors (#2632)
  • [hypnohub] move to gelbooru_v02 instances (#2631)
  • [instagram] fix and update extractors (#2644)
  • [nozomi] remove slashes from search terms (#2653)
  • [pixiv] include .gif in background fallback URLs (#2495)
  • [sankaku] extend URL patterns (#2647)
  • [subscribestar] fix date metadata (#2642)

1.22.0 - 2022-05-25

Additions

  • [gelbooru_v01] add favorite extractor (#2546)
  • [Instagram] add tagged_users to keywords for stories (#2582, #2584)
  • [lolisafe] implement domain option (#2575)
  • [naverwebtoon] support (best)challenge comics (#2542)
  • [nijie] support /history_nuita.php listings (#2541)
  • [pixiv] provide more data when metadata is enabled (#2594)
  • [shopify] support several more sites by default (#2089)
  • [twitter] extract alt texts as description (#2617)
  • [twitter] recognize vxtwitter URLs (#2621)
  • [weasyl] implement metadata option (#2610)
  • implement --cookies-from-browser (#1606)
  • implement output.colors options (#2532)
  • implement string literals in replacement fields
  • support using extended format strings for archive keys

Changes

  • [foolfuuka] match 4chan filenames (#2577)
  • [pixiv] implement include option
    • provide avatar/background downloads as separate extractors (#2495)
  • [twitter] use a better strategy for user URLs
  • [twitter] disable cards by default
  • delay directory creation (#2461, #2474)
  • flush writes to stdout/stderr (#2529)
  • build executables on GitHub Actions with Python 3.10

Fixes

  • [artstation] use "browser": "firefox" by default (#2527)
  • [imgur] prevent exception with empty albums (#2557)
  • [instagram] report redirects to captcha challenges (#2543)
  • [khinsider] fix metadata extraction (#2611)
  • [mangafox] send Referer headers (#2592)
  • [mangahere] send Referer headers (#2592)
  • [mangasee] use randomly generated PHPSESSID cookie (#2560)
  • [pixiv] make retrieving ugoira metadata non-fatal (#2562)
  • [readcomiconline] update deobfuscation code (#2481)
  • [realbooru] fix extraction (#2530)
  • [vk] handle photos without width/height info (#2535)
  • [vk] fix user ID extraction (#2535)
  • [webtoons] extract real episode numbers (#2591)
  • create missing directories for archive files (#2597)
  • detect circular references with -K (#2609)
  • replace "\f" in --filename arguments with a form feed character (#2396)

Removals

  • [gelbooru_v01] remove tlb.booru.org from supported domains

1.21.2 - 2022-04-27

Additions

  • [deviantart] implement pagination option (#2488)
  • [pixiv] implement background option (#623, #1124, #2495)
  • [postprocessor:ugoira] report ffmpeg/mkvmerge errors (#2487)

Fixes

  • [cyberdrop] match cyberdrop.to URLs (#2496)
  • [e621] fix 403 errors (#2533)
  • [issuu] fix extraction (#2483)
  • [mangadex] download from available chapters despite externalUrl (#2503)
  • [photovogue] update domain and api endpoint (#2494)
  • [sexcom] add fallback for empty files (#2485)
  • [twitter] improve syndication video selection (#2354)
  • [twitter] fix various syndication issues (#2499, #2354)
  • [vk] fix extraction (#2512)
  • [weibo] fix infinite retries for deleted accounts (#2521)
  • [postprocessor:ugoira] use compatible paths with mkvmerge (#2487)
  • [postprocessor:ugoira] do not auto-select the image2 demuxer (#2492)

1.21.1 - 2022-04-08

Additions

  • [gofile] add gofile.io extractor (#2364)
  • [instagram] add previews option (#2135)
  • [kemonoparty] add duplicates option (#2440)
  • [pinterest] add extractor for created pins (#2452)
  • [pinterest] support multiple files per pin (#1619, #2452)
  • [telegraph] Add telegra.ph extractor (#2312)
  • [twitter] add syndication option (#2354)
  • [twitter] accept fxtwitter.com URLs (#2484)
  • [downloader:http] support using an arbitrary method and sending POST data (#2433)
  • [postprocessor:metadata] implement archive options (#2421)
  • [postprocessor:ugoira] add mtime option (#2307)
  • [postprocessor:ugoira] support setting timecodes with mkvmerge (#1550)
  • [formatter] support evaluating f-string literals
  • add --ugoira-conv-copy command-line option (#1550)
  • implement a contains() function for filter statements (#2446)

Fixes

  • [aryion] provide correct date metadata independent of DST
  • [furaffinity] fix search result pagination (#2402)
  • [hitomi] update and fix metadata extraction (#2444)
  • [kissgoddess] extract all images (#2473)
  • [mangasee] unescape manga names (#2454)
  • [newgrounds] update and fix pagination (#2456)
  • [newgrounds] warn about age-restricted posts (#2456)
  • [pinterest] do not force m3u8_native for video downloads (#2436)
  • [twibooru] fix posts without name (#2434)
  • [unsplash] replace dash with space in search API queries (#2429)
  • [postprocessor:mtime] fix timestamps from datetime objects (#2307)
  • fix yet another bug in _check_cookies() (#2372)
  • fix loading/storing cookies without domain

1.21.0 - 2022-03-14

Additions

  • [fantia] add num enumeration index (#2377)
  • [fantia] support "Blog Post" content (#2381)
  • [imagebam] add support for /view/ paths (#2378)
  • [kemonoparty] match beta.kemono.party URLs (#2348)
  • [kissgoddess] add gallery and model extractors (#1052, #2304)
  • [mememuseum] add tag and post extractors (#2264)
  • [newgrounds] add post_url metadata field (#2328)
  • [patreon] add image_large file type (#2257)
  • [toyhouse] support art listings (#1546, #2331)
  • [twibooru] add extractors for searches, galleries, and posts (#2219)
  • [postprocessor:metadata] implement mtime option (#2307)
  • [postprocessor:mtime] add event option (#2307)
  • add fish shell completion (#2363)
  • add timedelta class to global namespace in filter expressions

Changes

  • [seiga] require authentication with user_session cookie (#2372)
    • remove username & password login due to 2FA
  • refactor proxy support (#2357)
    • allow gallery-dl proxy settings to overwrite environment proxies
    • allow specifying different proxies for data extraction and download

Fixes

  • [bunkr] fix mp4 downloads (#2239)
  • [fanbox] fetch data for each individual post (#2388)
  • [hentaicosplays] send Referer header (#2317)
  • [imagebam] set nsfw_inter cookie (#2334)
  • [kemonoparty] limit default filename length (#2373)
  • [mangadex] fix chapters without translatedLanguage (#2352)
  • [newgrounds] fix video descriptions (#2328)
  • [skeb] add sent-requests option (#2322, #2330)
  • [slideshare] fix extraction
  • [subscribestar] unescape attachment URLs (#2370)
  • [twitter] fix handling of 429 Too Many Requests responses (#2339)
  • [twitter] warn about age-restricted Tweets (#2354)
  • [twitter] handle Tweets with "softIntervention" entries
  • [twitter] update query hashes
  • fix another bug in _check_cookies() (#2160)

1.20.5 - 2022-02-14

Additions

  • [furaffinity] add layout option (#2277)
  • [lightroom] add Lightroom gallery extractor (#2263)
  • [reddit] support standalone submissions on personal user pages (#2301)
  • [redgifs] support i.redgifs.com URLs (#2300)
  • [wallpapercave] add extractor for images and search results (#2205)
  • add signals-ignore option (#2296)

Changes

  • [danbooru] merge danbooru and e621 extractors
    • support atfbooru (#2283)
    • remove support for old e621 tag search URLs

Fixes

  • [furaffinity] improve new/old layout detection (#2277)
  • [imgbox] fix ImgboxExtractor (#2281)
  • [inkbunny] rename search parameters to their API equivalents
  • [kemonoparty] handle files without names (#2276)
  • [twitter] fix extraction (#2275, #2295)
  • [vk] fix infinite pagination loops (#2297)
  • [downloader:ytdl] make ImportErrors non-fatal (#2273)

1.20.4 - 2022-02-06

Additions

  • [e621] add favorite extractor (#2250)
  • [hitomi] add format option (#2260)
  • [kohlchan] add Kohlchan extractors (#2251)
  • [sexcom] add pins extractor (#2265)
  • [twitter] add warnings option (#2258)
  • add ability to disable TLS 1.2 (#2243)
  • add examples for custom gelbooru instances (#2262)

Fixes

  • [bunkr] fix mp4 downloads (#2239)
  • [gelbooru] improve and fix pagination (#2230, #2232)
  • [hitomi] "fix" 403 errors (#2260)
  • [kemonoparty] fix downloading smaller text files (#2267)
  • [patreon] disable TLS 1.2 by default (#2249)
  • [twitter] restore errors for protected timelines etc (#2237)
  • [twitter] restore logout functionality (#1719)
  • [twitter] provide fallback URLs for card images
  • [weibo] update pagination code (#2244)

1.20.3 - 2022-01-26

Fixes

  • [kemonoparty] fix DMs extraction (#2008)
  • [twitter] fix crash on Tweets with deleted quotes (#2225)
  • [twitter] fix crash on suspended Tweets without legacy entry (#2216)
  • [twitter] fix crash on unified cards without type
  • [twitter] prevent crash on invalid/deleted Retweets (#2225)
  • [twitter] update query hashes

1.20.2 - 2022-01-24

Additions

  • [twitter] add event extractor (closes #2109)
  • [twitter] support image_carousel_website unified cards
  • add --source-address command-line option (#2206)
  • add environment variable syntax to formatting.md (#2065)

Changes

  • [twitter] changes to cards option
    • enable cards by default
    • require cards to be set to "ytdl" to invoke youtube-dl/yt-dlp on unsupported cards

Fixes

  • [blogger] support new image domain (#2204)
  • [gelbooru] improve video file detection (#2188)
  • [hitomi] fix tag extraction (#2189)
  • [instagram] fix highlights extraction (#2197)
  • [mangadex] re-enable warning for external chapters (#2193)
  • [newgrounds] set suitabilities filter before starting a search (#2173)
  • [philomena] fix search parameter escaping (#2215)
  • [reddit] allow downloading from quarantined subreddits (#2180)
  • [sexcom] extend URL pattern (#2220)
  • [twitter] update to GraphQL API (#2212)

1.20.1 - 2022-01-08

Additions

  • [newgrounds] add search extractor (#2161)

Changes

  • restore -d/--dest functionality from before 1.20.0 (#2148)
  • change short option for --directory to -D

Fixes

  • [gelbooru] handle changed API response format (#2157)
  • [hitomi] fix image URLs (#2153)
  • [mangadex] fix extraction (#2177)
  • [rule34] use https://api.rule34.xxx for API requests
  • fix cookie checks for patreon, fanbox, fantia
  • improve UNC path handling (#2126)

1.20.0 - 2021-12-29

Additions

  • [500px] add favorite extractor (#1927)
  • [exhentai] add source option
  • [fanbox] support pixiv redirects (#2122)
  • [inkbunny] add search extractor (#2094)
  • [kemonoparty] support coomer.party (#2100)
  • [lolisafe] add generic album extractor for lolisafe/chibisafe instances (#2038, #2105)
  • [rule34us] add tag and post extractors (#1527)
  • add a generic extractor (#735, #683)
  • add -d/--directory and -f/--filename command-line options
  • add --sleep-request and --sleep-extractor command-line options
  • allow specifying sleep-* options as string

Changes

  • [cyberdrop] include file ID in default filenames
  • [hitomi] disable metadata by default
  • [kemonoparty] use service as subcategory (#2147)
  • [kemonoparty] change default files order to attachments,file,inline (#1991)
  • [output] write download progress indicator to stderr
  • [ytdl] prefer yt-dlp over youtube-dl (#1850, #2028)
  • rename --write-infojson to --write-info-json

Fixes

  • [500px] create directories per photo
  • [artstation] create directories per asset (#2136)
  • [deviantart] use /browse/newest for most-recent searches (#2096)
  • [hitomi] fix image URLs
  • [instagram] fix error when PostPage data is not in GraphQL format (#2037)
  • [instagran] match post URLs with usernames (#2085)
  • [instagram] allow downloading specific stories (#2088)
  • [furaffinity] warn when no session cookies were found
  • [pixiv] respect date ranges in search URLs (#2133)
  • [sexcom] fix and improve embed extraction (#2145)
  • [tumblrgallery] fix extraction (#2112)
  • [tumblrgallery] improve id extraction (#2115)
  • [tumblrgallery] improve search pagination (#2132)
  • [twitter] include 4096x4096 as a default image fallback (#1881, #2107)
  • [ytdl] update argument parsing to latest yt-dlp changes (#2124)
  • handle UNC paths (#2113)

1.19.3 - 2021-11-27

Additions

  • [dynastyscans] add manga extractor (#2035)
  • [instagram] include user metadata for tagged downloads (#2024)
  • [kemonoparty] implement files option (#1991)
  • [kemonoparty] add dms option (#2008)
  • [mangadex] always provide artist, author, and group metadata fields (#2049)
  • [philomena] support furbooru.org (#1995)
  • [reactor] support thatpervert.com (#2029)
  • [shopify] support loungeunderwear.com (#2053)
  • [skeb] add thumbnails option (#2047, #2051)
  • [subscribestar] add num enumeration index (#2040)
  • [subscribestar] emit metadata for posts without media (#1569)
  • [ytdl] implement cmdline-args and config-file options to allow parsing ytdl command-line options (#1680)
  • [formatter] implement D format specifier
  • extend blacklist/whitelist syntax (#2025)

Fixes

  • [dynastyscans] provide date as datetime object (#2050)
  • [exhentai] fix extraction for disowned galleries (#2055)
  • [gelbooru] apply workaround for pagination limits
  • [kemonoparty] skip duplicate files (#2032, #1991, #1899)
  • [kemonoparty] provide date metadata for gumroad (#2007)
  • [mangoxo] fix metadata extraction
  • [twitter] distinguish between fatal & nonfatal errors (#2020)
  • [twitter] fix extractor for direct image links (#2030)
  • [webtoons] use download URLs that do not require a Referer header (#2005)
  • [ytdl] improve error handling (#1680)
  • [downloader:ytdl] prevent crash in _progress_hook() (#1680)

Removals

  • [seisoparty] remove module

1.19.2 - 2021-11-05

Additions

  • [kemonoparty] add comments option (#1980)
  • [skeb] add user and post extractors (#1031, #1971)
  • [twitter] add pinned option
  • support accessing environment variables and the current local datetime in format strings (#1968)
  • add special type format strings to docs (#1987)

Fixes

  • [cyberdrop] fix video extraction (#1993)
  • [deviantart] fix index values for stashed deviations
  • [gfycat] provide consistent userName values for user downloads (#1962)
  • [gfycat] show warning when there are no available formats
  • [hitomi] fix image URLs (#1975, #1982, #1988)
  • [instagram] update query hashes
  • [mangakakalot] update domain and fix extraction
  • [mangoxo] fix login and extraction
  • [reddit] prevent crash for galleries with no media_metadata (#2001)
  • [redgifs] update to API v2 (#1984)
  • fix calculating retry sleep times (#1990)

1.19.1 - 2021-10-24

Additions

  • [inkbunny] add following extractor (#515)
  • [inkbunny] add pool extractor (#1937)
  • [kemonoparty] add discord extractor (#1827, #1940)
  • [nhentai] add tag extractor (#1950, #1955)
  • [patreon] add files option (#1935)
  • [picarto] add gallery extractor (#1931)
  • [pixiv] add sketch extractor (#1497)
  • [seisoparty] add favorite extractor (#1906)
  • [twitter] add size option (#1881)
  • [vk] add album extractor (#474, #1952)
  • [postprocessor:compare] add equal option (#1592)

Fixes

  • [cyberdrop] extract direct download URLs (#1943)
  • [deviantart] update search argument handling (#1911)
  • [deviantart] full resolution for non-downloadable images (#293)
  • [furaffinity] unquote search queries (#1958)
  • [inkbunny] match "long" URLs for pools and favorites (#1937)
  • [kemonoparty] improve inline extraction (#1899)
  • [mangadex] update parameter handling for API requests (#1908)
  • [patreon] better filenames for content images (#1954)
  • [redgifs][gfycat] provide fallback URLs (#1962)
  • [downloader:ytdl] prevent crash in _progress_hook()
  • restore SOCKS support for Windows executables

1.19.0 - 2021-10-01

Additions

  • [aryion] add tag extractor (#1849)
  • [desktopography] implement desktopography extractors (#1740)
  • [deviantart] implement auto-unwatch option (#1466, #1757)
  • [fantia] add date metadata field (#1853)
  • [fappic] add image extractor (#1898)
  • [gelbooru_v02] add favorite extractor (#1834)
  • [kemonoparty] add favorite extractor (#1824)
  • [kemonoparty] implement login with username & password (#1824)
  • [mastodon] add following extractor (#1891)
  • [mastodon] support specifying accounts by ID
  • [twitter] support /with_replies URLs (#1833)
  • [twitter] add quote_by metadata field (#1481)
  • [postprocessor:compare] extend action option (#1592)
  • implement a download progress indicator (#1519)
  • implement a page-reverse option (#1854)
  • implement a way to specify extended format strings
  • allow specifying a minimum/maximum for sleep-* options (#1835)
  • add a --write-infojson command-line option

Changes

  • [cyberdrop] change directory name format (#1871)
  • [instagram] update default delay to 6-12 seconds (#1835)
  • [reddit] extend subcategory depending on input URL (#1836)
  • move util.Formatter and util.PathFormat into their own modules

Fixes

  • [artstation] use /album/all view for user portfolios (#1826)
  • [aryion] update/improve pagination (#1849)
  • [deviantart] fix bug with fetching premium content (#1879)
  • [deviantart] update default archive_fmt for single deviations (#1874)
  • [erome] send Referer header for file downloads (#1829)
  • [hiperdex] fix extraction
  • [kemonoparty] update file download URLs (#1902, #1903)
  • [mangadex] fix extraction (#1852)
  • [mangadex] fix retrieving chapters from "pornographic" titles (#1908)
  • [nozomi] preserve case of search tags (#1860)
  • [redgifs][gfycat] remove webtoken code (#1907)
  • [twitter] ensure card entries have a url (#1868)
  • implement a way to correctly shorten displayed filenames containing east-asian characters (#1377)

1.18.4 - 2021-09-04

Additions

  • [420chan] add thread and board extractors (#1773)
  • [deviantart] add tag extractor (#1803)
  • [deviantart] add comments option (#1800)
  • [deviantart] implement a auto-watch option (#1466, #1757)
  • [foolfuuka] add gallery extractor (#1785)
  • [furaffinity] expand URL pattern for searches (#1780)
  • [kemonoparty] automatically generate required DDoS-GUARD cookies (#1779)
  • [nhentai] add favorite extractor (#1814)
  • [shopify] support windsorstore.com (#1793)
  • [twitter] add url to user objects (#1787, #1532)
  • [twitter] expand t.co links in user descriptions (#1787, #1532)
  • show a warning if an extractor doesn`t yield any results (#1428, #1759)
  • add a j format string conversion
  • implement a fallback option (#1770)
  • implement a path-strip option

Changes

  • [shopify] use API for product listings (#1793)
  • update default User-Agent headers

Fixes

  • [deviantart] prevent exceptions for "empty" videos (#1796)
  • [exhentai] improve image limits check (#1808)
  • [inkbunny] fix extraction (#1816)
  • [mangadex] prevent exceptions for manga without English title (#1815)
  • [oauth] use defaults when config values are set to null (#1778)
  • [pixiv] fix pixivision title extraction
  • [reddit] delay RedditAPI initialization (#1813)
  • [twitter] improve error reporting (#1759)
  • [twitter] fix issue when filtering quote tweets (#1792)
  • [twitter] fix logout option (#1719)

Removals

  • [deviantart] remove the "you need session cookies to download mature scraps" warning (#1777, #1776)
  • [foolslide] remove entry for kobato.hologfx.com

1.18.3 - 2021-08-13

Additions

  • [bbc] add width option (#1706)
  • [danbooru] add external option (#1747)
  • [furaffinity] add external option (#1492)
  • [luscious] add gif option (#1701)
  • [newgrounds] add format option (#1729)
  • [reactor] add gif option (#1701)
  • [twitter] warn about suspended accounts (#1759)
  • [twitter] extend replies option (#1254)
  • [twitter] add option to log out and retry when blocked (#1719)
  • [wikieat] add thread and board extractors (#1699, #1607)

Changes

  • [instagram] increase default delay between HTTP requests from 5s to 8s (#1732)

Fixes

  • [bbc] improve image dimensions (#1706)
  • [bbc] support multi-page gallery listings (#1730)
  • [behance] fix collection extraction
  • [deviantart] get original files for GIF previews (#1731)
  • [furaffinity] fix errors when using category-transfer (#1274)
  • [hitomi] fix image URLs (#1765)
  • [instagram] use custom User-Agent header for video downloads (#1682, #1623, #1580)
  • [kemonoparty] fix username extraction (#1750)
  • [kemonoparty] update file server domain (#1764)
  • [newgrounds] fix errors when using category-transfer (#1274)
  • [nsfwalbum] retry backend requests when extracting image URLs (#1733, #1271)
  • [vk] prevent exception for empty/private profiles (#1742)

1.18.2 - 2021-07-23

Additions

  • [bbc] add gallery and programme extractors (#1706)
  • [comicvine] add extractor (#1712)
  • [kemonoparty] add max-posts option (#1674)
  • [kemonoparty] parse o query parameters (#1674)
  • [mastodon] add reblogs and replies options (#1669)
  • [pixiv] add extractor for pixivision articles (#1672)
  • [ytdl] add experimental extractor for sites supported by youtube-dl (#1680, #878)
  • extend parent-metadata functionality (#1687, #1651, #1364)
  • add archive-prefix option (#1711)
  • add url-metadata option (#1659, #1073)

Changes

  • [kemonoparty] skip duplicated patreon files (#1689, #1667)
  • [mangadex] use custom User-Agent header (#1535)

Fixes

  • [hitomi] fix image URLs (#1679)
  • [imagevenue] fix extraction (#1677)
  • [instagram] fix extraction of /explore/tags/ posts (#1666)
  • [moebooru] fix tags ending with a + when logged in (#1702)
  • [naverwebtoon] fix comic extraction
  • [pururin] update domain and fix extraction
  • [vk] improve metadata extraction and URL pattern (#1691)
  • [downloader:ytdl] fix outtmpl setting for yt-dlp (#1680)

1.18.1 - 2021-07-04

Additions

  • [mangafox] add manga extractor (#1633)
  • [mangasee] add chapter and manga extractors
  • [mastodon] implement text-posts option (#1569, #1669)
  • [seisoparty] add user and post extractors (#1635)
  • implement conditional directories (#1394)
  • add T format string conversion (#1646)
  • document format string syntax

Changes

  • [twitter] set retweet_id for original retweets (#1481)

Fixes

  • [directlink] manually encode Referer URLs (#1647)
  • [hiperdex] use domain from input URL
  • [kemonoparty] fix username extraction (#1652)
  • [kemonoparty] warn about missing DDoS-GUARD cookies
  • [twitter] ensure guest tokens are returned as string (#1665)
  • [webtoons] match arbitrary language codes (#1643)
  • fix depth counter in UrlJob when specifying -g multiple times

1.18.0 - 2021-06-19

Additions

  • [foolfuuka] support archive.wakarimasen.moe (#1595)
  • [mangadex] implement login with username & password (#1535)
  • [mangadex] add extractor for a user's followed feed (#1535)
  • [pixiv] support fetching privately followed users (#1628)
  • implement conditional filenames (#1394)
  • implement filter option for post processors (#1460)
  • add -T/--terminate command-line option (#1399)
  • add -P/--postprocessor command-line option (#1583)

Changes

  • [kemonoparty] update default filenames and archive IDs (#1514)
  • [twitter] update default settings
    • change retweets and quoted options from true to false
    • change directory format for search results to the same as other extractors
  • require an argument for --clear-cache

Fixes

  • [500px] update GraphQL queries
  • [furaffinity] improve metadata extraction (#1630)
  • [hitomi] update image URL generation (#1637)
  • [idolcomplex] improve and fix pagination (#1594, #1601)
  • [instagram] fix login (#1631)
  • [instagram] update query hashes
  • [mangadex] update to API v5 (#1535)
  • [mangafox] improve URL pattern (#1608)
  • [oauth] prevent exceptions when reporting errors (#1603)
  • [philomena] fix tag escapes handling (#1629)
  • [redgifs] update API server address (#1632)
  • [sankaku] handle empty tags (#1617)
  • [subscribestar] improve attachment filenames (#1609)
  • [unsplash] update collections URL pattern (#1627)
  • [postprocessor:metadata] handle dicts in mode:tags (#1598)

1.17.5 - 2021-05-30

Additions

  • [kemonoparty] add metadata option (#1548)
  • [kemonoparty] add type metadata field (#1556)
  • [mangapark] recognize v2.mangapark URLs (#1578)
  • [patreon] extract user-defined tags (#1539, #1540)
  • [pillowfort] implement login with username & password (#846)
  • [pillowfort] add inline and external options (#846)
  • [pixiv] implement max-posts option (#1558)
  • [pixiv] add metadata option (#1551)
  • [twitter] add text-tweets option (#570)
  • [weibo] extend retweets option (#1542)
  • [postprocessor:ugoira] support using the image2 demuxer (#1550)
  • [postprocessor:ugoira] add repeat-last-frame option (#1550)
  • support XDG_CONFIG_HOME (#1545)
  • implement parent-skip and "skip": "terminate" options (#1399)

Changes

  • [twitter] resolve t.co URLs in content (#1532)

Fixes

  • [500px] update query hashes (#1573)
  • [aryion] find text posts in recursive=false mode (#1568)
  • [imagebam] fix extraction of NSFW images (#1534)
  • [imgur] update URL patterns (#1561)
  • [manganelo] update domain to manganato.com
  • [reactor] skip deleted/empty posts
  • [twitter] add missing retweet media entities (#1555)
  • fix ISO 639-1 code for Japanese (jp -> ja)

1.17.4 - 2021-05-07

Additions

  • [gelbooru] add extractor for /redirect.php URLs (#1530)
  • [inkbunny] add favorite extractor (#1521)
  • add output.skip option
  • add an optional argument to --clear-cache to select which cache entries to remove (#1230)

Changes

  • [pixiv] update translated-tags option (#1507)
    • rename to tags
    • accept "japanese", "translated", and "original" as values

Fixes

  • [500px] update query hashes
  • [kemonoparty] fix download URLs (#1514)
  • [imagebam] fix extraction
  • [instagram] update query hashes
  • [nozomi] update default archive-fmt for tag and search extractors (#1529)
  • [pixiv] remove duplicate translated tags (#1507)
  • [readcomiconline] change domain to readcomiconline.li (#1517)
  • [sankaku] update invalid-token detection (#1515)
  • fix crash when using --no-download with --ugoira-conv (#1507)

1.17.3 - 2021-04-25

Additions

  • [danbooru] add option for extended metadata extraction (#1458)
  • [fanbox] add extractors (#1459)
  • [fantia] add extractors (#1459)
  • [gelbooru] add an option to extract notes (#1457)
  • [hentaicosplays] add extractor (#907, #1473, #1483)
  • [instagram] add extractor for tagged posts (#1439)
  • [naverwebtoon] ignore non-comic images
  • [pixiv] also save untranslated tags when translated-tags is enabled (#1501)
  • [shopify] support omgmiamiswimwear.com (#1280)
  • implement output.fallback option
  • add archive format to InfoJob output (#875)
  • build executables with SOCKS proxy support (#1424)

Fixes

  • [500px] update query hashes
  • [8muses] fix JSON deobfuscation
  • [artstation] download /4k/ images (#1422)
  • [deviantart] fix pagination for Eclipse results (#1444)
  • [deviantart] improve folder name matching (#1451)
  • [erome] skip deleted albums (#1447)
  • [exhentai] fix image limit detection (#1437)
  • [exhentai] restore limits option (#1487)
  • [gelbooru] fix tag category extraction (#1455)
  • [instagram] update query hashes
  • [komikcast] fix extraction
  • [simplyhentai] fix extraction
  • [slideshare] fix extraction
  • [webtoons] update agegate/GDPR cookies (#1431)
  • fix category-transfer option

Removals

  • [yuki] remove module for yuki.la

1.17.2 - 2021-04-02

Additions

  • [deviantart] add support for posts from watched users (#794)
  • [manganelo] add chapter and manga extractors (#1415)
  • [pinterest] add search extractor (#1411)
  • [sankaku] add tag_string metadata field (#1388)
  • [sankaku] add enumeration index for books (#1388)
  • [tapas] add series and episode extractors (#692)
  • [tapas] implement login with username & password (#692)
  • [twitter] allow specifying a custom format for user results (#1337)
  • [twitter] add extractor for direct image links (#1417)
  • [vk] add support for albums (#474)

Fixes

  • [aryion] unescape paths (#1414)
  • [bcy] improve pagination
  • [deviantart] update watch URL pattern (#794)
  • [deviantart] fix arguments for search/popular results (#1408)
  • [deviantart] use fallback for /intermediary/ URLs
  • [exhentai] improve and simplify image limit checks
  • [komikcast] fix extraction
  • [pixiv] fix favorite URL pattern (#1405)
  • [sankaku] simplify pool tags (#1388)
  • [twitter] improve error message when trying to log in with 2FA (#1409)
  • [twitter] don't use youtube-dl for cards when videos are disabled (#1416)

1.17.1 - 2021-03-19

Additions

  • [architizer] add project and firm extractors (#1369)
  • [deviantart] add watch extractor (#794)
  • [exhentai] support /tag/ URLs (#1363)
  • [gelbooru_v01] support drawfriends.booru.org, vidyart.booru.org, and tlb.booru.org by default
  • [nozomi] support /index-N.html URLs (#1365)
  • [philomena] add generalized extractors for philomena sites (#1379)
  • [philomena] support post URLs without /images/
  • [twitter] implement users option (#1337)
  • implement parent-metadata option (#1364)

Changes

  • [deviantart] revert previous changes to extra option (#1356, #1387)

Fixes

  • [exhentai] improve favorites count extraction (#1360)
  • [gelbooru] update domain for video downloads (#1368)
  • [hentaifox] improve image and metadata extraction (#1366, #1378)
  • [imgur] fix and improve rate limit handling (#1386)
  • [weasyl] improve favorites URL pattern (#1374)
  • use type check before applying browser option (#1358)
  • ensure -s/--simulate always prints filenames (#1360)

Removals

  • [hentaicafe] remove module
  • [hentainexus] remove module
  • [mangareader] remove module
  • [mangastream] remove module

1.17.0 - 2021-03-05

Additions

  • [cyberdrop] add support for https://cyberdrop.me/ (#1328)
  • [exhentai] add metadata option; extract more metadata from gallery pages (#1325)
  • [hentaicafe] add search and tag extractors (#1345)
  • [hentainexus] add original option (#1322)
  • [instagram] support /user/reels/ URLs (#1329)
  • [naverwebtoon] add support for https://comic.naver.com/ (#1331)
  • [pixiv] add translated-tags option (#1354)
  • [tbib] add support for https://tbib.org/ (#473, #1082)
  • [tumblrgallery] add support for https://tumblrgallery.xyz/ (#1298)
  • [twitter] add extractor for followed users (#1337)
  • [twitter] add option to download all media from conversations (#1319)
  • [wallhaven] add collections extractor (#1351)
  • [snap] allow access to user's .netrc for site authentication (#1352)
  • add extractors for Gelbooru v0.1 sites (#234, #426, #473, #767, #1238)
  • add -E/--extractor-info command-line option (#875)
  • add GitHub Actions workflow for building standalone executables (#1312)
  • add browser and headers options (#1117)
  • add option to use different youtube-dl forks (#1330)
  • support using multiple input files at once (#1353)

Changes

  • [deviantart] extend extra option to also download embedded DeviantArt posts.
  • [exhentai] rename metadata fields to match API results (#1325)
  • [mangadex] use api.mangadex.org as default API server
  • [mastodon] cache OAuth tokens (#616)
  • replace wait-min and wait-max with sleep-request

Fixes

  • [500px] skip unavailable photos (#1335)
  • [komikcast] fix extraction
  • [readcomiconline] download high quality image versions (#1347)
  • [twitter] update GraphQL endpoints
  • fix crash when base-directory is an empty string (#1339)

Removals

  • remove support for formerly deprecated options
  • remove cloudflare module

1.16.5 - 2021-02-14

Additions

  • [behance] support video modules (#1282)
  • [erome] add album, user, and search extractors (#409)
  • [hentaifox] support searching by group (#1294)
  • [imgclick] add image extractor (#1307)
  • [kemonoparty] extract inline images (#1286)
  • [kemonoparty] support URLs with non-numeric user and post IDs (#1303)
  • [pillowfort] add user and post extractors (#846)

Changes

  • [kemonoparty] include service in directories and archive keys
  • [pixiv] require a refresh-token to login (#1304)
  • [snap] use core18 as base

Fixes

  • [500px] update query hashes
  • [deviantart] update parameters for /browse/popular (#1267)
  • [deviantart] provide filename extension for original file downloads (#1272)
  • [deviantart] fix folders option (#1302)
  • [inkbunny] add sid parameter to private file downloads (#1281)
  • [kemonoparty] fix absolute file URLs
  • [mangadex] revert to https://mangadex.org/api/ and add api-server option (#1310)
  • [nsfwalbum] use fallback for deleted content (#1259)
  • [sankaku] update invalid token detection (#1309)
  • [slideshare] fix extraction
  • [postprocessor:metadata] fix crash with extension-format (#1285)

1.16.4 - 2021-01-23

Additions

  • [furaffinity] add descriptions option (#1231)
  • [kemonoparty] add user and post extractors (#1216)
  • [nozomi] add num enumeration index (#1239)
  • [photovogue] added portfolio extractor (#1253)
  • [twitter] match /i/user/ID URLs
  • [unsplash] add extractors (#1197)
  • [vipr] add image extractor (#1258)

Changes

  • [derpibooru] use "Everything" filter by default (#862)

Fixes

  • [derpibooru] update date parsing
  • [foolfuuka] stop search when results are exhausted (#1174)
  • [instagram] fix regex for /saved URLs (#1251)
  • [mangadex] update API URLs
  • [mangakakalot] fix extraction
  • [newgrounds] fix flash file extraction (#1257)
  • [sankaku] simplify login process
  • [twitter] fix retries after hitting rate limit

1.16.3 - 2021-01-10

Fixes

  • fix crash when using a dict for path-restrict
  • [postprocessor:metadata] sanitize custom filenames

1.16.2 - 2021-01-09

Additions

  • [derpibooru] add search and gallery extractors (#862)
  • [foolfuuka] add board and search extractors (#1044, #1174)
  • [gfycat] add date metadata field (#1138)
  • [pinterest] add support for getting all boards of a user (#1205)
  • [sankaku] add support for book searches (#1204)
  • [twitter] fetch media from pinned tweets (#1203)
  • [wikiart] add extractor for single paintings (#1233)
  • [downloader:http] add MIME type and signature for .ico files (#1211)
  • add d format string conversion for timestamp values
  • add "ascii" as a special path-restrict value

Fixes

  • [hentainexus] fix extraction (#1234)
  • [instagram] categorize single highlight URLs as highlights (#1222)
  • [redgifs] fix search results
  • [twitter] fix login with username & password
  • [twitter] fetch tweets from homeConversation entries

1.16.1 - 2020-12-27

Additions

  • [instagram] add include option (#1180)
  • [pinterest] implement video support (#1189)
  • [sankaku] reimplement login support (#1176, #1182)
  • [sankaku] add support for sankaku.app URLs (#1193)

Changes

  • [e621] return pool posts in order (#1195)
  • [hentaicafe] prefer title of /hc.fyi/ pages (#1106)
  • [hentaicafe] simplify default filenames
  • [sankaku] normalize created_at metadata (#1190)
  • [postprocessor:exec] do not add missing {} to command (#1185)

Fixes

  • [booru] improve error handling
  • [instagram] warn about private profiles (#1187)
  • [keenspot] improve redirect handling
  • [mangadex] respect chapter-reverse settings (#1194)
  • [pixiv] output debug message on failed login attempts (#1192)
  • increase SQLite connection timeouts (#1173)

Removals

  • [mangapanda] remove module

1.16.0 - 2020-12-12

Additions

  • [booru] implement generalized extractors for *booru and moebooru sites
    • add support for sakugabooru.com (#1136)
    • add support for lolibooru.moe (#1050)
    • provide formattable date metadata fields (#1138)
  • [postprocessor:metadata] add event and filename options (#315, #866, #984)
  • [postprocessor:exec] add event option (#992)

Changes

  • [flickr] update default directories and improve metadata consistency (#828)
  • [sankaku] use API endpoints from beta.sankakucomplex.com
  • [downloader:http] improve filename extension handling (#776)
  • replace all JPEG filename extensions with jpg by default

Fixes

  • [hentainexus] fix extraction (#1166)
  • [instagram] rewrite (#1113, #1122, #1128, #1130, #1149)
  • [mangadex] handle external chapters (#1154)
  • [nozomi] handle empty date fields (#1163)
  • [paheal] create directory for each post (#1147)
  • [piczel] update API URLs
  • [twitter] update image URL format (#1145)
  • [twitter] improve x-csrf-token header handling (#1170)
  • [webtoons] update ageGate cookies

Removals

  • [sankaku] remove login support

1.15.4 - 2020-11-27

Fixes

  • [2chan] skip external links
  • [hentainexus] fix extraction (#1125)
  • [mangadex] switch to API v2 (#1129)
  • [mangapanda] use http://
  • [mangoxo] fix extraction
  • [reddit] skip invalid gallery items (#1127)

1.15.3 - 2020-11-13

Additions

  • [sankakucomplex] extract videos and embeds (#308)
  • [twitter] add support for lists (#1096)
  • [postprocessor:metadata] accept string-lists for content-format (#1080)
  • implement modules and extension-map options

Fixes

  • [500px] update query hashes
  • [8kun] fix file URLs of older posts (#1101)
  • [exhentai] update image URL parsing (#1094)
  • [hentaifoundry] update YII_CSRF_TOKEN cookie handling (#1083)
  • [hentaifoundry] use scheme from input URLs (#1095)
  • [mangoxo] fix metadata extraction
  • [paheal] fix extraction (#1088)
  • collect post processors from basecategory entries (#1084)

1.15.2 - 2020-10-24

Additions

  • [pinterest] implement login support (#1055)
  • [reddit] add date metadata field (#1068)
  • [seiga] add metadata for single image downloads (#1063)
  • [twitter] support media from Cards (#937, #1005)
  • [weasyl] support api-key authentication (#1057)
  • add a t format string conversion for trimming whitespace (#1065)

Fixes

  • [blogger] handle URLs with specified width/height (#1061)
  • [fallenangels] fix extraction of .5 chapters
  • [gelbooru] rewrite mp4 video URLs (#1048)
  • [hitomi] fix image URLs and gallery URL pattern
  • [mangadex] unescape more metadata fields (#1066)
  • [mangahere] ensure download URLs have a scheme (#1070)
  • [mangakakalot] ignore "Go Home" buttons in chapter pages
  • [newgrounds] handle embeds without scheme (#1033)
  • [newgrounds] provide fallback URLs for video downloads (#1042)
  • [xhamster] fix user profile extraction

1.15.1 - 2020-10-11

Additions

  • [hentaicafe] add manga_id metadata field (#1036)
  • [hentaifoundry] add support for stories (#734)
  • [hentaifoundry] add include option
  • [newgrounds] extract image embeds (#1033)
  • [nijie] add include option (#1018)
  • [reactor] match URLs without subdomain (#1053)
  • [twitter] extend retweets option (#1026)
  • [weasyl] add extractors (#977)

Fixes

  • [500px] update query hashes
  • [behance] fix collection extraction
  • [newgrounds] fix video extraction (#1042)
  • [twitter] improve twitpic extraction (#1019)
  • [weibo] handle posts with more than 9 images (#926)
  • [xvideos] fix title extraction
  • fix crash when using --download-archive with --no-skip (#1023)
  • fix issues with blacklist/whitelist defaults (#1051, #1056)

Removals

  • [kissmanga] remove module

1.15.0 - 2020-09-20

Additions

  • [deviantart] support watchers-only/paid deviations (#995)
  • [myhentaigallery] add gallery extractor (#1001)
  • [twitter] support specifying users by ID (#980)
  • [twitter] support /intent/user?user_id=… URLs (#980)
  • add --no-skip command-line option (#986)
  • add blacklist and whitelist options (#492, #844)
  • add filesize-min and filesize-max options (#780)
  • add sleep-extractor and sleep-request options (#788)
  • write skipped files to archive (#550)

Changes

  • [exhentai] update wait time before original image downloads (#978)
  • [imgur] use new API endpoints for image/album data
  • [tumblr] create directories for each post (#965)
  • support format string replacement fields in download archive paths (#985)
  • reduce wait time growth rate for HTTP retries from exponential to linear

Fixes

  • [500px] update query hash
  • [aryion] improve post ID extraction (#981, #982)
  • [danbooru] handle posts without id (#1004)
  • [furaffinity] update download URL extraction (#988)
  • [imgur] fix image/album detection for galleries
  • [postprocessor:zip] defer zip file creation (#968)

Removals

  • [jaiminisbox] remove extractors
  • [worldthree] remove extractors

1.14.5 - 2020-08-30

Additions

  • [aryion] add username/password support (#960)
  • [exhentai] add ability to specify a custom image limit (#940)
  • [furaffinity] add search extractor (#915)
  • [imgur] add search and tag extractors (#934)

Fixes

  • [500px] fix extraction and update URL patterns (#956)
  • [aryion] update folder mime type list (#945)
  • [gelbooru] fix extraction without API
  • [hentaihand] update to new site layout
  • [hitomi] fix redirect processing
  • [reddit] handle deleted galleries (#953)
  • [reddit] improve gallery extraction (#955)

1.14.4 - 2020-08-15

Additions

  • [blogger] add search extractor (#925)
  • [blogger] support searching posts by labels (#925)
  • [inkbunny] add user and post extractors (#283)
  • [instagram] support /reel/ URLs
  • [pinterest] support pinterest.co.uk URLs (#914)
  • [reddit] support gallery posts (#920)
  • [subscribestar] extract attached media files (#852)

Fixes

  • [blogger] improve error messages for missing posts/blogs (#903)
  • [exhentai] adjust image limit costs (#940)
  • [gfycat] skip malformed gfycat responses (#902)
  • [imgur] handle 403 overcapacity responses (#910)
  • [instagram] wait before GraphQL requests (#901)
  • [mangareader] fix extraction
  • [mangoxo] fix login
  • [pixnet] detect password-protected albums (#177)
  • [simplyhentai] fix gallery_id extraction
  • [subscribestar] update date parsing
  • [vsco] handle missing description fields
  • [xhamster] fix extraction (#917)
  • allow parent-directory to work recursively (#905)
  • skip external OAuth tests (#908)

Removals

  • [bobx] remove module

1.14.3 - 2020-07-18

Additions

  • [8muses] support comics.8muses.com URLs
  • [artstation] add following extractor (#888)
  • [exhentai] add domain option (#897)
  • [gfycat] add user and search extractors
  • [imgur] support all /t/... URLs (#880)
  • [khinsider] add format option (#840)
  • [mangakakalot] add manga and chapter extractors (#876)
  • [redgifs] support gifsdeliverynetwork.com URLs (#874)
  • [subscribestar] add user and post extractors (#852)
  • [twitter] add support for nitter.net URLs (#890)
  • add Zsh completion script (#150)

Fixes

  • [gfycat] retry 404'ed videos on redgifs.com (#874)
  • [newgrounds] fix favorites extraction
  • [patreon] yield images and attachments before post files (#871)
  • [reddit] fix AttributeError when using recursion (#879)
  • [twitter] raise proper exception if a user doesn't exist (#891)
  • defer directory creation (#722)
  • set pseudo extension for Metadata messages (#865)
  • prevent exception on Cloudflare challenges (#868)

1.14.2 - 2020-06-27

Additions

  • [artstation] add date metadata field (#839)
  • [mastodon] add date metadata field (#839)
  • [pinterest] add support for board sections (#835)
  • [twitter] add extractor for liked tweets (#837)
  • [twitter] add option to filter media from quoted tweets (#854)
  • [weibo] add date metadata field to status objects (#829)

Fixes

  • [aryion] fix user gallery extraction (#832)
  • [imgur] build directory paths for each file (#842)
  • [tumblr] prevent errors when using reblogs=same-blog (#851)
  • [twitter] always provide an author metadata field (#831, #833)
  • [twitter] don't download video previews (#833)
  • [twitter] improve handling of deleted tweets (#838)
  • [twitter] fix search results (#847)
  • [twitter] improve handling of quoted tweets (#854)
  • fix config lookups when multiple locations are involved (#843)
  • improve output of -K/--list-keywords for parent extractors (#825)
  • call flush() after writing JSON in DataJob() (#727)

1.14.1 - 2020-06-12

Additions

  • [furaffinity] add artist_url metadata field (#821)
  • [redgifs] add user and search extractors (#724)

Changes

  • [deviantart] extend extra option; also search journals for sta.sh links (#712)
  • [twitter] rewrite; use new interface (#806, #740)

Fixes

  • [kissmanga] work around CAPTCHAs (#818)
  • [nhentai] fix extraction (#819)
  • [webtoons] generalize comic extraction code (#820)

1.14.0 - 2020-05-31

Additions

  • [imagechest] add new extractor for imgchest.com (#750)
  • [instagram] add post_url, tags, location, tagged_users metadata (#743)
  • [redgifs] add image extractor (#724)
  • [webtoons] add new extractor for webtoons.com (#761)
  • implement --write-pages option (#736)
  • extend path-restrict option (#662)
  • implement path-replace option (#662, #755)
  • make path and keywords available in logging messages (#574, #575)

Changes

  • [danbooru] change default value of ugoira to false
  • [downloader:ytdl] change default value of forward-cookies to false
  • [downloader:ytdl] fix file extensions when merging into .mkv (#720)
  • write OAuth tokens to cache (#616)
  • use %APPDATA%\gallery-dl for config files and cache on Windows
  • use util.Formatter for formatting logging messages
  • reuse HTTP connections from parent extractors

Fixes

  • [deviantart] use private access tokens for Journals (#738)
  • [gelbooru] simplify and fix pool extraction
  • [imgur] fix extraction of animated images without mp4 entry
  • [imgur] treat /t/unmuted/ URLs as galleries
  • [instagram] fix login with username & password (#756, #771, #797, #803)
  • [reddit] don't send OAuth headers for file downloads (#729)
  • fix/improve Cloudflare bypass code (#728, #757)
  • reset filenames on empty file extensions (#733)

1.13.6 - 2020-05-02

Additions

  • [patreon] respect filters and sort order in query parameters (#711)
  • [speakerdeck] add a new extractor for speakerdeck.com (#726)
  • [twitter] add replies option (#705)
  • [weibo] add videos option
  • [downloader:http] add MIME types for .psd files (#714)

Fixes

  • [artstation] improve embed extraction (#720)
  • [deviantart] limit API wait times (#721)
  • [newgrounds] fix URLs produced by the following extractor (#684)
  • [patreon] improve file hash extraction (#713)
  • [vsco] fix user gallery extraction
  • fix/improve Cloudflare bypass code (#728)

1.13.5 - 2020-04-27

Additions

  • [500px] recognize web.500px.com URLs
  • [aryion] support downloading from folders (#694)
  • [furaffinity] add extractor for followed users (#515)
  • [hitomi] add extractor for tag searches (#697)
  • [instagram] add post_id and num metadata fields (#698)
  • [newgrounds] add extractor for followed users (#684)
  • [patreon] recognize URLs with creator IDs (#711)
  • [twitter] add reply metadata field (#705)
  • [xhamster] recognize xhamster.porncache.net URLs (#700)

Fixes

  • [gelbooru] improve post ID extraction in pool listings
  • [hitomi] fix extraction of galleries without tags
  • [jaiminisbox] update metadata decoding procedure (#702)
  • [mastodon] fix pagination (#701)
  • [mastodon] improve account searches (#704)
  • [patreon] fix hash extraction from download URLs (#693)
  • improve parameter extraction when solving Cloudflare challenges

1.13.4 - 2020-04-12

Additions

  • [aryion] add gallery and post extractors (#390, #673)
  • [deviantart] detect and handle folders in sta.sh listings (#659)
  • [hentainexus] add circle, event, and title_conventional metadata fields (#661)
  • [hiperdex] add artist extractor (#606)
  • [mastodon] add access tokens for mastodon.social and baraag.net (#665)

Changes

  • [deviantart] retrieve all download URLs through the OAuth API
  • automatically read config files in PyInstaller executable directories (#682)

Fixes

  • [deviantart] handle "Request blocked" errors (#655)
  • [deviantart] improve JPEG quality replacement pattern
  • [hiperdex] fix extraction
  • [mastodon] handle API rate limits (#665)
  • [mastodon] update OAuth credentials for pawoo.net (#665)
  • [myportfolio] fix extraction of galleries without title
  • [piczel] fix extraction of single images
  • [vsco] fix collection extraction
  • [weibo] accept status URLs with non-numeric IDs (#664)

1.13.3 - 2020-03-28

Additions

  • [instagram] Add support for user's saved medias (#644)
  • [nozomi] support multiple images per post (#646)
  • [35photo] add tag extractor

Changes

  • [mangadex] transform timestamps from date fields to datetime objects

Fixes

  • [deviantart] handle decode errors for extended_fetch results (#655)
  • [e621] fix bug in API rate limiting and improve pagination (#651)
  • [instagram] update pattern for user profile URLs
  • [mangapark] fix metadata extraction
  • [nozomi] sort search results (#646)
  • [piczel] fix extraction
  • [twitter] fix typo in x-twitter-auth-type header (#625)
  • remove trailing dots from Windows directory names (#647)
  • fix crash with missing stdout/stderr/stdin handles (#653)

1.13.2 - 2020-03-14

Additions

  • [furaffinity] extract more metadata
  • [instagram] add post_shortcode metadata field (#525)
  • [kabeuchi] add extractor (#561)
  • [newgrounds] add extractor for favorited posts (#394)
  • [pixiv] implement avatar option (#595, #623)
  • [twitter] add extractor for bookmarked Tweets (#625)

Fixes

  • [bcy] reduce number of HTTP requests during data extraction
  • [e621] update to new interface (#635)
  • [exhentai] handle incomplete MIME types (#632)
  • [hitomi] improve metadata extraction
  • [mangoxo] fix login
  • [newgrounds] improve error handling when extracting post data

1.13.1 - 2020-03-01

Additions

  • [hentaihand] add extractors (#605)
  • [hiperdex] add chapter and manga extractors (#606)
  • [oauth] implement option to write DeviantArt refresh-tokens to cache (#616)
  • [downloader:http] add more MIME types for .bmp and .rar files (#621, #628)
  • warn about expired cookies

Fixes

  • [bcy] fix partial image URLs (#613)
  • [danbooru] fix Ugoira downloads and metadata
  • [deviantart] check availability of /intermediary/ URLs (#609)
  • [hitomi] follow multiple redirects & fix image URLs
  • [piczel] improve and update
  • [tumblr] replace - with in tag searches (#611)
  • [vsco] update gallery URL pattern
  • fix --verbose and --quiet command-line options

1.13.0 - 2020-02-16

Additions

  • Support for
  • [blogger] implement video extraction (#587)
  • [oauth] add option to specify port number used by local server (#604)
  • [pixiv] add rating metadata field (#595)
  • [pixiv] recognize tags at the end of new bookmark URLs
  • [reddit] add videos option
  • [weibo] use youtube-dl to download from m3u8 manifests
  • implement parent-directory option (#551)
  • extend filename formatting capabilities:
    • implement field name alternatives (#525)
    • allow multiple "special" format specifiers per replacement field (#595)
    • allow for numeric list and string indices

Changes

  • [reddit] handle reddit-hosted images and videos natively (#551)
  • [twitter] change default value for videos to true

Fixes

  • [cloudflare] unescape challenge URLs
  • [deviantart] fix video extraction from extended_fetch results
  • [hitomi] implement workaround for "broken" redirects
  • [khinsider] fix and improve metadata extraction
  • [patreon] filter duplicate files per post (#590)
  • [piczel] fix extraction
  • [pixiv] fix user IDs for bookmarks API calls (#596)
  • [sexcom] fix image URLs
  • [twitter] force old login page layout (#584, #598)
  • [vsco] skip "invalid" entities
  • improve functions to load/save cookies.txt files (#586)

Removals

  • [yaplog] remove module

1.12.3 - 2020-01-19

Additions

  • [hentaifoundry] extract more metadata (#565)
  • [twitter] add option to extract TwitPic embeds (#579)
  • implement a post-processor module to compare file versions (#530)

Fixes

  • [hitomi] update image URL generation
  • [mangadex] revert domain to mangadex.org
  • [pinterest] improve detection of invalid pin.it links
  • [pixiv] update URL patterns for user profiles and bookmarks (#568)
  • [twitter] Fix stop before real end (#573)
  • remove temp files before downloading from fallback URLs

Removals

  • [erolord] remove extractor

1.12.2 - 2020-01-05

Additions

  • [deviantart] match new search/popular URLs (#538)
  • [deviantart] match /favourites/all URLs (#555)
  • [deviantart] add extractor for followed users (#515)
  • [pixiv] support listing followed users (#515)
  • [imagefap] handle beta.imagefap.com URLs (#552)
  • [postprocessor:metadata] add directory option (#520)

Fixes

  • [artstation] fix search result pagination (#537)
  • [directlink] send Referer headers (#536)
  • [exhentai] restrict default directory name length (#545)
  • [mangadex] change domain to mangadex.cc (#559)
  • [mangahere] send isAdult cookies (#556)
  • [newgrounds] fix tags metadata extraction
  • [pixiv] retry after rate limit errors (#535)
  • [twitter] handle quoted tweets (#526)
  • [twitter] handle API rate limits (#526)
  • [twitter] fix URLs forwarded to youtube-dl (#540)
  • prevent infinite recursion when spawning new extractors (#489)
  • improve output of --list-keywords for "parent" extractors (#548)
  • provide fallback for SQLite versions with missing WITHOUT ROWID support (#553)

1.12.1 - 2019-12-22

Additions

  • [4chan] add extractor for entire boards (#510)
  • [realbooru] add extractors for pools, posts, and tag searches (#514)
  • [instagram] implement a videos option (#521)
  • [vsco] implement a videos option
  • [postprocessor:metadata] implement a bypost option for downloading the metadata of an entire post (#511)

Changes

  • [reddit] change the default value for comments to 0
  • [vsco] improve image resolutions
  • make filesystem-related errors during file downloads non-fatal (#512)

Fixes

  • [foolslide] add fallback for chapter data extraction
  • [instagram] ignore errors during post-page extraction
  • [patreon] avoid errors when fetching user info (#508)
  • [patreon] improve URL pattern for single posts
  • [reddit] fix errors with t1 submissions
  • [vsco] fix user profile extraction … again
  • [weibo] handle unavailable/deleted statuses
  • [downloader:http] improve rate limit handling
  • retain trailing zeroes in Cloudflare challenge answers

1.12.0 - 2019-12-08

Additions

  • [flickr] support 3k, 4k, 5k, and 6k photo sizes (#472)
  • [imgur] add extractor for subreddit links (#500)
  • [newgrounds] add extractors for audio listings and general media files (#394)
  • [newgrounds] implement login support (#394)
  • [postprocessor:metadata] implement a extension-format option (#477)
  • --exec-after

Changes

  • [deviantart] ensure consistent username capitalization (#455)
  • [directlink] split {path} into {path}/{filename}.{extension}
  • [twitter] update metadata fields with user/author information
  • [postprocessor:metadata] filter private entries & rename format to content-format
  • Enable cookies-update by default

Fixes

  • [2chan] fix metadata extraction
  • [behance] get images from 'media_collection' modules
  • [bobx] fix image downloads by randomly generating session cookies (#482)
  • [deviantart] revert to getting download URLs from OAuth API calls (#488)
  • [deviantart] fix URL generation from '/extended_fetch' results (#505)
  • [flickr] adjust OAuth redirect URI (#503)
  • [hentaifox] fix extraction
  • [imagefap] adapt to new image URL format
  • [imgbb] fix error in galleries without user info (#471)
  • [instagram] prevent errors with missing 'video_url' fields (#479)
  • [nijie] fix date parsing
  • [pixiv] match new search URLs (#507)
  • [plurk] fix comment pagination
  • [sexcom] send specific Referer headers when downloading videos
  • [twitter] fix infinite loops (#499)
  • [vsco] fix user profile and collection extraction (#480)
  • Fix Cloudflare DDoS protection bypass

Removals

  • --abort-on-skip

1.11.1 - 2019-11-09

Fixes

  • Fix inclusion of bash completion and man pages in source distributions

1.11.0 - 2019-11-08

Additions

Changes

  • Rewrite imgur using its public API (#446)
  • Rewrite luscious using GraphQL queries (#457)
  • Adjust default nijie filenames to match pixiv
  • Change enumeration index for gallery extractors from page to num
  • Return non-zero exit status when errors occurred
  • Forward proxy settings to youtube-dl downloader
  • Install bash completion script into share/bash-completion/completions

Fixes

  • Adapt to new instagram page layout when logged in (#391)
  • Support protected twitter videos (#452)
  • Extend hitomi URL pattern and fix gallery extraction
  • Restore OAuth2 authentication error messages
  • Miscellaneous fixes for patreon (#444), deviantart (#455), sexcom (#464), imgur (#467), simplyhentai

1.10.6 - 2019-10-11

Additions

  • --exec command-line option to specify a command to run after each file download (#421)

Changes

  • Include titles in gfycat default filenames (#434)

Fixes

  • Fetch working download URLs for deviantart (#436)
  • Various fixes and improvements for yaplog blogs (#443)
  • Fix image URL generation for hitomi galleries
  • Miscellaneous fixes for behance and xvideos

1.10.5 - 2019-09-28

Additions

  • instagram.highlights option to include highlighted stories when downloading user profiles (#329)
  • Support for /user/ URLs on reddit (#350)
  • Support for imgur user profiles and favorites (#420)
  • Additional metadata fields on nijie(#423)

Fixes

  • Improve handling of private deviantart artworks (#414) and 429 status codes (#424)
  • Prevent fatal errors when trying to open download-archive files (#417)
  • Detect and ignore unavailable videos on weibo (#427)
  • Update the scope of new reddit refresh-tokens (#428)
  • Fix inconsistencies with the reddit.comments option (#429)
  • Extend URL patterns for hentaicafe manga and pixiv artworks
  • Improve detection of unavailable albums on luscious and imgbb
  • Miscellaneous fixes for tsumino

1.10.4 - 2019-09-08

Additions

Fixes

  • Download the correct files from twitter replies (#403)
  • Prevent crash when trying to use unavailable downloader modules (#405)
  • Fix pixiv authentication (#411)
  • Improve exhentai image limit checks
  • Miscellaneous fixes for hentaicafe, simplyhentai, tumblr

1.10.3 - 2019-08-30

Additions

  • Provide filename metadata for all deviantart files (#392, #400)
  • Implement a ytdl.outtmpl option to let youtube-dl handle filenames by itself (#395)
  • Support seiga mobile URLs (#401)

Fixes

  • Extract more than the first 32 posts from piczel galleries (#396)
  • Fix filenames of archives created with --zip (#397)
  • Skip unavailable images and videos on flickr (#398)
  • Fix filesystem paths on Windows with Python 3.6 and lower (#402)

1.10.2 - 2019-08-23

Additions

  • Support for instagram stories and IGTV (#371, #373)
  • Support for individual imgbb images (#363)
  • deviantart.quality option to set the JPEG compression quality for newer images (#369)
  • enumerate option for extractor.skip (#306)
  • adjust-extensions option to control filename extension adjustments
  • path-remove option to remove control characters etc. from filesystem paths

Changes

  • Rename restrict-filenames to path-restrict
  • Adjust pixiv metadata and default filename format (#366)
    • Set filename to "{category}_{user[id]}_{id}{suffix}.{extension}" to restore the old default
  • Improve and optimize directory and filename generation

Fixes

  • Allow the classify post-processor to handle files with unknown filename extension (#138)
  • Fix rate limit handling for OAuth APIs (#368)
  • Fix artwork and scraps extraction on deviantart (#376, #392)
  • Distinguish between imgur album and gallery URLs (#380)
  • Prevent crash when using --ugoira-conv (#382)
  • Handle multi-image posts on patreon (#383)
  • Miscellaneous fixes for *reactor, simplyhentai

1.10.1 - 2019-08-02

Fixes

  • Use the correct domain for exhentai.org input URLs

1.10.0 - 2019-08-01

Warning

  • Prior to version 1.10.0 all cache files were created world readable (mode 644) leading to possible sensitive information disclosure on multi-user systems
  • It is recommended to restrict access permissions of already existing files (/tmp/.gallery-dl.cache) with chmod 600
  • Windows users should not be affected

Additions

Changes

  • The default cache file location on non-Windows systems is now
    • $XDG_CACHE_HOME/gallery-dl/cache.sqlite3 or
    • ~/.cache/gallery-dl/cache.sqlite3
  • New cache files are created with mode 600
  • exhentai extractors will always use e-hentai.org as domain

Fixes

  • Better handling of exhentai image limits and errors (#356, #360)
  • Try to prevent ZIP file corruption (#355)
  • Miscellaneous fixes for behance, ngomik

1.9.0 - 2019-07-19

Additions

  • Support for
  • Add login support for instagram (#195)
  • Add --no-download and extractor.*.download disable file downloads (#220)
  • Add -A/--abort to specify the number of consecutive download skips before aborting
  • Interpret -1 as infinite retries (#300)
  • Implement custom log message formats per log-level (#304)
  • Implement an mtime post-processor that sets file modification times according to metadata fields (#332)
  • Implement a twitter.content option to enable tweet text extraction (#333, #338)
  • Enable date-min/-max/-format options for tumblr (#337)

Changes

  • Set file modification times according to their Last-Modified header when downloading (#236, #277)
    • Use --no-mtime or downloader.*.mtime to disable this behavior
  • Duplicate download URLs are no longer silently ignored (controllable with extractor.*.image-unique)
  • Deprecate --abort-on-skip

Fixes

  • Retry downloads on OpenSSL exceptions (#324)
  • Ignore unavailable pins on sexcom instead of raising an exception (#325)
  • Use Firefox's SSL/TLS ciphers to prevent Cloudflare CAPTCHAs (#342)
  • Improve folder name matching on deviantart (#343)
  • Forward cookies to youtube-dl to allow downloading private videos
  • Miscellaneous fixes for 35photo, 500px, newgrounds, simplyhentai

1.8.7 - 2019-06-28

Additions

Changes

  • Delete empty archives created by the zip post-processor (#316)

Fixes

  • Handle hitomi Game CG galleries correctly (#321)
  • Miscellaneous fixes for deviantart, hitomi, pururin, kissmanga, keenspot, mangoxo, imagefap

1.8.6 - 2019-06-14

Additions

Changes

  • Detect directlink URLs with upper case filename extensions (#296)

Fixes

  • Improved error handling for tumblr API calls (#297)
  • Fixed extraction of livedoor blogs (#301)
  • Fixed extraction of special deviantart Sta.sh items (#307)
  • Fixed pagination for specific keenspot comics

1.8.5 - 2019-06-01

Additions

Changes

  • Adjust wallhaven extractors to new page layout:
    • use API and add api-key option
    • removed traditional login support
  • Provide original filenames for patreon downloads (#268)
  • Use e-hentai.org or exhentai.org depending on input URL (#278)

Fixes

  • Fix pagination over sankaku popular listings (#265)
  • Fix folder and collection extraction on deviantart (#271)
  • Detect "AreYouHuman" redirects on readcomiconline (#279)
  • Miscellaneous fixes for hentainexus, livedoor, ngomik

1.8.4 - 2019-05-17

Additions

Changes

  • Improved flickr metadata and video extraction (#246)

Fixes

  • Download original GIF animations from deviantart (#242)
  • Ignore missing edge_media_to_comment fields on instagram (#250)
  • Fix serialization of datetime objects for --write-metadata (#251, #252)
  • Allow multiple post-processor command-line options at once (#253)
  • Prevent crash on booru sites when no tags are available (#259)
  • Fix extraction on instagram after rhx_gis field removal (#266)
  • Avoid Cloudflare CAPTCHAs for Python interpreters built against OpenSSL < 1.1.1
  • Miscellaneous fixes for luscious

1.8.3 - 2019-05-04

Additions

Changes

  • Standalone executables are now built using PyInstaller:
    • uses the latest CPython interpreter (Python 3.7.3)
    • available on several platforms (Windows, Linux, macOS)
    • includes the certifi CA bundle, youtube-dl, and pyOpenSSL on Windows

Fixes

  • Patch urllib3's default list of SSL/TLS ciphers to prevent Cloudflare CAPTCHAs (#227) (Windows users need to install pyOpenSSL for this to take effect)
  • Provide fallback URLs for twitter images (#237)
  • Send Referer headers when downloading from hitomi (#239)
  • Updated login procedure on mangoxo

1.8.2 - 2019-04-12

Additions

Changes

  • Increased required Requests version to 2.11.0

Fixes

  • Improved image quality on reactor sites (#210)
  • Support imagebam galleries with more than 100 images (#219)
  • Updated Cloudflare bypass code

1.8.1 - 2019-03-29

Additions

Changes

  • Better FFmpeg arguments for --ugoira-conv
  • Adjusted metadata for luscious albums

Fixes

  • Proper handling of instagram multi-image posts (#178, #201)
  • Fixed tumblr avatar URLs when not using OAuth1.0 (#193)
  • Miscellaneous fixes for exhentai, komikcast

1.8.0 - 2019-03-15

Additions

Changes

  • More metadata for nhentai galleries
  • Combined myportfolio extractors into one
  • Renamed name metadata field to filename and removed the original filename field
  • Simplified and improved internal data structures
  • Optimized creation of child extractors

Fixes

  • Filter empty tumblr URLs (#165)
  • Filter ads and improve connection speed on hentaifoundry
  • Show proper error messages if luscious galleries are unavailable
  • Miscellaneous fixes for mangahere, ngomik, simplyhentai, imgspice

Removals

  • seaotterscans

1.7.0 - 2019-02-05

1.6.3 - 2019-01-18

  • Added metadata post-processor to write image metadata to an external file (#135)
  • Added option to reverse chapter order of manga extractors (#149)
  • Added authentication support for danbooru (#151)
  • Added tag metadata for exhentai and hbrowse galleries
  • Improved *reactor extractors (#148)
  • Fixed extraction issues for nhentai (#156), pinterest, mangapark

1.6.2 - 2019-01-01

  • Added support for:
  • Added support for multiple items on sta.sh pages (#113)
  • Added option to download tumblr avatars (#137)
  • Changed defaults for visited post types and inline media on tumblr
  • Improved inline extraction of tumblr posts (#133, #137)
  • Improved error handling and retry behavior of all API calls
  • Improved handling of missing fields in format strings (#136)
  • Fixed hash extraction for unusual tumblr URLs (#129)
  • Fixed image subdomains for hitomi galleries (#142)
  • Fixed and improved miscellaneous issues for kissmanga (#20), luscious, mangapark, readcomiconline

1.6.1 - 2018-11-28

1.6.0 - 2018-11-17

  • Added support for:
  • Added youtube-dl integration and video downloads for twitter (#99), behance, artstation
  • Added per-extractor options for network connections (retries, timeout, verify)
  • Added a --no-check-certificate command-line option
  • Added ability to specify the number of skipped downloads before aborting/exiting (#115)
  • Added extractors for scraps, favorites, popular and recent images on hentaifoundry (#110)
  • Improved login procedure for pixiv to avoid unwanted emails on each new login
  • Improved album metadata and error handling for flickr (#109)
  • Updated default User-Agent string to Firefox 62 (#122)
  • Fixed twitter API response handling when logged in (#123)
  • Fixed issue when converting Ugoira using H.264
  • Fixed miscellaneous issues for 2chan, deviantart, fallenangels, flickr, imagefap, pinterest, turboimagehost, warosu, yuki (#112)

1.5.3 - 2018-09-14

  • Added support for:
  • Added black-/whitelist options for post-processor modules
  • Added support for tumblr inline videos (#102)
  • Fixed extraction of smugmug albums without owner (#100)
  • Fixed issues when using default config values with reddit extractors (#104)
  • Fixed pagination for user favorites on sankaku (#106)
  • Fixed a crash when processing deviantart journals (#108)

1.5.2 - 2018-08-31

  • Added support for twitter timelines (#96)
  • Added option to suppress FFmpeg output during ugoira conversions
  • Improved filename formatter performance
  • Improved inline image quality on tumblr (#98)
  • Fixed image URLs for newly released mangadex chapters
  • Fixed a smaller issue with deviantart journals
  • Replaced subapics with ngomik

1.5.1 - 2018-08-17

  • Added support for:
  • Added support for related pins on pinterest
  • Fixed accessing "offensive" galleries on exhentai (#97)
  • Fixed extraction issues for mangadex, komikcast and behance
  • Removed original-image functionality from tumblr, since "raw" images are no longer accessible

1.5.0 - 2018-08-03

  • Added support for:
  • Added custom format string options to handle long strings (#92, #94)
    • Slicing: "{field[10:40]}"
    • Replacement: "{field:L40/too long/}"
  • Improved frame rate handling for ugoira conversions
  • Improved private access token usage on deviantart
  • Fixed metadata extraction for some images on nijie
  • Fixed chapter extraction on mangahere
  • Removed whatisthisimnotgoodwithcomputers
  • Removed support for Python 3.3

1.4.2 - 2018-07-06

  • Added image-pool extractors for safebooru and rule34
  • Added option for extended tag information on booru sites (#92)
  • Added support for DeviantArt's new URL format
  • Added support for mangapark mirrors
  • Changed imagefap extractors to use HTTPS
  • Fixed crash when skipping downloads for files without known extension

1.4.1 - 2018-06-22

  • Added an ugoira post-processor to convert pixiv animations to WebM
  • Added --zip and --ugoira-conv command-line options
  • Changed how ugoira frame information is handled
    • instead of being written to a separate file, it is now made available as metadata field of the ZIP archive
  • Fixed manga and chapter titles for mangadex
  • Fixed file deletion by post-processors

1.4.0 - 2018-06-08

  • Added support for:
  • Added extractors for
    • pixiv search results and followed users
    • deviantart search results and popular listings
  • Added post-processors to perform actions on downloaded files
  • Added options to configure logging behavior
  • Added OAuth support for smugmug
  • Changed pixiv extractors to use the AppAPI
    • this breaks favorite archive IDs and changes some metadata fields
  • Changed the default filename format for tumblr and renamed offset to num
  • Fixed a possible UnicodeDecodeError during installation (#86)
  • Fixed extraction of mangadex manga with more than 100 chapters (#84)
  • Fixed miscellaneous issues for imgur, reddit, komikcast, mangafox and imagebam

1.3.5 - 2018-05-04

  • Added support for:
  • Added title information for mangadex chapters
  • Improved the pinterest API implementation (#83)
  • Improved error handling for deviantart and tumblr
  • Removed gomanga and puremashiro

1.3.4 - 2018-04-20

  • Added support for custom OAuth2 credentials for pinterest
  • Improved rate limit handling for tumblr extractors
  • Improved hentaifoundry extractors
  • Improved imgur URL patterns
  • Fixed miscellaneous extraction issues for luscious and komikcast
  • Removed loveisover and spectrumnexus

1.3.3 - 2018-04-06

  • Added extractors for
    • nhentai search results
    • exhentai search results and favorites
    • nijie doujins and favorites
  • Improved metadata extraction for exhentai and nijie
  • Improved tumblr extractors by avoiding unnecessary API calls
  • Fixed Cloudflare DDoS protection bypass
  • Fixed errors when trying to print unencodable characters

1.3.2 - 2018-03-23

  • Added extractors for artstation albums, challenges and search results
  • Improved URL and metadata extraction for hitomiand nhentai
  • Fixed page transitions for danbooru API results (#82)

1.3.1 - 2018-03-16

  • Added support for:
  • Added Cloudflare DDoS protection bypass to komikcast extractors
  • Changed archive ID formats for deviantart folders and collections
  • Improved error handling for deviantart API calls
  • Removed imgchili and various smaller image hosts

1.3.0 - 2018-03-02

  • Added --proxy to explicitly specify a proxy server (#76)
  • Added options to customize archive ID formats and undefined replacement fields
  • Changed various archive ID formats to improve their behavior for favorites / bookmarks / etc.
    • Affected modules are deviantart, flickr, tumblr, pixiv and all …boorus
  • Improved sankaku and idolcomplex support by
    • respecting page and next URL parameters (#79)
    • bypassing the page-limit for unauthenticated users
  • Improved directlink metadata by properly unquoting it
  • Fixed pixiv ugoira extraction (#78)
  • Fixed miscellaneous extraction issues for mangastream and tumblr
  • Removed yeet, chronos, coreimg, hosturimage, imageontime, img4ever, imgmaid, imgupload

1.2.0 - 2018-02-16

  • Added support for:
  • Added --download-archive to record downloaded files in an archive file
  • Added --write-log to write logging output to a file
  • Added a filetype check on download completion to fix incorrectly assigned filename extensions (#63)
  • Added the tumblr:... pseudo URI scheme to support custom domains for Tumblr blogs (#71)
  • Added fallback URLs for tumblr images (#64)
  • Added support for reddit-hosted images (#68)
  • Improved the input file format by allowing comments and per-URL options
  • Fixed OAuth 1.0 signature generation for Python 3.3 and 3.4 (#75)
  • Fixed smaller issues for luscious, hentai2read, hentaihere and imgur
  • Removed the batoto module

1.1.2 - 2018-01-12

  • Added support for:
  • Added an option to filter reblogs on tumblr (#61)
  • Added OAuth user authentication for tumblr (#65)
  • Added support for slideshare mobile URLs (#67)
  • Improved pagination for various …booru sites to work around page limits
  • Fixed chapter information parsing for certain manga on kissmanga (#58) and batoto (#60)

1.1.1 - 2017-12-22

  • Added support for:
  • Added pool- and post-extractors for sankaku
  • Added OAuth user authentication for deviantart
  • Updated luscious to support members.luscious.net URLs (#55)
  • Updated mangahere to use their new domain name (mangahere.cc) and support mobile URLs
  • Updated gelbooru to not be restricted to the first 20,000 images (#56)
  • Fixed extraction issues for nhentai and khinsider

1.1.0 - 2017-12-08

  • Added the -r/--limit-rate command-line option to set a maximum download rate
  • Added the --sleep command-line option to specify the number of seconds to sleep before each download
  • Updated gelbooru to no longer use their now disabled API
  • Fixed SWF extraction for sankaku (#52)
  • Fixed extraction issues for hentai2read and khinsider
  • Removed the deprecated --images and --chapters options
  • Removed the mangazuki module

1.0.2 - 2017-11-24

  • Added an option to set a custom user-agent string
  • Improved retry behavior for failed HTTP requests
  • Improved seiga by providing better metadata and getting more than the latest 200 images
  • Improved tumblr by adding support for all post types, scanning for inline images and following external links (#48)
  • Fixed extraction issues for hbrowse, khinsider and senmanga

1.0.1 - 2017-11-10

  • Added support for:
  • Fixed exception handling during file downloads which could lead to a premature exit
  • Fixed an issue with tumblr where not all images would be downloaded when using tags (#48)
  • Fixed extraction issues for imgbox (#47), mangastream (#49) and mangahere

1.0.0 - 2017-10-27

  • Added support for:
  • Added support for pixiv ranking lists
  • Added support for booru popular lists (danbooru, e621, konachan, yandere, 3dbooru)
  • Added the --cookies command-line and cookies config option to load additional cookies
  • Added the --filter and --chapter-filter command-line options to select individual images or manga-chapters by their metadata using simple Python expressions (#43)
  • Added the verify config option to control certificate verification during file downloads
  • Added config options to overwrite internally used API credentials (API Tokens & IDs)
  • Added -K as a shortcut for --list-keywords
  • Changed the --images and --chapters command-line options to --range and --chapter-range
  • Changed keyword names for various modules to make them accessible by --filter. In general minus signs have been replaced with underscores (e.g. gallery-id -> gallery_id).
  • Changed default filename formats for manga extractors to optionally use volume and title information
  • Improved the downloader modules to use .part files and support resuming incomplete downloads (#29)
  • Improved deviantart by distinguishing between users and groups (#26), always using HTTPS, and always downloading full-sized original images
  • Improved sankaku by adding authentication support and fixing various other issues (#44)
  • Improved URL pattern for direct image links (#30)
  • Fixed an issue with luscious not getting original image URLs (#33)
  • Fixed various smaller issues for batoto, hentai2read (#38), jaiminisbox, khinsider, kissmanga (#28, #46), mangahere, pawoo, twitter
  • Removed kisscomic and yonkouprod modules

0.9.1 - 2017-07-24

0.9.0 - 2017-06-28

  • Added support for:
  • Added support for direct image links
  • Added user authentication via OAuth for reddit and flickr
  • Added support for user authentication data from .netrc files (#22)
  • Added a simple progress indicator for multiple URLs (#19)
  • Added the --write-unsupported command-line option to write unsupported URLs to a file
  • Added documentation for all available config options (configuration.rst)
  • Improved pixiv to support tags for user downloads (#17)
  • Improved pixiv to support shortened and http://pixiv.me/... URLs (#23)
  • Improved imgur to properly handle .gifv images and provide better metadata
  • Fixed an issue with kissmanga where metadata parsing for some series failed (#20)
  • Fixed an issue with getting filename extensions from Content-Type response headers

0.8.4 - 2017-05-21

  • Added the --abort-on-skip option to stop extraction if a download would be skipped
  • Improved the output format of the --list-keywords option
  • Updated deviantart to support all media types and journals
  • Updated fallenangels to support their Vietnamese version
  • Fixed an issue with multiple tags on ...booru sites
  • Removed the yomanga module

0.8.3 - 2017-05-01

  • Added support for https://pawoo.net/
  • Added manga extractors for all FoOlSlide-based modules
  • Added the -q/--quiet and -v/--verbose options to control output verbosity
  • Added the -j/--dump-json option to dump extractor results in JSON format
  • Added the --ignore-config option
  • Updated the exhentai extractor to fall back to using the e-hentai version if no username is given
  • Updated deviantart to support sta.sh URLs
  • Fixed an issue with kissmanga which prevented image URLs from being decrypted properly (again)
  • Fixed an issue with pixhost where for an image inside an album it would always download the first image of that album (#13)
  • Removed the mangashare and readcomics modules

0.8.2 - 2017-04-10

  • Fixed an issue in kissmanga which prevented image URLs from being decrypted properly

0.8.1 - 2017-04-09

  • Added new extractors:
  • Added a favourites extractor for deviantart
  • Re-enabled the kissmanga module
  • Updated nijie to support multi-page image listings
  • Updated mangastream to support readms.net URLs
  • Updated exhentai to support e-hentai.org URLs
  • Updated fallenangels to support their new domain and site layout

0.8.0 - 2017-03-28

  • Added logging support
  • Added the -R/--retries option to specify how often a download should be retried before giving up
  • Added the --http-timeout option to set a timeout for HTTP connections
  • Improved error handling/tolerance during HTTP file downloads (#10)
  • Improved option parsing and the help message from -h/--help
  • Changed the way configuration values are used by prioritizing top-level values
    • This allows for cmdline options like -u/--username to overwrite values set in configuration files
  • Fixed an issue with imagefap.com where incorrectly reported gallery sizes would cause the extractor to fail (#9)
  • Fixed an issue with seiga.nicovideo.jp where invalid characters in an API response caused the XML parser to fail
  • Fixed an issue with seiga.nicovideo.jp where the filename extension for the first image would be used for all others
  • Removed support for old configuration paths on Windows
  • Removed several modules:
    • mangamint: site is down
    • whentai: now requires account with VIP status for original images
    • kissmanga: encrypted image URLs (will be re-added later)

0.7.0 - 2017-03-06

  • Added --images and --chapters options
    • Specifies which images (or chapters) to download through a comma-separated list of indices or index-ranges
    • Example: --images -2,4,6-8,10- will select images with index 1, 2, 4, 6, 7, 8 and 10 up to the last one
  • Changed the -g/--get-urls option
    • The amount of how often the -g option is given now determines up until which level URLs are resolved.
    • See 3bca86618505c21628cd9c7179ce933a78d00ca2
  • Changed several option keys:
    • directory_fmt -> directory
    • filename_fmt -> filename
    • download-original -> original
  • Improved FoOlSlide-based extractors
  • Fixed URL extraction for hentai2read
  • Fixed an issue with deviantart, where the API access token wouldn't get refreshed

0.6.4 - 2017-02-13

  • Added new extractors:
    • fallenangels (famatg.com)
  • Fixed url- and data-extraction for:
    • nhentai
    • mangamint
    • twitter
    • imagetwist
  • Disabled InsecureConnectionWarning when no certificates are available

0.6.3 - 2017-01-25

  • Added new extractors:
    • gomanga
    • yomanga
    • mangafox
  • Fixed deviantart extractor failing - switched to using their API
  • Fixed an issue with SQLite on Python 3.6
  • Automated test builds via Travis CI
  • Standalone executables for Windows

0.6.2 - 2017-01-05

  • Added new extractors:
    • kisscomic
    • readcomics
    • yonkouprod
    • jaiminisbox
  • Added manga extractor to batoto-module
  • Added user extractor to seiga-module
  • Added -i/--input-file argument to allow local files and stdin as input (like wget)
  • Added basic support for file:// URLs
    • this allows for the recursive extractor to be applied to local files:
    • $ gallery-dl r:file://[path to file]
  • Added a utility extractor to run unit test URLs
  • Updated luscious to deal with API changes
  • Fixed twitter to provide the original image URL
  • Minor fixes to hentaifoundry
  • Removed imgclick extractor

0.6.1 - 2016-11-30

  • Added new extractors:
    • whentai
    • readcomiconline
    • sensescans, worldthree
    • imgmaid, imagevenue, img4ever, imgspot, imgtrial, pixhost
  • Added base class for extractors of FoOlSlide-based sites
  • Changed default paths for configuration files on Windows
    • old paths are still supported, but that will change in future versions
  • Fixed aborting downloads if a single one failed (#5)
  • Fixed cloudflare-bypass cache containing outdated cookies
  • Fixed image URLs for hitomi and 8chan
  • Updated deviantart to always provide the highest quality image
  • Updated README.rst
  • Removed doujinmode extractor

0.6.0 - 2016-10-08

  • Added new extractors:
    • hentaihere
    • dokireader
    • twitter
    • rapidimg, picmaniac
  • Added support to find filename extensions by Content-Type response header
  • Fixed filename/path issues on Windows (#4):
    • Enable path names with more than 260 characters
    • Remove trailing spaces in path segments
  • Updated Job class to automatically set category/subcategory keywords

0.5.2 - 2016-09-23

  • Added new extractors:
    • pinterest
    • rule34
    • dynastyscans
    • imagebam, coreimg, imgcandy, imgtrex
  • Added login capabilities for batoto
  • Added --version cmdline argument to print the current program version and exit
  • Added --list-extractors cmdline argument to print names of all extractor classes together with descriptions and example URLs
  • Added proper error messages if an image/user does not exist
  • Added unittests for every extractor

0.5.1 - 2016-08-22

  • Added new extractors:
    • luscious
    • doujinmode
    • hentaibox
    • seiga
    • imagefap
  • Changed error output to use stderr instead of stdout
  • Fixed broken pipes causing an exception-dump by catching BrokenPipeErrors

0.5.0 - 2016-07-25

0.4.1 - 2015-12-03

  • New modules (imagetwist, turboimagehost)
  • Manga-extractors: Download entire manga and not just single chapters
  • Generic extractor (provisional)
  • Better and configurable console output
  • Windows support

0.4.0 - 2015-11-26

0.3.3 - 2015-11-10

0.3.2 - 2015-11-04

0.3.1 - 2015-10-30

0.3.0 - 2015-10-05

0.2.0 - 2015-06-28

0.1.0 - 2015-05-27