[TikTok] Add support

[CasualYT31]

This PR adds support for TikTok slideshows.

List of supported URL formats:

• /photo/
• /video/
• vm.tiktok.com
• vt.tiktok.com
• /t/
• www.tiktokv.com/share/video
• User profiles (tiktok.com/@username)

I haven't performed extensive testing yet so there could well be edge cases that aren't covered by the extractor as it stands right now. I have tested with cookies, however, and I am able to extract photos from private accounts (as well as detect when cookies are required to extract photo URLs).

Closes: #3061
Closes: #4177
Closes: #5646
Closes: #6878

[CasualYT31]

Hi @mikf, do you have any outstanding feedback for this PR? Otherwise it's ready for merging now. I've been testing this for a few weeks now and I haven't run into any issues.

[idanyekutiel]

Would be nice to have this feature!

[Jalenzzz]

bump

[mikf]

If I were to merge this and kept things the way they are right now, end users would complain to no end that TikTok doesn't work with gallery-dl.exe.

[AAndyProgram]

Well you could write TikTok's requirements somewhere. Or direct all questions to the plugin author.

[mikf]

direct all questions to the plugin author

Sure. That's a brilliant idea.

[CasualYT31]

The code in this PR has a hard requirement on yt-dlp, did I get that right? I'm not a big fan of this, but given that the yt-dlp tiktok module is nearly 2,000 lines of code, it seems like the best option.

So yt-dlp is only required for videos, audio, and user profiles (this includes profile avatars, as it makes little sense to download those if someone is only providing individual photo links). If you provide individual photo links, yt-dlp is not required. This is detailed in docs/supportedsites.md.

I hold the same opinion. It's smelly but we would end up duplicating so much code for little benefit imo. I don't agree with adding video support directly into gallery-dl because that feels like the domain of yt-dlp to me, and if a user is requesting an entire profile, they are likely to want videos, too, so in my mind it makes more sense to introduce a hard dependency there instead of duplicating thousands of lines of code. When yt-dlp's TikTok extractor inevitably breaks, ours would too. The difference is you'd only need to change the code in one place if we just have yt-dlp be a hard dependency (assuming the break isn't in the little amount of code I did duplicate).

This would also make it basically impossible to use with the current gallery-dl.exe builds, given that they ship with Python 3.8 and legacy youtube-dl. There are some planned changes for the 1.29.0 gallery-dl release, which would "fix" this, though: 52ac3a7

If I were to merge this and kept things the way they are right now, end users would complain to no end that TikTok doesn't work with gallery-dl.exe.

I didn't realise that build came shipped with youtube-dl (I assume it's embedded into the executable?), that is nice to know. Yes, even though the 3.8 tests pass for TikTok it won't be possible to use with that executable since TikTok extraction in youtube-dl has long since been broken. Hopefully youtube-dl could be replaced with yt-dlp seeing as you're hoping to upgrade to 3.13 and then this should be addressed (yt-dlp isn't officially supported in 3.8 as you might know). Although whenever yt-dlp's TikTok extractor breaks, gallery-dl will "break" too... But I'm not sure how we could avoid this.

Just point out that TikTok is heavily dependent on yt-dlp.

Well you could write TikTok's requirements somewhere.

I've added notes on this to the docs/supportedsites.md file as part of this PR. @mikf Let me know if there is a better place for this information.

[mikf]

Sorry this took so long ...

[mikf]

I updated the code a bit: 0679641 34c776f a88547a 1de45b4

Tests all pass, but could someone try it with these changes anyway and report back if anything broke?

[CasualYT31]

I updated the code a bit: 0679641 34c776f a88547a 1de45b4

Tests all pass, but could someone try it with these changes anyway and report back if anything broke?

Sure, I'll do it tomorrow, and I'll give your changes a once over. I didn't implement your fixes because I was on break unfortunately, but I'm back now.

[CasualYT31]

@mikf Nothing's broken on my end as of yet. Besides the exe issue this PR is looking good from my point of view. I'll keep an eye on #6684. Version 3.9 should be satisfactory for yt-dlp. Besides using a newer version of Python for the executables, do you foresee any other issues with replacing youtube-dl with yt-dlp? Obviously we want to maintain compatibility for both generally, but even without this TikTok extractor, switching to yt-dlp for the binaries seems like a no-brainer considering youtube-dl hasn't seen an official release for a few years now.

[mikf]

The main gallery-dl.exe executables for the next release will be shipping with Python 3.13 and yt-dlp, same as the ones from https://github.com/gdl-org/builds. There will also be a gallery-dl_x86.exe build using the same versions as the current gallery-dl.exe for 1.28.5.

The reason for using youtube-dl is compatibility with py2exe, which doesn't work all that well with yt-dlp + its (optional) dependencies. The new executables will be built with PyInstaller, which will make them start slower, but that's "progress" for you.

[mikf]

Thank you for all your work

[CasualYT31]

Thank you for the work you've put into this program 😄!