V7.0.0 - feat: New Playlist Class, fixes bot detection, update docstring (#1215)

* fix: unable to find byted acrawler
* fix: playwright timeout sessions with provide opt-out params when creating sessions (#1196)
* timeout: 300s -> 30s
* fix: tiktok returned invalid reponse by bot detection (#1197)
* chore: move wait for load state between mouse moves
* Update docstring for create_session() (#1202)
* Add playlist class (#1207)
* feat: add playlist class
* bump version, fix tests

---------

Co-authored-by: Rahmat Slamet <[email protected]>
Co-authored-by: brandon <[email protected]>
Co-authored-by: Lukas <[email protected]>

Author: 4 people

Diff for: .sphinx/conf.py

@@ -16,7 +16,7 @@
 project = "TikTokAPI"
 copyright = "2023, David Teather"
 author = "David Teather"
-release = "v6.5.2"
+release = "v7.0.0"
 
 # -- General configuration ---------------------------------------------------
 # https://www.sphinx-doc.org/en/main/usage/configuration.html#general-configuration

Diff for: CITATION.cff

@@ -5,5 +5,5 @@ authors:
     orcid: "https://orcid.org/0000-0002-9467-4676"
 title: "TikTokAPI"
 url: "https://github.com/davidteather/tiktok-api"
-version: 6.5.2
-date-released: 2024-08-24
+version: 7.0.0
+date-released: 2025-01-20

Diff for: README.md

@@ -4,11 +4,11 @@ This is an unofficial api wrapper for TikTok.com in python. With this api you ar
 
 [![DOI](https://zenodo.org/badge/188710490.svg)](https://zenodo.org/badge/latestdoi/188710490) [![LinkedIn](https://img.shields.io/badge/LinkedIn-0077B5?style=for-the-badge&logo=linkedin&logoColor=white&style=flat-square)](https://www.linkedin.com/in/davidteather/) [![Sponsor Me](https://img.shields.io/static/v1?label=Sponsor&message=%E2%9D%A4&logo=GitHub)](https://github.com/sponsors/davidteather) [![GitHub release (latest by date)](https://img.shields.io/github/v/release/davidteather/TikTok-Api)](https://github.com/davidteather/TikTok-Api/releases) [![GitHub](https://img.shields.io/github/license/davidteather/TikTok-Api)](https://github.com/davidteather/TikTok-Api/blob/main/LICENSE) [![Downloads](https://pepy.tech/badge/tiktokapi)](https://pypi.org/project/TikTokApi/) ![](https://visitor-badge.laobi.icu/badge?page_id=davidteather.TikTok-Api) [![Support Server](https://img.shields.io/discord/783108952111579166.svg?color=7289da&logo=discord&style=flat-square)](https://discord.gg/yyPhbfma6f)
 
-This api is designed to **retrieve data** TikTok. It **can not be used post or upload** content to TikTok on the behalf of a user. It has **no support any user-authenticated routes**, if you can't access it while being logged out on their website you can't access it here.
+This api is designed to **retrieve data** TikTok. It **can not be used post or upload** content to TikTok on the behalf of a user. It has **no support for any user-authenticated routes**, if you can't access it while being logged out on their website you can't access it here.
 
 ## Sponsors
 
-These sponsors have paid to be placed here and beyond that I do not have any affiliation with them, the TikTokAPI package will always be free and open-source. If you wish to be a sponsor of this project check out my [GitHub sponsors page](https://github.com/sponsors/davidteather).
+These sponsors have paid to be placed here or are my own affiliate links which I may earn a commission from, and beyond that I do not have any affiliation with them. The TikTokAPI package will always be free and open-source. If you wish to be a sponsor of this project check out my [GitHub sponsors page](https://github.com/sponsors/davidteather).
 
 <div align="center">
     <a href="https://tikapi.io/?ref=davidteather" target="_blank">
@@ -33,6 +33,14 @@ These sponsors have paid to be placed here and beyond that I do not have any aff
          <b>TikTok Captcha Solver: </b> Bypass any TikTok captcha in just two lines of code.<br> Scale your TikTok automation and get unblocked with SadCaptcha.
         </div>
     </a>
+    <br>
+    <a href="https://www.webshare.io/?referral_code=3x5812idzzzp" target="_blank">
+        <img src="https://raw.githubusercontent.com/davidteather/TikTok-Api/main/imgs/webshare.png" width="100" alt="TikTok Captcha Solver">
+        <b></b>
+        <div>
+         <b>Cheap, Reliable Proxies: </b> Supercharge your web scraping with fast, reliable proxies. Try 10 free datacenter proxies today!
+        </div>
+    </a>
 </div>
 
 ## Table of Contents
@@ -93,7 +101,8 @@ docker run -v TikTokApi --rm tiktokapi:latest python3 your_script.py
 
 ### Common Issues
 
-Please don't open an issue if you're experiencing one of these just comment if the provided solution do not work for you.
+- **EmptyResponseException** - this means TikTok is blocking the request and detects you're a bot. This can be a problem with your setup or the library itself
+  - you may need a proxy to successfuly scrape TikTok, I've made a [web scraping lesson](https://github.com/davidteather/everything-web-scraping/tree/main/002-proxies) explaining the differences of "tiers" of proxies, I've personally had success with [webshare's residential proxies](https://www.webshare.io/?referral_code=3x5812idzzzp) (affiliate link), but you might have success on their free data center IPs or a cheaper competitor.
 
 - **Browser Has no Attribute** - make sure you ran `python3 -m playwright install`, if your error persists try the [playwright-python](https://github.com/microsoft/playwright-python) quickstart guide and diagnose issues from there.
 
@@ -114,7 +123,7 @@ ms_token = os.environ.get("ms_token", None) # get your own ms_token from your co
 
 async def trending_videos():
     async with TikTokApi() as api:
-        await api.create_sessions(ms_tokens=[ms_token], num_sessions=1, sleep_after=3)
+        await api.create_sessions(ms_tokens=[ms_token], num_sessions=1, sleep_after=3, browser=os.getenv("TIKTOK_BROWSER", "chromium"))
         async for video in api.trending.videos(count=30):
             print(video)
             print(video.as_dict)

Diff for: TikTokApi/api/playlist.py

@@ -0,0 +1,167 @@
+from __future__ import annotations
+from typing import TYPE_CHECKING, ClassVar, Iterator, Optional
+from ..exceptions import InvalidResponseException
+
+if TYPE_CHECKING:
+    from ..tiktok import TikTokApi
+    from .video import Video
+    from .user import User
+
+
+class Playlist:
+    """
+    A TikTok video playlist.
+
+    Example Usage:
+        .. code-block:: python
+
+            playlist = api.playlist(id='7426714779919797038')
+    """
+
+    parent: ClassVar[TikTokApi]
+
+    id: Optional[str]
+    """The ID of the playlist."""
+    name: Optional[str]
+    """The name of the playlist."""
+    video_count: Optional[int]
+    """The video count of the playlist."""
+    creator: Optional[User]
+    """The creator of the playlist."""
+    cover_url: Optional[str]
+    """The cover URL of the playlist."""
+    as_dict: dict
+    """The raw data associated with this Playlist."""
+
+    def __init__(
+        self,
+        id: Optional[str] = None,
+        data: Optional[dict] = None,
+    ):
+        """
+        You must provide the playlist id or playlist data otherwise this
+        will not function correctly.
+        """
+
+        if id is None and data.get("id") is None:
+            raise TypeError("You must provide id parameter.")
+        
+        self.id = id
+
+        if data is not None:
+            self.as_dict = data
+            self.__extract_from_data()
+
+    async def info(self, **kwargs) -> dict:
+        """
+        Returns a dictionary of information associated with this Playlist.
+
+        Returns:
+            dict: A dictionary of information associated with this Playlist.
+
+        Raises:
+            InvalidResponseException: If TikTok returns an invalid response, or one we don't understand.
+
+        Example Usage:
+            .. code-block:: python
+
+                user_data = await api.playlist(id='7426714779919797038').info()
+        """
+
+        id = getattr(self, "id", None)
+        if not id:
+            raise TypeError(
+                "You must provide the playlist id when creating this class to use this method."
+            )
+
+        url_params = {
+            "mixId": id,
+            "msToken": kwargs.get("ms_token"),
+        }
+
+        resp = await self.parent.make_request(
+            url="https://www.tiktok.com/api/mix/detail/",
+            params=url_params,
+            headers=kwargs.get("headers"),
+            session_index=kwargs.get("session_index"),
+        )
+
+        if resp is None:
+            raise InvalidResponseException(resp, "TikTok returned an invalid response.")
+
+        self.as_dict = resp["mixInfo"]
+        self.__extract_from_data()
+        return resp
+    
+    async def videos(self, count=30, cursor=0, **kwargs) -> Iterator[Video]:
+        """
+        Returns an iterator of videos in this User's playlist.
+
+        Returns:
+            Iterator[dict]: An iterator of videos in this User's playlist.
+
+        Raises:
+            InvalidResponseException: If TikTok returns an invalid response, or one we don't understand.
+
+        Example Usage:
+            .. code-block:: python
+
+                playlist_videos = await api.playlist(id='7426714779919797038').videos()
+        """
+        id = getattr(self, "id", None)
+        if id is None or id == "":
+            await self.info(**kwargs)
+
+        found = 0
+        while found < count:
+            params = {
+              "mixId": id,
+              "count": min(count, 30),
+              "cursor": cursor,
+          }
+
+            resp = await self.parent.make_request(
+                url="https://www.tiktok.com/api/mix/item_list/",
+                params=params,
+                headers=kwargs.get("headers"),
+                session_index=kwargs.get("session_index"),
+            )
+
+            if resp is None:
+                raise InvalidResponseException(
+                    resp, "TikTok returned an invalid response."
+                )
+
+            for video in resp.get("itemList", []):
+                yield self.parent.video(data=video)
+                found += 1
+
+            if not resp.get("hasMore", False):
+                return
+
+            cursor = resp.get("cursor")
+
+    def __extract_from_data(self):
+        data = self.as_dict
+        keys = data.keys()
+
+        if "mixInfo" in keys:
+            data = data["mixInfo"]
+
+        self.id = data.get("id", None) or data.get("mixId", None)
+        self.name = data.get("name", None) or data.get("mixName", None)
+        self.video_count = data.get("videoCount", None)
+        self.creator = self.parent.user(data=data.get("creator", {}))
+        self.cover_url = data.get("cover", None)
+
+        if None in [self.id, self.name, self.video_count, self.creator, self.cover_url]:
+            User.parent.logger.error(
+                f"Failed to create Playlist with data: {data}\nwhich has keys {data.keys()}"
+            )
+    
+    def __repr__(self):
+        return self.__str__()
+
+    def __str__(self):
+        id = getattr(self, "id", None)
+        return f"TikTokApi.playlist(id='{id}'')"

Diff for: TikTokApi/api/user.py

@@ -5,6 +5,7 @@
 if TYPE_CHECKING:
     from ..tiktok import TikTokApi
     from .video import Video
+    from .playlist import Playlist
 
 
 class User:
@@ -87,20 +88,21 @@ async def info(self, **kwargs) -> dict:
         self.__extract_from_data()
         return resp
 
-    async def playlists(self, count=20, cursor=0, **kwargs) -> Iterator[dict]:
+    async def playlists(self, count=20, cursor=0, **kwargs) -> Iterator[Playlist]:
         """
-        Returns a dictionary of information associated with this User's playlist.
+        Returns a user's playlists.
 
         Returns:
-            dict: A dictionary of information associated with this User's playlist.
+            async iterator/generator: Yields TikTokApi.playlist objects.
 
         Raises:
             InvalidResponseException: If TikTok returns an invalid response, or one we don't understand.
 
         Example Usage:
             .. code-block:: python
 
-                user_data = await api.user(username='therock').playlist()
+                async for playlist in await api.user(username='therock').playlists():
+                    # do something
         """
 
         sec_uid = getattr(self, "sec_uid", None)
@@ -109,30 +111,30 @@ async def playlists(self, count=20, cursor=0, **kwargs) -> Iterator[dict]:
         found = 0
 
         while found < count:
-          params = {
-              "secUid": sec_uid,
-              "count": 20,
-              "cursor": cursor,
-          }
-
-          resp = await self.parent.make_request(
-              url="https://www.tiktok.com/api/user/playlist",
-              params=params,
-              headers=kwargs.get("headers"),
-              session_index=kwargs.get("session_index"),
-          )
-
-          if resp is None:
-              raise InvalidResponseException(resp, "TikTok returned an invalid response.")
-          
-          for playlist in resp.get("playList", []):
-              yield playlist
-              found += 1
-          
-          if not resp.get("hasMore", False):
-              return
-          
-          cursor = resp.get("cursor")
+            params = {
+                "secUid": self.sec_uid,
+                "count": min(count, 20),
+                "cursor": cursor,
+            }
+
+            resp = await self.parent.make_request(
+                url="https://www.tiktok.com/api/user/playlist",
+                params=params,
+                headers=kwargs.get("headers"),
+                session_index=kwargs.get("session_index"),
+            )
+
+            if resp is None:
+                raise InvalidResponseException(resp, "TikTok returned an invalid response.")
+            
+            for playlist in resp.get("playList", []):
+                yield self.parent.playlist(data=playlist)
+                found += 1
+            
+            if not resp.get("hasMore", False):
+                return
+            
+            cursor = resp.get("cursor")
 
 
     async def videos(self, count=30, cursor=0, **kwargs) -> Iterator[Video]:

Diff for: TikTokApi/stealth/js/navigator_userAgent.py

@@ -1,8 +1,16 @@
 navigator_userAgent = """
 // replace Headless references in default useragent
-const current_ua = navigator.userAgent
+const current_ua = navigator.userAgent;
 Object.defineProperty(Object.getPrototypeOf(navigator), 'userAgent', {
-    get: () => opts.navigator_user_agent || current_ua.replace('HeadlessChrome/', 'Chrome/')
-})
-
+    get: () => {
+        try {
+            if (typeof opts !== 'undefined' && opts.navigator_user_agent) {
+                return opts.navigator_user_agent;
+            }
+        } catch (error) {
+            console.warn('Error accessing opts:', error);
+        }
+        return current_ua.replace('HeadlessChrome/', 'Chrome/');
+    }
+});
 """