-
Notifications
You must be signed in to change notification settings - Fork 139
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
storyseedling.com no content because adding protection #1560
Comments
Seems they add encryption using custom font. Idk how they do that, because copying text directly resulted in chinese text and alphanumeric words. |
They seems to be doing character replacement for the content in the back end and just use a custom font that's generated directly from code to render everything in English. All the characters of the content is just a character shift from alphanumeric into some random unicode character. I think there is only a few options left to scrape this site:
|
It seems like it's just a character swap, similar to second life translations, just not using English characters. a:⽜ |
Yeah, It's a very simple character swap scheme. However, they can update the character swap mapping anytime and dynamically for every document too. That's why in the long run it would be safer to do a cryptographic decoding or OCR approach. One thing that can help with decoding is the HTML meta tag. I notice in all the chapters they have at least a few sentences of the content available in normal English for SEO purpose. Those sentences are then encoded in the full content part. For simple character swap cypher like that, it would eliminate quite a few characters in advance. |
They're not just doing character replacement. The content isn't on the initial page that is downloaded. Instead a second call is made to get the content. e.g. For first chapter with URL of https://storyseedling.com/series/177491/v1/1/, content is obtained with a POST to https://storyseedling.com/series/177491/v1/1/content Unfortunately, I'm having some problems reproducing the call. Time taken: 42 minutes |
I apologise if I maybe oversimplified this issue, but, given the blank output Webtoepub now generates (due to the new protection scheme), would it be possible to be able to still extract the encrypted source content (that presently has the Chinese characters and other html miscellany in the html data), and then use an in-built script in ePubEditor to decrypt the character swap scheme using the cryptographic decoding method a user above mentioned so any new changes to the site can still possibly be undone (like add an additional button for Story Seedling, the same as Chrysanthemum Garden)? Since the extension wouldn't be attempting to directly bypass these protections and therefore not violate Chrome policy. Just putting my two cents in hope there's some good workaround to still extract the source content from this site, since it has such a wide selection of novels... 😭🥺😢 |
According to my analysis of their website script. They are using cloudflare turnstile as the captcha method for the content call. Instead of doing a fetch directly, this might require a slow full page load and extract the content after captcha have been cleared. |
Also came for a resolution to this site no longer working (but didn't want to open a duplicate issue)... TT_TT |
Bug:
Storyseedling seems update the site. Chapters are need to load a few secs for english content to appear. And before english words seems they load chinese(?) content.
To Reproduce
Steps to reproduce the behavior:
sample TOC: https://storyseedling.com/series/177491/
sample chapter: https://storyseedling.com/series/177491/v1/1/
When finished, you cant find content of chapter.
Expected behavior
Containing content of chapter.
Desktop:
Smartphone:
The text was updated successfully, but these errors were encountered: