Releases: iofu728/spider
🏎iofu728/[email protected]
🚀 Enhancement
- Structure the
basic_req
andproxy_req
API which have better scalability. - auto-build db eniverment.
- refactor proxy code
- refactor
bilibili
code, which split to three module,basicBilibili
,loginBilibili
,upBilibili
- increase average GPA calculate.
- refactor
blog/titleview
,brushclass/brushclass
,buildmd/activateArticle
code
- 🐥About Bilibili
- change av->bv.
Due to the Unique identifier of bilibili video change to Bv in 2020.3.23, refer https://www.bilibili.com/blackboard/activity-BV-PC.html.- View: https://api.bilibili.com/x/web-interface/view?cid=&bvid=
- Stat num: http://api.bilibili.com/x/relation/stat?jsonp=jsonp&callback=__jp0&vmid=%d
- Space avs: https://api.bilibili.com/x/space/arc/search?mid=%s&pn=1&ps=50&jsonp=jsonp
- Rank: https://api.bilibili.com/x/web-interface/ranking?rid=%d&day=%d&type=1&arc_type=%d&jsonp=jsonp&callback=__jp1
- playlist
- change av->bv.
- 🦆About Proxy
- Update proxy source and remove gatherproxy.
🏷 Pull Request: 4
#30 🍩Spider v4.0.0-alpha1
#31 🕌 change the modified time
#32 🚠 Change Av -> Bv
#34 🏎 version 3.3.0
👤 Committers: 1
Gunjianpan(@iofu728 )
🐔iofu728/[email protected]
🚀 Enhancement
- 👹fast load configure
- 🍯zimuzu configure
- add OceanBall v2 decoder
- support keyword re
- add proxy explain
- add view abnormal judge
- Mafengwo js decoder
- ctrip hotel detail js decoder
- update some common function
- proxies 4.0
- API update of basic_req & proy_req
- douban spider(movie info, user info, comment info)
- 🐪66ip
- 🦃update document
- finish OceanBall v2 decode
- 🐥About Bilibili (12 commit)
- Support history analysis.
- Auto-update history analysis data.
- add clean CSV script
- fix bilibili API change (about comment)
- reduce the frequency request sending.
- add a max time limit of bsocket.
- 🦆About Proxy
- Update proxy process.
🐛 Bug Fix
- 🙈fix zhihu view
- 🙊fix memory bigbang
- 🎫fix no rank email bug
- fix star check
- fix no comment check
- 👹 fix ctrip hotel Detail
- 🙈fix time
🏷 Pull Request: 9
#18 👹fast load configure
#19 🎫fix no rank email bug
#20 🏋prepare for version 4.0
#21 👹 fix ctrip hotel Detail
#23 🏌prepare for 3.2
#24 🦆prepare v3.2
#25 🙈fix time
#27 🐕prepare for version 3.2.0
#28 🐔update modified time
👤 Committers: 1
Gunjianpan(@iofu728 )
🐕 iofu728/[email protected]
🚀 Enhancement
- add star crontab
- proxy time_str
- support keyword re
- add proxy explain
- add view abnormal judge
🐛 Bug Fix
- fix email check os
- fix star num error
- fix no rank judge
- fix read the gather proxy bug
🏷 Pull Request: 3
#15 👹fix no rank judge
#16 🐤fix some bug for v3.0
#17 🦃pre_version3.1
👤 Committers: 1
Gunjianpan(@iofu728 )
👾iofu728/[email protected]
👘 iofu728/[email protected]
🚀 Enhancement
- proxy pool common Api
- req basic Api
- gather proxy autoload
- some application
- news(google & baidu)
- youdao Note
- alibaba
- alimama
- brush pku class
- zhihu & csdn & jianshu
- zimuzu
- bilibili(http & websocket)
- eastmoney
- shaoq
- support websocket
🐛 Bug Fix
🏷 Pull Request: 7
#6 🎅change readme
#7 🐷update spider
#8 🎅blog backend
#9 🎅blog backend
#10 😏 auto load gather
#11 😏 update new applicaton
#12 🌈 prepare for v3.0
👤 Committers: 1
Gunjianpan(@iofu728 )