0.8.0
-
Security bug fix:
If you use
HttpAuthMiddleware
(i.e. thehttp_user
andhttp_pass
spider attributes) for Splash authentication, any non-Splash request will expose your credentials to the request target. This includesrobots.txt
requests sent by Scrapy when theROBOTSTXT_OBEY
setting is set toTrue
.Use the new
SPLASH_USER
andSPLASH_PASS
settings instead to set your Splash authentication credentials safely. -
Responses now expose the HTTP status code and headers from Splash as
response.splash_response_status
andresponse.splash_response_headers
(#158) -
The
meta
argument passed to thescrapy_splash.request.SplashRequest
constructor is no longer modified (#164) -
Website responses with 400 or 498 as HTTP status code are no longer handled as the equivalent Splash responses (#158)
-
Cookies are no longer sent to Splash itself (#156)
-
scrapy_splash.utils.dict_hash
now also works withobj=None
(225793b) -
Our test suite now includes integration tests (#156) and tests can be run in parallel (6fb8c41)
-
There’s a new ‘Getting help’ section in the
README.rst
file (#161, #162), the documentation aboutSPLASH_SLOT_POLICY
has been improved (#157) and a typo as been fixed (#121)