You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Filing this after terribot's encouragement earlier in the evening:
<+dgw> !how much deprecation?
<+terribot> dgw: 816 deprecations
On a more serious note, we had a look at what's actually useful in the tools.web module. The answer? Not much.
In my fact-based-but-still-opinionated view, the only possibly-useful utilities left are search_urls(), trim_url(), iri_to_uri() (maybe), USER_AGENT, and DEFAULT_HEADERS.
search_urls() is used in core code (but should probably be called find_urls() or extract_urls() instead)
trim_url() is used in search_urls()
iri_to_uri() seems helpful vs. just using encodings.idna because it understands the whole URL and IDN-encodes components of the domain name + urlencodes non-ASCII components of the path
urlencode_non_ascii is probably replaceable with something in stdlib, but I haven't found anything yet
USER_AGENT and DEFAULT_HEADERS are for plugin authors' convenience
DEFAULT_HEADERS should also be better insulated against being modified by plugins (i.e. some module-level equivalent to a property would be great, something that looks like a module attribute but returns a copy of its default-header dictionary)
Or we could go the simple way and replace DEFAULT_HEADERS with a function like get_default_headers() that returns a copy plugins can tamper with without affecting other code
Unless I've missed some other important stuff, that's going to leave a pretty barebones module behind. We might consider moving the remaining bits around (again; sopel.web moved to sopel.tools.web in #1616 / 7.0) but I'd feel weird about having the few bits and bobs left live in the root sopel.tools module.
Plenty of time to think about this stuff, though.
The text was updated successfully, but these errors were encountered:
Filing this after terribot's encouragement earlier in the evening:
On a more serious note, we had a look at what's actually useful in the
tools.web
module. The answer? Not much.In my fact-based-but-still-opinionated view, the only possibly-useful utilities left are
search_urls()
,trim_url()
,iri_to_uri()
(maybe),USER_AGENT
, andDEFAULT_HEADERS
.search_urls()
is used in core code (but should probably be calledfind_urls()
orextract_urls()
instead)trim_url()
is used insearch_urls()
iri_to_uri()
seems helpful vs. just usingencodings.idna
because it understands the whole URL and IDN-encodes components of the domain name + urlencodes non-ASCII components of the pathurlencode_non_ascii
is probably replaceable with something in stdlib, but I haven't found anything yetUSER_AGENT
andDEFAULT_HEADERS
are for plugin authors' convenienceDEFAULT_HEADERS
should also be better insulated against being modified by plugins (i.e. some module-level equivalent to aproperty
would be great, something that looks like a module attribute but returns a copy of its default-header dictionary)DEFAULT_HEADERS
with a function likeget_default_headers()
that returns a copy plugins can tamper with without affecting other codeUnless I've missed some other important stuff, that's going to leave a pretty barebones module behind. We might consider moving the remaining bits around (again;
sopel.web
moved tosopel.tools.web
in #1616 / 7.0) but I'd feel weird about having the few bits and bobs left live in the rootsopel.tools
module.Plenty of time to think about this stuff, though.
The text was updated successfully, but these errors were encountered: