Support persisting cookies from multiple domains

[ducaale]

This can be useful when reusing anonymous sessions for multiple domains or when cookies are set as users are redirected from one domain to another.

Before:

{
  "__meta__": { "about": "xh session file", "xh": "0.0.0" },
  "auth": { "type": null, "raw_auth": null },
  "cookies": {
    "lang": { "value": "en" }
  },
  "headers": []
}

After:

{
  "__meta__": { "about": "xh session file", "xh": "0.0.0" },
  "auth": { "type": null, "raw_auth": null },
  "cookies": [
    { "name": "lang", "value": "en", "domain": "example.com" },
    { "name": "lang", "value": "ar", "domain": "example.org" }
  ],
  "headers": []
}

Resolves #244

[blyxxyz]

Haven't done a functional test yet but the code looks good!

[blyxxyz]

For https://www.youtube.com, HTTPie stores the cookies under the domain .youtube.com rather than youtube.com. (Cookies for https://www.wikipedia.org do end up under wikipedia.org.)

I haven't read the spec but I assume these have different meanings that we need to keep?

(Reusing an xh session file in HTTPie causes one of the three YouTube cookies to be stored twice, once with the dot and once without. Then HTTPie will also send it twice, but xh only once.)

[ducaale]

The cookie crate is where the leading dot is removed from the domain. See rwf2/cookie-rs#73

I was initially tempted to declare it as an HTTPie bug, but the behaviour is consistent with Requests' cookie jar

Python 3.10.12 (main, Jun 11 2023, 05:26:28) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
>>> import http.cookies
>>>
>>> cookie_jar = requests.cookies.RequestsCookieJar()
>>> cookie_jar.update(http.cookies.SimpleCookie('hello=world; Path=/; HttpOnly; Secure; Domain=example.com;'))
>>> cookie_jar.update(http.cookies.SimpleCookie('hello=world; Path=/; HttpOnly; Secure; Domain=.example.com;'))
>>> cookie_jar
<RequestsCookieJar[Cookie(version=0, name='hello', value='world', port=None, port_specified=False, domain='.example.com', domain_specified=True, domain_initial_dot=True, path='/', path_specified=True, secure=True, expires=None, discard=False, comment='', comment_url=False, rest={'HttpOnly': True}, rfc2109=False), Cookie(version=0, name='hello', value='world', port=None, port_specified=False, domain='example.com', domain_specified=True, domain_initial_dot=False, path='/', path_specified=True, secure=True, expires=None, discard=False, comment='', comment_url=False, rest={'HttpOnly': True}, rfc2109=False)]>

[blyxxyz]

Hmmm. Both Firefox and Chromium seem to remember (in the dev tools and in the cookies sqlite db) whether there was a leading dot, even if they presumably ignore it in the end.

Is there an easy way for us to remember it too?

[ducaale]

I can't think of a way that doesn't involve forking cookie-rs and cookie-store.

A somewhat random question; do you go with git-rebase or git-merge when bringing changes back from master to a feature branch?

[blyxxyz]

Yeah, I don't see a way either. Let's leave this as it is.

I think it'd be good to document this divergence from HTTPie somewhere, though it's too obscure to go in the README. ("Formatted output is always UTF-8" also doesn't really belong there I think, it needs a longer explanation to clarify what it even means.) Maybe a separate markdown file?

do you go with git-rebase or git-merge when bringing changes back from master to a feature branch?

I usually don't rebase if I already made a PR or if I touched code/logic that changed in master. Otherwise things can get messy.

[ducaale]

I think it'd be good to document this divergence from HTTPie somewhere, though it's too obscure to go in the README. ("Formatted output is always UTF-8" also doesn't really belong there I think, it needs a longer explanation to clarify what it even means.) Maybe a separate markdown file?

Shall we track it as an issue for now? I have now documented couple of existing differences in #322