Improve handling of 8bit message content and message/global
#4926
Commits on Jul 25, 2024
-
charset: refactor write_mime_param to append_mime_param
Turns out that most uses of this function force a newline before the parameter anyway, so don't require the caller to provide the header prefix length. Signed-off-by: Robert Stepanek <[email protected]>
-
jmap_mail: remove unused jpart field from struct emailpart
Signed-off-by: Robert Stepanek <[email protected]>
-
JMAPEmail: fix email_set_bodystructure test
It passed a non-ascii blob as text/plain content.
-
JMAPEmail: update email_set_encode_plain_text_attachment test
Email/set now sanitizes bare LF in inlined text bodies, rather than keeping it as-is and using quoted-printable encoding.
-
-
JMAPEmail: fix tests creating attachments without type
A couple of tests created email attachments using binary data without specifying the MIME type of the body part. The new safeguards catch this as inappropriate values for the default text/plain MIME type.
-
-
jmap_mail: rewrite JMAP Email to MIME conversion
This updates how Email/set{create} creates MIME message from JMAP Email objects. - Inline plain text is rewritten before conversion regardless if its content is sourced from an EmailBodyValue or Blob. Non-plain text content now never is rewritten, even if it is defined with in a EmailBodyValue. - If the charset property for inline plain text declares the content to be ASCII or UTF-8, then the EmailBodyPart is rejected for invalid characters. - The content transfer encoding for "message" media types is chosen by the media subtype, e.g. "message/rfc822" and "message/global" are handled differently. - Bare carriage returns in inline plain text now get expanded to CRLF, rather than omitted from the body part contents. - Transfer encoding names are written in lowercase, e.g. "8bit" rather than "8BIT". -
-
-
message.c: support encoded message/rfc822 attachments
Up until now the Content-Transfer-Encoding header for message/rfc822 attachments was ignored. That is fine for rfc822 but we want to treat message/rfc822 and message/global equally, and the global subtype does allow its content to be encoded. This patch changes the parse code to not throw away the encoding identifier of a message/rfc822 body part before storing it in cache. It bumps the mailbox minor cache version to indicate this and handles previous versions on read.
-
-
-
-
jmap_mail.c: do not set "User-Agent" header in Email/set{create}
The User-Agent header is not defined for MIME messages, it only is defined for NetNews and HTTP. Use "X-Mailer" instead which at least is mentioned in RFC 2076.
-
-
xapian_wrap.cc: normalize text using libicu's NFKC with CaseFolding
This changes the Xapian backend to normalize any text using NFKC with CaseFolding before indexing and querying. It bumps the Xapian index version to 17. The default stemmer keeps using lowercase Cyrus Search Form. If any of the existing indexes contains version 16 or less, then queries search for any terms that either match the NFKC-normalized term or the unnormalized term.
-
-
message.h: refactor find_msgid from global.h to message_iter_msgid
It really didn't belong to global.h in the first place.
-
index.c: convert all search part text before indexing
Up until now, Cyrus did not convert the contents of header fields and a couple of iCalendar properties to search form. This required us to convert those part contents to lower case in the Xapian wrapper, but we rather want this all proper NFKC-casefolded. Calling charset_convert on these values before sending them to the indexer makes sure all indexed text is in proper form.
-
jmap_mail.c: calculate query costs after search, not before
The query runner might rewrite the search query before running it, so report the query costs after that.
-
-
-
