Mail: Filter and Forward Mail/Procmail Mail Filter

[iliajie]

Jamie, there is a bug for non English users.

How to reproduce:

1. Go to Usermin/Mail/Filter and Forward Mail
2. Click Add New Mail Filter
3. Conditions: Based on header. Header let's say based on Subject, value that Contains, for example: это просто тест.
4. Hit Save at the bottom.

At first all looks fine but if you go to Procmail Filter than you will see broken chars, which doesn't seem to work either, example:

If I manually edit them in Procmail Mail Filter and hit save, then they are both displayed correctly in both modules.

Could you fix that?

[jcameron]

Seems like a unicode / character set issue. Which language and character set do you have selected in webmin?

[iliajie]

Always UTF-8.

[iliajie]

I'm talking about Usermin.

[jcameron]

Just tested this with Usermin in en.UTF-8 language, but I was able to create a filter with the same text you used just fine.

One warning though - in practice this probably wouldn't work anyway, as email headers can only be in ASCII without some special encoding (that procmail won't match on).

[iliajie]

No, Jamie, you misunderstood. It's displayed correctly in Filter and Forward Mail but in Procmail Mail Filter it's not displayed correctly after adding a rule in Filter and Forward Mail.

To reproduce it add it in Filter and Forward Mail, then go to Procmail Mail Filter and you will see abracadabra.

Match regexp ^From: \�\�\�\�\�\�\�\�\�\�\�\�\�\�\�\�\�\�\�\�\�\�\ \�\�\�\�\�\�\�\�\�\�\�\�\ \�\�\�\�\�\�\�\�\�\�\�\�\�\�\�\�\�\�\�\�\�\�\�\�

In case it's still true and Procmail can't do it, we need to inform a user with an error, explaining it, not just failing silently.

I will test it more and share results. It's hard to believe that it wouldn't work with unicode headers in 2018.

[jcameron]

Ok, I see the cause of this now, and will fix it in the next release. The real issue is that the string written to .procmailrc has a bunch of extra un-necessary backslashes in it.

I'm still not sure if putting unicode in .procmailrc will work though. You should try it out on some real emails by editing .procmailrc directly.

For example, here's the Subject: header from a chinese spam message I received recently :

Subject: [SPAM] =?utf-8?B?NOaciOWfueiureebruaghyDmlrDku7vnu4/nkIbjgIHpg6jpl6jkuLvnrqHnrqHnkIbmioDog70=?=
	=?utf-8?B?5o+Q5Y2H6K6t57uD?=

[iliajie]

Okay, correct. It doesn't work with non-Lating alphabet.

Jamie, could you test the input data for header and in case it contains non-Latin chars, could you automatically add conversion, so

это просто тест

would become:

=?UTF-8?B?0Y3RgtC+INC/0YDQvtGB0YLQviDRgtC10YHRgg==?=

That isn't hard to do?

[jcameron]

That should be possible, although I can see it getting complex if the user enters a regexp that involves unicode characters.

[iliajie]

Maybe just starting from the simple part would be enough, usually having just a string is fine. In case you can't make it for regexp you could return the message, something like Regexp not supported for Unicode characters. It could be whatever as long as it doesn't fail silently.