Skip to content

YAXBPC v2.3.1

Compare
Choose a tag to compare
@dreamer2908 dreamer2908 released this 09 Oct 08:24
· 6 commits to master since this release

After a few hours trying to get a clear answer why xdelta3 has problems with some (not all) unicode filenames in Windows but works perfectly in Linux with any filenames, I found the elephant in the room: xdelta3 doesn't have proper unicode support.

In Linux, Unix, Mac OS, and the like, command-line options (filenames included) are passed to xdelta3 in UTF-8 encoding (byte array). xdelta3 receives them as is, uses them as is to access files, stores them as is in vcdiff file header, loads them from vcdiff file header as is, displays them as is. The filenames are not transformed/encoded/decoded/whatever-ed at all. UTF-8 is an Unicode encoding, supporting all characters. So, everything works fine, unicode or ascii or whatever. Nothing special is required. You can say it just happens to work.

In Windows, command-line options are passed to xdelta3's main (int argc, char **argv) in Microsoft Windows Codepage 1252 (also byte array). xdelta3 also receives/blablah-s them as is. The filenames are not transformed/etc-ed at all. However, cp1252 only supports a small set of character (see here http://www.ascii.ca/cp1252.htm ). All characters in path/filename not supported in cp1252 will become either question marks (?) or equivalent cp1252 character if available (like full-width <>:"/\|?* becomes ascii <>:"/\|?*). Unicode filenames are doomed.

For Unicode support in Windows, xdelta3 needs a new wmain(int, wchar_t**) to receive command-line options as wide char (UTF-16), and then uses the unicode versions of IO API like CreateFileW, etc. This has been requested years ago here jmacd/xdelta#89

It is (not) funny that I have already seen it and then forgotten it completely.
Oh well.

This release brings the following change:

  • Applying the you-know-what workaround whenever there're any non-cp1252 characters, not only <>:"/\|?*, in filenames, to improve Unicode support in Windows.