gix-command on Unix can use non-POSIX /bin/sh even if POSIX sh is first in PATH

[EliahKagan]

Current behavior 😯

On Unix-like systems, /bin/sh is usually a POSIX-compatible shell, but POSIX does not require that anything at /bin/sh even exist. It requires that sh run a POSIX-compatible shell (so long as PATH has not been changed to a value that prevents this) and that the shell language that interprets commands in system() and popen() calls is POSIX-compatible. But sh need not resolve to /bin/sh.

Because a lot of software assumes /bin/sh is widespread, it usually does exist, even in systems that don't otherwise use traditional locations for executables (#1810). But some Unix-like operating systems have a Bourne-style but substantially POSIX-incompatible sh implementation as /bin/sh. If the system does not also provide a POSIX-compatible sh elsewhere, then there is probably nothing we can or should do about it. More likely on a system used today, though, is that a POSIX-compatible sh does also exist, at another location.

An informative (i.e. official but supplementary and non-binding) section of the POSIX standard on the sh command includes this recommendation:

Applications should note that the standard PATH to the shell cannot be assumed to be either /bin/sh or /usr/bin/sh, and should be determined by interrogation of the PATH returned by getconf PATH , ensuring that the returned pathname is an absolute pathname and not a shell built-in.

For example, to determine the location of the standard sh utility:

command -v sh

On some implementations this might return:

/usr/xpg4/bin/sh

I think, though I am not sure, that in practice today this is mainly important on Solaris and illumos systems.

Solaris

On Solaris 11, /bin/sh is ksh93, which is POSIX-compatible.

In Solaris 10 and earlier, /bin/sh is a legacy Bourne shell implementation, as described in User Environment Feature Changes for Solaris 11.

Solaris 10 remains in extended support until January 2027.

illumos?

For now, I am having trouble finding definitive information about /bin/sh on illumos. It is a family of operating systems, and while I think /bin/sh is the same on all illumos distributions of the same version of illumos, I am not sure about that either. However, OmniOS is an illumos system. On my OmniOS r151050 virtual machine, /bin/sh is POSIX-compatible:

$ ls -l /bin/sh
lrwxrwxrwx   1 root     root           9 Jun 23  2024 /bin/sh -> i86/ksh93

So the issue doesn't affect my OmniOS system. I think there are older supported versions of OmniOS and of other illumos operating systems, and I haven't researched or examined /bin/sh in them.

Expected behavior 🤔

It may be that we cannot change the behavior in a way that produces an overall improvement:

• We could use sh instead of /bin/sh. This moderately decreases consistency with Git, which I suspect should work this way but does not (described below). This would apply to cases where /bin/sh -c ... is used, not for scripts with shebangs, so there is no problem with using the wrong shell when /bin/sh is actually called for. But having a shell command, such as the value of core.sshCommand, be interpreted differently for the same user on the same system in the same environment between Git and gitoxide seems like it might lead to strange problems.
• We could use sh when present and fall back to /bin/sh. This has the same effect as just using sh on most systems, including the decreased consistency with Git and increased difficulty to users who want to configure nontrivial shell commands on systems with non-POSIX /bin/sh that are compatible with both Git and gitoxide. It might cover the case where PATH is unset better than just using sh, though.
• Using /bin/sh first and falling back to sh on a Unix-like system is probably the least worthwhile change, because I don't think any such systems without any /bin/sh are in common use.

We could do any of those just on systems that are known to have non-POSIX /bin/sh. Detecting the system makes things more complicated but probably not excessively so. The tradeoffs still apply on the affected systems, though.

If on some systems downstream builds of Git have a different shell, then that would be a compelling case for covering them separately. Otherwise it may not be worthwhile.

If this is not to be changed, then I think it is worth documenting in gix-command, and in gix_path::env::shell().

(Either a change to the behavior or the documentation seems like it would conflict with #1862 and #1864. I have no objection to such changes being made at any time and I can rebase to solve the conflict. But if I do them, I may wait until after those are done or to include them there.)

Git behavior

The prepare_shell_cmd function in Git calls git_shell_path to decide what shell to use to run commands of the form sh -c ...:

char *git_shell_path(void)
{
#ifndef GIT_WINDOWS_NATIVE
	return xstrdup(SHELL_PATH);
#else
	char *p = locate_in_PATH("sh");
	convert_slashes(p);
	return p;
#endif
}

That is:

• On Windows, the Windows locate_in_PATH function (which is a macro for mingw_locate_in_PATH in another file, and not the Unix-specific locate_in_PATH function in this same file) is called to do a PATH search, then \ separators are changed to /.
• On all systems, including all Unix-like systems (including Cygwin), the compile-time SHELL_PATH constant is used. This could in principle be set differently when compiling on different systems. But unless configured differently at compile-time, it is /bin/sh.

Steps to reproduce 🕹

This can be double-checked by examining the above references. It could be useful to show results of a simple experiment on a system that continues to be used in practice, in a default or otherwise common realistic configuration, wherein /bin/sh exists but is not a POSIX-compatible shell and a separate POSIX-compatible sh exists and is present in a bin or sbin directory listed ahead of /bin in the PATH. In particular, that could help verify any examples or details being considered for inclusion in docstrings. I currently have no such system, but I may be able to set one up at some point.

[Byron]

Thanks a lot for the detailed analysis!

To me it feels that even though gitoxide could be happy to inherit the shortcomings of Git as well, now that the time was already spent to allow for being more portable, we might as well use it and act.

What I understand is that /bin/sh shouldn't be hard-coded, and instead a search in PATH should be done to find a POSIX-compatible version of sh. As gitoxide isn't using/supporting compile-time configuration like Git does, packagers wouldn't be configuring the shell on systems where /bin/sh isn't POSIX compatible, so we'd want it to be correct there as well.

Generally, I'd prefer a solution which is simple, while documenting it's shortcomings sufficiently well to help improve it later should an issue be reported.