Build is broken on ppc64: 32-bit target is forced

[barracuda156]

The build does not work on ppc64: apparently 32-bit target is forced, which, unsurprisingly, fails:

/usr/bin/gcc-4.2 -I/opt/local/include -DPPC32 -Itppc32osx/boot/tppc32osx -Itppc32osx/c -Ic/ -pipe -Os -arch ppc64 -o tppc32osx/c/random.o -c c/random.c
/usr/bin/gcc-4.2 -I/opt/local/include -DPPC32 -Itppc32osx/boot/tppc32osx -Itppc32osx/c -Ic/ -pipe -Os -arch ppc64 -o tppc32osx/c/ffi.o -c c/ffi.c
/usr/bin/gcc-4.2 -I/opt/local/include -DPPC32 -Itppc32osx/boot/tppc32osx -Itppc32osx/c -Ic/ -pipe -Os -arch ppc64 -o tppc32osx/c/self-exe.o -c c/self-exe.c
/usr/bin/gcc-4.2 -I/opt/local/include -DPPC32 -Itppc32osx/boot/tppc32osx -Itppc32osx/c -Ic/ -pipe -Os -arch ppc64 -o tppc32osx/c/ppc32.o -c c/ppc32.c
ar rc tppc32osx/boot/tppc32osx/libkernel.a tppc32osx/c/statics.o tppc32osx/c/segment.o tppc32osx/c/alloc.o tppc32osx/c/symbol.o tppc32osx/c/intern.o tppc32osx/c/gcwrapper.o tppc32osx/c/gc-011.o tppc32osx/c/gc-par.o tppc32osx/c/gc-ocd.o tppc32osx/c/gc-oce.o tppc32osx/c/number.o tppc32osx/c/schsig.o tppc32osx/c/io.o tppc32osx/c/new-io.o tppc32osx/c/print.o tppc32osx/c/fasl.o tppc32osx/c/vfasl.o tppc32osx/c/stats.o tppc32osx/c/foreign.o tppc32osx/c/prim.o tppc32osx/c/prim5.o tppc32osx/c/flushcache.o tppc32osx/c/schlib.o tppc32osx/c/thread.o tppc32osx/c/expeditor.o tppc32osx/c/scheme.o tppc32osx/c/compress-io.o tppc32osx/c/random.o tppc32osx/c/ffi.o tppc32osx/c/self-exe.o tppc32osx/c/ppc32.o
ranlib: file: tppc32osx/boot/tppc32osx/libkernel.a(ffi.o) has no symbols
/usr/bin/gcc-4.2 -pipe -Os -arch ppc64 -o tppc32osx/bin/tppc32osx/scheme tppc32osx/boot/tppc32osx/main.o tppc32osx/boot/tppc32osx/libkernel.a -L/opt/local/lib -Wl,-headerpad_max_install_names -arch ppc64 -liconv -lm -lncurses -lz -llz4
: tppc32osx/bin/tppc32osx/scheme
running tppc32osx/bin/tppc32osx/scheme to build tppc32osx/s/cmacros.so
running tppc32osx/bin/tppc32osx/scheme to build tppc32osx/s/nanopass.so
sizeof(ptr) * 8 [8] != ptr_bits [32]
sizeof(long) * 8 [8] != long_bits [32]
sizeof(size_t) * 8 [8] != size_t_bits [32]
sizeof(ssize_t) * 8 [8] != size_t_bits [32]
sizeof(ptrdiff_t) * 8 [8] != ptrdiff_t_bits [32]
sizeof(ptr) * 8 [8] != ptr_bits [32]
sizeof(long) * 8 [8] != long_bits [32]
sizeof(size_t) * 8 [8] != size_t_bits [32]
sizeof(ssize_t) * 8 [8] != size_t_bits [32]
sizeof(ptrdiff_t) * 8 [8] != ptrdiff_t_bits [32]
failed
 in build-one
 in loop
 in module->hash
make: *** [build] Error 1

[mflatt]

The configure script could be improved for detecting 64-bit PowerPC Mac OS, but the deeper problem is that there's no 64-bit PowerPC back end. You could supply -m=tpb64b to build in interpreted mode.

[barracuda156]

@mflatt Thank you for responding.
There is no such a thing as 64-bit-only macOS here, ability to build for ppc64 depends on OS/SDK version (and for execution the physical cpu must be G5), but it is a choice, since the same system always supports 32-bit ppc as well.
So detection should rely on a chosen build target or arch flags passed to a compiler (again, not supported by compiler, since it may support both ppc and ppc64). As the very minimum, I would expect it to fail at configure time, if ppc64 is used, so that a user does not waste time for the build which is guaranteed to fail down the road.
I can try interpreter mode, that will probably do for my case (I wanted to bootstrap idris2).

[LiberalArtist]

I agree that the configure script should be improved to not incorrectly infer tppc32osx in this case.

Debian distributes Chez Scheme for 64-bit PowerPC using the tpb64l backend, which is interpreted. (Guix does, too, but we have limited CI machines for 64-bit PowerPC, and it looks like our build is currently broken due to some dependency issue.)

since the same system always supports 32-bit ppc as well

I had not realized that 64-bit PowerPC OSes could run 32-bit binaries. From a little reading, it sounds quite complicated, and, crucially, it sounds like the endianess of the 64-bit OS kernel and the 32-bit application binary have to match. The Chez Scheme compiler can generate big-endian 32-bit PowerPC machine code (only), so, if you have an environment that can build and run that, it should work, at least if you specify it explicitly.

---

Tangentially, re:

You could supply -m=tpb64b to build in interpreted mode.

my understanding is that:

POWER8 and POWER9 chips can be run in either big-endian or little-endian mode: config.sub normalizes them as e.g. powerpc64-unknown-linux-gnu and powerpc64le-unknown-linux-gnu. IIUC, this is set in the firmware or something: a given operating system runs in big-endian or little-endian mode, not both, so Debian has ports for both ppc64 (big-endian) and ppc64el (little-endian). (Guix only supports little-endian mode.)

(One source of confusion is that le means “Linux” in Chez Scheme machine types, but “little endian” in config.sub output.)

Also, my impression is that little-endian has won for 64-bit PowerPC. (But really, I know very little about any variant of PowerPC architecture.)

[barracuda156]

I had not realized that 64-bit PowerPC OSes could run 32-bit binaries.

Not sure how that works in Linux; on macOS the kernel is 32-bit, but 10.5.8 supports ppc64 in userland (so either binaries can run).

Also, my impression is that little-endian has won for 64-bit PowerPC

Kinda yes, if you mean that LE is more prevalent, but I think that at least some *BSD systems are exclusively BE on powerpc, including ppc64. (Not 100% of the current status, info could be out-of-date.)
Might also apply to some Linux distros. And I am pretty sure AIX is only BE.

[barracuda156]

@mflatt Looks like -m=tpb64b worked, thank you! At least chez-scheme has built successfully, and apparently compiling idris2 with it works – build is running now, no errors so far, and I think it is being compiled for ppc64.

The configure script could be improved for detecting 64-bit PowerPC Mac OS

If this script is meant

ChezScheme/configure

Lines 237 to 246 in 2f4c2cc

	elif uname -m | egrep 'power|ppc' > /dev/null 2>&1 ; then
	m32=ppc32${unixsuffix}
	tm32=tppc32${unixsuffix}
	m64=ppc64${unixsuffix}
	tm64=tppc64${unixsuffix}
	if uname -m | egrep 'ppc64le' > /dev/null 2>&1 ; then
	pbendian=l
	else
	pbendian=b
	fi

then we cannot use uname -m there (or uname at all), since it returns not a target arch but cpu family of the build machine. On macOS on powerpc that will be Power Macintosh, regardless of whether ppc64 is supported by the CPU (and in any case what we need is target arch, since it is perfectly valid to build for 32-bit ppc on a G5).

[LiberalArtist]

Looks like -m=tpb64b worked, thank you! … and I think it is being compiled for ppc64.

At the risk of being pedantic, I want to be sure you're clear about what is being compiled how in this setup. That may help to elucidate the configure script.

The implementation of Chez Scheme is in two parts: a small kernel is written in C, and the majority of the implementation is written in Scheme. On a fully-supported architecture like x86_64, the C portion is compiled to x86_64 machine code by your C compiler of choice, and the Scheme portion is compiled to x86_64 machine code by a bootstrapping version of Chez Scheme. That's the "normal" scenario from the Chez Scheme point of view.

When you build for one of the "machine types" with pb in the name, the Scheme portion of Chez Scheme is not compiled to machine code at all: Scheme code is compiled to a "portable bytecode" format, and the C kernel implements an interpreter for the bytecode. The plain pb machine type can work on any platform where the C kernel can be built, and it is primarily used for bootstrapping, so that we can check in just one set of pre-built bootfiles. When you configure with -m=tpb64b, you are selecting a threaded build of using bytecode specialized for a 64-bit big-endian processor, which is designed to provide better performance than plain pb on architectures for which Chez lacks a machine-code backend.

So, when you write, "I think it is being compiled for ppc64," the C kernel is certainly being compiled for ppc64, but the Scheme code is being compiled to bytecode, not to machine code of any kind.

But if your platform is Mac OS Leopard on a G5 (and not one of the more complex Linux or BSD scenarios I was trying to figure out), then you definitely do have the option of using the natively supported tppc32osx machine type, which is what I think most users of those machines want. Is there a specific reason you want to build Chez Scheme in 64-bit mode, even though it means not being able to compile Scheme to machine code?

Looking again at your first post, it seems that the problem was that, in lines like:

/usr/bin/gcc-4.2 -I/opt/local/include -DPPC32 -Itppc32osx/boot/tppc32osx -Itppc32osx/c -Ic/ -pipe -Os -arch ppc64 -o tppc32osx/c/random.o -c c/random.c

somehow -arch ppc64 is getting passed to /usr/bin/gcc-4.2. I can't find anything in the configure script itself that would have caused added -arch ppc64 automatically. Did you explicitly configure it as a flag, or might it have come from some configuration in your build environment?

To compile Scheme code to ppc32, you would definitely need to compile the C in Chez Scheme's kernel to ppc32, also, rather than ppc64. The Scheme and C parts of the implementation need to agree on the layout of primitive data types, so it won't work for them to be built with different word sizes.

[barracuda156]

@LiberalArtist Thank you for a detailed explanation. What I meant above was that C parts are compiled for ppc64, since I could see arch flags being used in a coherent manner.

Re why I need ppc64 and not ppc: well, if it is possible to use ppc32 Scheme to compile for ppc64, then I would not particularly care about bitness of Scheme executable itself. (That still would introduce some minor complications for the set-up, since we have to ensure ppc32 version is accepted by every dependent on ppc64.)
If it is not possible, then I need it as ppc64, otherwise nothing which uses Scheme will be able to build for this arch.