Building on M1 Mac #50

thomaslima · 2023-02-16T15:23:49Z

I wonder if I could borrow your advice for an issue I have building Mac images on my MacBook (M1). I followed these steps:

I installed qt@5, [email protected], and [email protected] from homebrew
I ran ./macbuild/python3HB.py -v 3.9
I added the QMAKE_CXXFLAGS += -fdeclspec line to klayout.pri (for some reason your script didn't patch automatically.
Then I ran the following script:

#!/bin/bash
set -e

export QMAKE_CCACHE=1

# run ./macbuild/python3HB.py -v 3.9

./build4mac.py -p HB39 -r HB32 -q Qt5Brew
./build4mac.py -p HB39 -r HB32 -q Qt5Brew -y

This produced HW-qt5Brew.pkg.macos-Ventura-release-Rhb32Phb39/klayout.app. When I tried to open it, it crashed with the following log dump. Am I doing something wrong? I feel like I forgot some step.

Do you have any idea of what's going on?

-------------------------------------
Translated Report (Full Report Below)
-------------------------------------

Process:               klayout [4163]
Path:                  /Users/USER/*/klayout.app/Contents/MacOS/klayout
Identifier:            de.klayout
Version:               0.28.5 (0.28.5)
Code Type:             ARM-64 (Native)
Parent Process:        launchd [1]
User ID:               8803

Date/Time:             2023-02-16 10:21:42.0242 -0500
OS Version:            macOS 13.0.1 (22A400)
Report Version:        12
Anonymous UUID:        D2931AAD-F46B-72AF-9649-1D8A6B9062D6

Sleep/Wake UUID:       F9256465-18A2-45BA-B26C-EF90F5F8420B

Time Awake Since Boot: 660000 seconds
Time Since Wake:       4203 seconds

System Integrity Protection: enabled

Crashed Thread:        0

Exception Type:        EXC_BAD_ACCESS (SIGKILL (Code Signature Invalid))
Exception Codes:       UNKNOWN_0x32 at 0x0000000100b6c000
Exception Codes:       0x0000000000000032, 0x0000000100b6c000

Termination Reason:    Namespace CODESIGNING, Code 2 Invalid Page

VM Region Info: 0x100b6c000 is in 0x100b6c000-0x101044000;  bytes after start: 0  bytes before end: 5079039
      REGION TYPE                    START - END         [ VSIZE] PRT/MAX SHRMOD  REGION DETAIL
      mapped file                 100b04000-100b6c000    [  416K] r--/rwx SM=COW  ...t_id=9abb3401
--->  mapped file                 100b6c000-101044000    [ 4960K] r--/rwx SM=COW  ...t_id=99848d01
      GAP OF 0x6a9d4000 BYTES
      Stack Guard                 16ba18000-16f21c000    [ 56.0M] ---/rwx SM=NUL  

Thread 0 Crashed:
0   dyld                          	       0x18bcee3b0 dyld3::MachOFile::hasMachOMagic() const + 0
1   dyld                          	       0x18bcc56c4 dyld4::SyscallDelegate::mapFileReadOnly(Diagnostics&, char const*, unsigned long*, dyld4::FileID*, bool*, char*) const + 228
2   dyld                          	       0x18bcc5028 dyld4::SyscallDelegate::withReadOnlyMappedFile(Diagnostics&, char const*, bool, void (void const*, unsigned long, bool, dyld4::FileID const&, char const*) block_pointer) const + 88
3   dyld                          	       0x18bcbe184 dyld4::JustInTimeLoader::makeJustInTimeLoaderDisk(Diagnostics&, dyld4::RuntimeState&, char const*, dyld4::Loader::LoadOptions const&, bool, unsigned int, mach_o::Layout const*) + 208
4   dyld                          	       0x18bcb510c invocation function for block in dyld4::Loader::getLoader(Diagnostics&, dyld4::RuntimeState&, char const*, dyld4::Loader::LoadOptions const&) + 1260
5   dyld                          	       0x18bcb43b4 dyld4::Loader::forEachResolvedAtPathVar(dyld4::RuntimeState&, char const*, dyld4::Loader::LoadOptions const&, dyld4::ProcessConfig::PathOverrides::Type, bool&, void (char const*, dyld4::ProcessConfig::PathOverrides::Type, bool&) block_pointer) + 184
6   dyld                          	       0x18bca29b4 dyld4::ProcessConfig::PathOverrides::forEachPathVariant(char const*, dyld3::Platform, bool, bool, bool&, void (char const*, dyld4::ProcessConfig::PathOverrides::Type, bool&) block_pointer) const + 688
7   dyld                          	       0x18bcb413c dyld4::Loader::forEachPath(Diagnostics&, dyld4::RuntimeState&, char const*, dyld4::Loader::LoadOptions const&, void (char const*, dyld4::ProcessConfig::PathOverrides::Type, bool&) block_pointer) + 248
8   dyld                          	       0x18bcb488c dyld4::Loader::getLoader(Diagnostics&, dyld4::RuntimeState&, char const*, dyld4::Loader::LoadOptions const&) + 584
9   dyld                          	       0x18bcbc174 invocation function for block in dyld4::JustInTimeLoader::loadDependents(Diagnostics&, dyld4::RuntimeState&, dyld4::Loader::LoadOptions const&) + 408
10  dyld                          	       0x18bceeb64 invocation function for block in dyld3::MachOFile::forEachDependentDylib(void (char const*, bool, bool, bool, unsigned int, unsigned int, bool&) block_pointer) const + 148
11  dyld                          	       0x18bc9c2d8 dyld3::MachOFile::forEachLoadCommand(Diagnostics&, void (load_command const*, bool&) block_pointer) const + 296
12  dyld                          	       0x18bcee9ac dyld3::MachOFile::forEachDependentDylib(void (char const*, bool, bool, bool, unsigned int, unsigned int, bool&) block_pointer) const + 172
13  dyld                          	       0x18bcbbec0 dyld4::JustInTimeLoader::loadDependents(Diagnostics&, dyld4::RuntimeState&, dyld4::Loader::LoadOptions const&) + 164
14  dyld                          	       0x18bca0500 dyld4::prepare(dyld4::APIs&, dyld3::MachOAnalyzer const*) + 1236
15  dyld                          	       0x18bc9fdc4 start + 2404


Thread 0 crashed with ARM Thread State (64-bit):
    x0: 0x0000000100b6c000   x1: 0x0000000000000000   x2: 0x000000016fa15af8   x3: 0x0000000000040002
    x4: 0x0000000000000003   x5: 0x0000000000000000   x6: 0x000000016fa15af8   x7: 0x0000000000000b30
    x8: 0x000000016fa15990   x9: 0x000000016fa15998  x10: 0x000000000100000f  x11: 0x0000000000000001
   x12: 0x000000016fa16122  x13: 0x0000000000000000  x14: 0x0000000226f8bd2a  x15: 0x0000000000008000
   x16: 0x000000000000005c  x17: 0x000000018bc9de7c  x18: 0x0000000000000000  x19: 0x000000016fa16020
   x20: 0x0000000000000003  x21: 0x000000016fa15acf  x22: 0x0000000100b6c000  x23: 0x000000016fa15af8
   x24: 0x000000016fa15ad0  x25: 0x000000016fa15af0  x26: 0x000000016fa15f90  x27: 0x0000000000000000
   x28: 0x0000000000000000   fp: 0x000000016fa15ab0   lr: 0x000000018bcc56c4
    sp: 0x000000016fa15990   pc: 0x000000018bcee3b0 cpsr: 0x80001000
   far: 0x0000000100b6c000  esr: 0x92000007 (Data Abort) byte read Translation fault

Binary Images:
       0x18bc9a000 -        0x18bd244bf dyld (*) <de46dd52-4994-3fd8-b4b4-e352a1a19354> /usr/lib/dyld
               0x0 - 0xffffffffffffffff ??? (*) <00000000-0000-0000-0000-000000000000> ???

External Modification Summary:
  Calls made by other processes targeting this process:
    task_for_pid: 0
    thread_create: 0
    thread_set_state: 0
  Calls made by this process:
    task_for_pid: 0
    thread_create: 0
    thread_set_state: 0
  Calls made by all processes on this machine:
    task_for_pid: 0
    thread_create: 0
    thread_set_state: 0

VM Region Summary:
ReadOnly portion of Libraries: Total=763.0M resident=0K(0%) swapped_out_or_unallocated=763.0M(100%)
Writable regions: Total=8832K written=0K(0%) resident=0K(0%) swapped_out=0K(0%) unallocated=8832K(100%)

                                VIRTUAL   REGION 
REGION TYPE                        SIZE    COUNT (non-coalesced) 
===========                     =======  ======= 
STACK GUARD                       56.0M        1 
Stack                             8176K        1 
VM_ALLOCATE (reserved)             144K        1         reserved VM address space (unallocated)
__DATA                              27K        2 
__DATA_CONST                        38K        2 
__DATA_DIRTY                         7K        1 
__LINKEDIT                       762.4M        2 
__TEXT                             636K        2 
dyld private memory                512K        2 
mapped file                       8416K        4 
===========                     =======  ======= 
TOTAL                            836.0M       18 
TOTAL, minus reserved VM space   835.8M       18

The text was updated successfully, but these errors were encountered:

Kazzz-S · 2023-02-16T21:44:24Z

Hi @thomaslima,

I think this is the same M1-specific problem as #48.

@dsus4wang made some research and found a workaround (link).
However, I'm not in a position to test it since I do not own an M1 Mac.

To be honest, I was wondering if you could help regarding this issue.

I've started thinking to buy the one (Mac mini), but very reluctant so far.

Best regards,
Kazzz-S

thomaslima · 2023-02-17T12:10:03Z

Interesting. That sounds related. Thank you. I will investigate and report back.

thomaslima · 2023-02-22T20:38:54Z

This is definitely related to that. Unfortunately, I tried the CMAKE_XCODE_ATTRIBUTE_OTHER_CODE_SIGN_FLAGS workaround you proposed to no avail.

I can confirm that the LW build works fine. So one of the two deployment processes are messing with the code signatures:

macdeployqt
our own calls to install_name_tool (e.g. https://stackoverflow.com/questions/71744856/install-name-tool-errors-on-arm64). I do get the same warnings.

Maybe the fix will be on the lines of https://github.com/supercollider/supercollider/blob/908d88c9bd352e0b12597422c06537023be4b327/editors/sc-ide/CMakeLists.txt#L554

thomaslima · 2023-02-22T21:10:13Z

Lol it's not the Qt, it's our HW embeddings.

I think I found the strategy to fix it. It will be a bit painful. Take a look: https://stackoverflow.com/questions/54149144/cannot-codesign-an-app-with-python-and-node

I'm not sure which certificate we're gonna use to sign klayout.app. But I think that we should have been doing it not only for the M1 macs, but also Intel. I think the only difference here is that M1 Macs aren't going to even open unsigned apps.

Kazzz-S · 2023-02-22T21:42:52Z

Thank you for sharing your time for the investigation.
Yes, the above strategy looks painful.

thomaslima · 2023-02-22T21:54:48Z

Thoughts on compiling a dmg for system python/ruby only and making the other distro installable via homebrew/macports? Maybe this way we don't have to worry? Now system python is 3.8.

…

On Wed, Feb 22, 2023 at 4:43 PM Kazunari Sekigawa ***@***.***> wrote: Thank you for sharing your time for the investigation. Yes, the above strategy looks painful. — Reply to this email directly, view it on GitHub <#50 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAIJZEZKM6Y5HEEVG7ICS33WY2B6NANCNFSM6AAAAAAU6KBKL4> . You are receiving this because you were mentioned.Message ID: ***@***.***>

Kazzz-S · 2023-02-22T22:02:04Z

Thoughts on compiling a dmg for system python/ruby only and making the other distro installable via homebrew/macports? Maybe this way we don't have to worry? Now system python is 3.8.

Yes, that is an idea! I'll investigate on Intel Mac.

thomaslima · 2023-03-16T14:24:03Z

Hello @Kazzz-S this issue not contained to us anymore and some colleagues are reporting it to me. I am tagging a few people here so they're aware of the solution, maybe they can help crafting a script that would do the code signing for us.
@lukasc-ubc
@joamatab
@Hsuan-Tung
@mustafacc

lukasc-ubc · 2023-03-16T15:36:50Z

Thank you Thomas. Indeed, I am getting the same error when I try to compile a HW build on my M1.

Kazzz-S · 2023-03-16T21:41:48Z

Thank you, @thomaslima, for the info and for sharing your time.

thomaslima mentioned this issue Sep 12, 2023

Klayout quits unexpectedly after building from source code on MacBook Air M1, MacOS Ventura 13.5 KLayout/klayout#1481

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Building on M1 Mac #50

Building on M1 Mac #50

thomaslima commented Feb 16, 2023

Kazzz-S commented Feb 16, 2023

thomaslima commented Feb 17, 2023

thomaslima commented Feb 22, 2023

thomaslima commented Feb 22, 2023

Kazzz-S commented Feb 22, 2023

thomaslima commented Feb 22, 2023 via email

Kazzz-S commented Feb 22, 2023

thomaslima commented Mar 16, 2023

lukasc-ubc commented Mar 16, 2023

Kazzz-S commented Mar 16, 2023

Building on M1 Mac #50

Building on M1 Mac #50

Comments

thomaslima commented Feb 16, 2023

Kazzz-S commented Feb 16, 2023

thomaslima commented Feb 17, 2023

thomaslima commented Feb 22, 2023

thomaslima commented Feb 22, 2023

Kazzz-S commented Feb 22, 2023

thomaslima commented Feb 22, 2023 via email

Kazzz-S commented Feb 22, 2023

thomaslima commented Mar 16, 2023

lukasc-ubc commented Mar 16, 2023

Kazzz-S commented Mar 16, 2023