Skip to content

Conversation

0xfedcafe
Copy link

No description provided.

@github-iron github-iron added the forwarded-to-js-devs This report has been forwarded to Jane Street's internal review system. label Mar 26, 2025
@0xfedcafe
Copy link
Author

0xfedcafe commented Mar 28, 2025

Also, such compile and link check feels a bit strange to me. Should this probably be somehow implemented in the x86_64 check via running the compiled code with executing CPUID instruction and checking the 23rd bit (POPCNT bit)?

Also I ignore 32 bit architectures because its native support was dropped in OCaml 5+, is this fine?

@ManasJayanth
Copy link

For this to work on Windows with esy, I needed the following,

diff --git a/src/discover/discover.ml b/src/discover/discover.ml
index 5ea3fb0..d3c313a 100644
--- a/src/discover/discover.ml
+++ b/src/discover/discover.ml
@@ -17,7 +17,7 @@ let () =
     (fun c ->
        let arch = ocaml_config_var c "architecture" in
        let cc = ocaml_config_var_exn c "native_c_compiler" in
-       let is_gcc =
+       let is_gcc () =
          (Process.run c cc [ "--version" ] ~env:[]).stdout
          |> String.lowercase_ascii
          |> String.starts_with ~prefix:"gcc"
@@ -32,7 +32,7 @@ let () =
               if popcntd <> ""
               then popcntd
               (* needed to prevent crashing, only gcc has this option *)
-              else if is_gcc
+              else if is_gcc ()
               then try_flag c "-mpopcntb"
               else ""
             | "spa" -> try_flag c "-mpopc"

When cc --version is run, I see the following,

(cd _build/default/src && discover\discover.exe -o mpopcnt.sexp)
/usr/lib/gcc/x86_64-w64-mingw32/11/../../../../x86_64-w64-mingw32/bin/ld: cannot find  --version > C:UsersusernameAppDataRoamingnpmnode_modules@esy-nightlyesynode_modules@prometheansacrificeesy-bash.cygwintmpbuild_f4ea40_duneocaml-configuratora07b62stdout-0 2> C:UsersusernameAppDataRoamingnpmnode_modules@esy-nightlyesynode_modules@prometheansacrificeesy-bash.cygwintmpbuild_f4ea40_duneocaml-configuratora07b62stderr-0: File name too long
collect2: error: ld returned 1 exit status
run: "x86_64-w64-mingw32-gcc  -O2 -fno-strict-aliasing -fwrapv -mms-bitfields  -IC:/Users/username/AppData/Roaming/npm/node_modules/@esy-nightly/esy/3/i/esy_zlib-8f8db659/include   " --version
Fatal error: exception Sys_error("C:\\Users\\username\\AppData\\Roaming\\npm\\node_modules\\@esy-nightly\\esy\\node_modules\\@prometheansacrifice\\esy-bash\\.cygwin\\tmp\\build_f4ea40_dune\\ocaml-configuratora07b62\\stdout-0: No such file or directory")

Seem like a bug somewhere in the toolchain where backslashes are used which confuses bash on Windows.

@0xfedcafe
Copy link
Author

For this to work on Windows with esy, I needed the following,

diff --git a/src/discover/discover.ml b/src/discover/discover.ml
index 5ea3fb0..d3c313a 100644
--- a/src/discover/discover.ml
+++ b/src/discover/discover.ml
@@ -17,7 +17,7 @@ let () =
     (fun c ->
        let arch = ocaml_config_var c "architecture" in
        let cc = ocaml_config_var_exn c "native_c_compiler" in
-       let is_gcc =
+       let is_gcc () =
          (Process.run c cc [ "--version" ] ~env:[]).stdout
          |> String.lowercase_ascii
          |> String.starts_with ~prefix:"gcc"
@@ -32,7 +32,7 @@ let () =
               if popcntd <> ""
               then popcntd
               (* needed to prevent crashing, only gcc has this option *)
-              else if is_gcc
+              else if is_gcc ()
               then try_flag c "-mpopcntb"
               else ""
             | "spa" -> try_flag c "-mpopc"

When cc --version is run, I see the following,

(cd _build/default/src && discover\discover.exe -o mpopcnt.sexp)
/usr/lib/gcc/x86_64-w64-mingw32/11/../../../../x86_64-w64-mingw32/bin/ld: cannot find  --version > C:UsersusernameAppDataRoamingnpmnode_modules@esy-nightlyesynode_modules@prometheansacrificeesy-bash.cygwintmpbuild_f4ea40_duneocaml-configuratora07b62stdout-0 2> C:UsersusernameAppDataRoamingnpmnode_modules@esy-nightlyesynode_modules@prometheansacrificeesy-bash.cygwintmpbuild_f4ea40_duneocaml-configuratora07b62stderr-0: File name too long
collect2: error: ld returned 1 exit status
run: "x86_64-w64-mingw32-gcc  -O2 -fno-strict-aliasing -fwrapv -mms-bitfields  -IC:/Users/username/AppData/Roaming/npm/node_modules/@esy-nightly/esy/3/i/esy_zlib-8f8db659/include   " --version
Fatal error: exception Sys_error("C:\\Users\\username\\AppData\\Roaming\\npm\\node_modules\\@esy-nightly\\esy\\node_modules\\@prometheansacrifice\\esy-bash\\.cygwin\\tmp\\build_f4ea40_dune\\ocaml-configuratora07b62\\stdout-0: No such file or directory")

Seem like a bug somewhere in the toolchain where backslashes are used which confuses bash on Windows.

Probably a Windows path length issue (260 char limit) after Filename.quote_command is used the name may exceed 260 chars, plus backslash escaping problems. Windows is likely struggling with long paths when creating temp files for stdout and backslashes are lost during processing of the error string.

Have you tried enabling LongPaths in Windows? Might solve this.

Later the stdout_fn and stderr_fn look correct, so the issue is somewhere deeper - you're right.

Thanks for the suggested changes - I'll apply them.

@ManasJayanth
Copy link

I have both longPathsAware setting in the manifest of the binary and registry value set to 1. The issue here is backslashes getting escaped.

@jakubrpawlowski
Copy link

Does anyone know how this surfaced just now? I just started getting this error, but it looks like this base version has been out for a while.

@ManasJayanth
Copy link

Likely due to an macOS/XCode upgrade.

@vch9 vch9 mentioned this pull request Jun 4, 2025
@vch9
Copy link

vch9 commented Jun 4, 2025

I am facing issue with building the library on Mac, the recent patches have solved most of the issues. However when I try to install it within an homebrew formula it doesn't work for all the recently released versions. However, for example I have for instance created a branch on top of v0.16.4 cherry-picking your commit and I can now compile it.

I can provide steps to reproduce but from what I understand the cc used in my version of homebrew doesn't raise an error on the following:

cc toto.c -mpopcnt

Therefore it considers that -mpopcnt is supported, and it fails later with>

 (cd _build/default/src && /opt/homebrew/Library/Homebrew/shims/mac/super/cc -O2 -fno-strict-aliasing -fwrapv -pthread -pthread -D_FILE_OFFSET_BITS=64 -O2 -fno-strict-aliasing -fwrapv -pthread -pthread -D_LARGEFILE64_SOURCE -mpopcnt -g -I /private/tmp/octez-20250604-89130-6knbws/.opam/5.2.1/lib/ocaml -I /private/tmp/octez-20250604-89130-6knbws/.opam/5.2.1/lib/sexplib0 -I ../compiler-stdlib/src[...]
# clang: error: unsupported option '-mpopcnt' for target 'arm64-apple-darwin24.2.0'

I haven't looked at your patch in details but I just wanted to share that it fixes my problem on Mac

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
forwarded-to-js-devs This report has been forwarded to Jane Street's internal review system.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants