local install.packages type="source" does not honor verbose settings

[mik3hall]

---

From repo verbose

---

install.packages("fastmatch",quiet=FALSE,verbose=TRUE,keep_output=TRUE)
Installing package into ‘/Users/mjh/Documents/R/fastr’
(as ‘lib’ is unspecified)
system (cmd0): /Users/mjh/HalfPipe/HalfPipe_jpkg/outFastR/FastRGraalHPS.app/Contents/runtime/Contents/Home/languages/R/bin/R CMD INSTALL
Content type 'application/octet-stream' length 14290 bytes (13 KB)
foundpkgs: fastmatch, /var/folders/dh/91wmrk0n6lzfmr4tjhjmcfp40000gn/T/Rtmpl2ral6/downloaded_packages/fastmatch_1.1-0.tar.gz
files: /var/folders/dh/91wmrk0n6lzfmr4tjhjmcfp40000gn/T/Rtmpl2ral6/downloaded_packages/fastmatch_1.1-0.tar.gz

• installing source package ‘fastmatch’ ...
  ** package ‘fastmatch’ successfully unpacked and MD5 sums checked
  ** using staged installation
  ** libs
  "/Users/mjh/HalfPipe/HalfPipe_jpkg/outFastR/FastRGraalHPS.app/Contents/runtime/Contents/Home/languages/llvm/native/bin/graalvm-native-clang" -I"/Users/mjh/HalfPipe/HalfPipe_jpkg/outFastR/FastRGraalHPS.app/Contents/runtime/Contents/Home/languages/R/include" -DNDEBUG -O2 -DFASTR -DNO_GNUR -fPIC -O2 -DFASTR -DNO_GNUR -c ctapply.c -o ctapply.o
  "/Users/mjh/HalfPipe/HalfPipe_jpkg/outFastR/FastRGraalHPS.app/Contents/runtime/Contents/Home/languages/llvm/native/bin/graalvm-native-clang" -I"/Users/mjh/HalfPipe/HalfPipe_jpkg/outFastR/FastRGraalHPS.app/Contents/runtime/Contents/Home/languages/R/include" -DNDEBUG -O2 -DFASTR -DNO_GNUR -fPIC -O2 -DFASTR -DNO_GNUR -c fasthash.c -o fasthash.o
  In file included from fasthash.c:19:
  ./common.h:13:17: error: typedef redefinition with different types ('R_len_t' (aka 'int') vs 'ptrdiff_t' (aka 'long'))
  typedef R_len_t R_xlen_t;
  ^
  /Users/mjh/HalfPipe/HalfPipe_jpkg/outFastR/FastRGraalHPS.app/Contents/runtime/Contents/Home/languages/R/include/Rinternals.h:72:23: note: previous definition is here
  typedef ptrdiff_t R_xlen_t;
  ^
  1 error generated.
  make: *** [fasthash.o] Error 1
  ERROR: compilation failed for package ‘fastmatch’
• removing ‘/Users/mjh/Documents/R/fastr/fastmatch’

The downloaded source packages are in
‘/private/var/folders/dh/91wmrk0n6lzfmr4tjhjmcfp40000gn/T/Rtmpl2ral6/downloaded_packages’
Warning message:
In install.packages("fastmatch", quiet = FALSE, verbose = TRUE, :
installation of package ‘fastmatch’ had non-zero exit status

---

From local no verbose

---

install.packages("/Users/mjh/Downloads/fastmatch_1.1-0.tar.gz",repos=NULL,type="source",quiet=FALSE,verbose=TRUE,keep_output=TRUE)
Installing package into ‘/Users/mjh/Documents/R/fastr’
(as ‘lib’ is unspecified)
system (cmd0): /Users/mjh/HalfPipe/HalfPipe_jpkg/outFastR/FastRGraalHPS.app/Contents/runtime/Contents/Home/languages/R/bin/R CMD INSTALL
Warning message:
In install.packages("/Users/mjh/Downloads/fastmatch_1.1-0.tar.gz", :
installation of package ‘/Users/mjh/Downloads/fastmatch_1.1-0.tar.gz’ had non-zero exit status

---

From OS X R.app local, verbose and works

---

install.packages("/Users/mjh/Downloads/fastmatch_1.1-0.tar.gz",repos=NULL,type="source",quiet=FALSE,verbose=TRUE,keep_output=TRUE)
Installing package into ‘/Users/mjh/Library/R/3.6/library’
(as ‘lib’ is unspecified)
system (cmd0): /Library/Frameworks/R.framework/Resources/bin/R CMD INSTALL

• installing source package ‘fastmatch’ ...
  ** package ‘fastmatch’ successfully unpacked and MD5 sums checked
  ** using staged installation
  ** libs
  clang -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -I/usr/local/include -fPIC -Wall -g -O2 -c ctapply.c -o ctapply.o
  clang -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -I/usr/local/include -fPIC -Wall -g -O2 -c fasthash.c -o fasthash.o
  clang -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -I/usr/local/include -fPIC -Wall -g -O2 -c fastmatch.c -o fastmatch.o
  clang -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/usr/local/lib -o fastmatch.so ctapply.o fasthash.o fastmatch.o -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
  installing to /Users/mjh/Library/R/3.6/library/00LOCK-fastmatch/00new/fastmatch/libs
  ** R
  ** byte-compile and prepare package for lazy loading
  ** help
  *** installing help indices
  ** building package indices
  ** testing if installed package can be loaded from temporary location
  ** checking absolute paths in shared objects and dynamic libraries
  ** testing if installed package can be loaded from final location
  ** testing if installed package keeps a record of temporary installation path
• DONE (fastmatch)
  1): succeeded '/Library/Frameworks/R.framework/Resources/bin/R CMD INSTALL -l '/Users/mjh/Library/R/3.6/library' '/Users/mjh/Downloads/fastmatch_1.1-0.tar.gz''

---

It seems like it should never work. I'm still trying to figure out why it works this way and not FastR.
But I did notice the verbose inconsistency.

[mik3hall]

The actual package error turned out to be because the package included a header file with...

//#ifndef XLENGTH /* for compatibility with old R */
//#define XLENGTH(X) LENGTH(X)
//#define IS_LONG_VEC(X) 0
//typedef R_len_t R_xlen_t;
//#endif

uncommented. I think anytime XLENGTH is set on a 64 bit machine, which it must be for FastR and not for gnu R, you would get a compile error.
I emailed this to the maintainer.
Unfortunately even with this corrected FastR appears to have a linkage error for this package when it tries to fallback to the R match function.

Caused by: Attached Guest Language Frames (0)
Error: package or namespace load failed for ‘fastmatch’ in dyn.load(file, DLLpath = DLLpath, ...):
unable to load shared object '/Users/mjh/Documents/R/fastr/00LOCK-fastmatch/00new/fastmatch/libs/fastmatch.so'
dlopen(/Users/mjh/Documents/R/fastr/00LOCK-fastmatch/00new/fastmatch/libs/fastmatch.so, 6): Symbol not found: _match5
Referenced from: /Users/mjh/Documents/R/fastr/00LOCK-fastmatch/00new/fastmatch/libs/fastmatch.so
Expected in: flat namespace
in /Users/mjh/Documents/R/fastr/00LOCK-fastmatch/00new/fastmatch/libs/fastmatch.so

With this in the code...
/* match5 to fall-back to R's internal match for types we don't support */
SEXP match5(SEXP itable, SEXP ix, int nmatch, SEXP incomp, SEXP env);

For some reason isn't found in FastR?
This package is a dependency for the mlr package. A nice to have for running some R machine learning classifiers. Including from java.

[steve-s]

Hello,

thank you for the report. This is quite interesting. This package contains this C function prototype:

/* match5 to fall-back to R's internal match for types we don't support */
SEXP match5(SEXP itable, SEXP ix, int nmatch, SEXP incomp, SEXP env);

This is not a function exposed in Rinternals.h, hence not a public API for R packages, so the author had to copy and paste the prototype from GNU-R source code and relies on the fact that function of this name will be loaded in the R process.

If we are lucky, this match5 is used in some rare corner cases and then just defining dummy "unimplemented" match5 function in FastR (https://github.com/oracle/fastr/blob/master/com.oracle.truffle.r.native/fficall/src/common/unimplemented.c) will cover us w.r.t. building this package and running code that does not trigger these rare cases.

We can also properly implement match5 like we do implement Rf_match, which is part of the public API exposed to R packages. At this point our backlog is quite large, if you wanted to have it faster, it can be added like we added Rf_asS4 in 41c9b1f and it can probably piggy-back on the Rf_match implementation to some extent.

[mik3hall]

Thank you for the comment. I am glad to hear these issues are being actively looked at and sorry to hear there is a backlog. The Rf_asS4 I'm afraid is beyond my current understanding of C code and the R API's. I trust you will address this as best seems fit for how you prioritize the error and how it fits into what you already have to do.

[steve-s]

Should be fixed in 9d03a7b