Skip to content

Commit

Permalink
Update vendored abseil-cpp (#258)
Browse files Browse the repository at this point in the history
* copy some prior art

* remove previous absl files

* renove abseil files

* add vendor in tools/

* it works!

* fix some initial check issues

* lazier config

* remove r 3.6 on windows

* see if removing the braces helps

* maybe fix substitution

* just link it all

* try again

* helppers

* slight updates

* better flags

* more cmake

* just hard code it

* pkg-config

* add gnu make to makefile

* don't link rt

* ignore windows dir

* possible progress for R 4.2

* more win checks

* don't use configure.win

* maybe working on R 4.2

* also use cxx17

* check for absl

* maybe with quotes

* version and news

* maybe the right flags

* maybe try unifying makevars

* maybe work on R 4.2/4/1

* maybe fix makevars.win

* maybe fix for R 4.2

* maybe fix R 4.1

* see if we're getting the right if statement

* temp fix for old s2 warnings

* maybe fix again

* try again

* document configure.win

* maybe fix R 4.2 warning

* try again for globals

* maybe use different namespace

* use pkg-config abseil

* fix comment

* maybe fix

* quality of life improvements

* clean news

* better cleanup/configure

* reasonable path to not awful dev setup without system abseil

* enforce minimum version in pkg-config check
  • Loading branch information
paleolimbot authored Jun 17, 2024
1 parent ff89074 commit 29c11b4
Show file tree
Hide file tree
Showing 660 changed files with 77,346 additions and 9,139 deletions.
4 changes: 4 additions & 0 deletions .Rbuildignore
Original file line number Diff line number Diff line change
Expand Up @@ -12,3 +12,7 @@
^README\.Rmd$
^vignettes/articles$
^\.clang-format$
^tools/build$
^tools/dist$
^compile_commands\.json$
^\.cache$
2 changes: 2 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@

tools/vendor/abseil-cpp linguist-vendored
3 changes: 2 additions & 1 deletion .github/workflows/check-standard.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,9 @@ jobs:
matrix:
config:
- {os: macOS-latest, r: 'release'}
- {os: windows-latest, r: '3.6'}
- {os: windows-latest, r: '4.1'}
- {os: windows-latest, r: '4.2'}
- {os: windows-latest, r: '4.3'}
- {os: windows-latest, r: 'release'}
- {os: ubuntu-latest, r: 'devel', http-user-agent: 'release'}
- {os: ubuntu-latest, r: 'release'}
Expand Down
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,6 @@
.vscode
docs
configure.log
.cache
compile_commands.json
windows
14 changes: 7 additions & 7 deletions DESCRIPTION
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
Package: s2
Title: Spherical Geometry Operators Using the S2 Geometry Library
Version: 1.1.6
Version: 1.1.6.9000
Authors@R: c(
person(given = "Dewey",
family = "Dunnington",
Expand All @@ -19,26 +19,26 @@ Authors@R: c(
Description: Provides R bindings for Google's s2 library for geometric calculations on
the sphere. High-performance constructors and exporters provide high compatibility
with existing spatial packages, transformers construct new geometries from existing
geometries, predicates provide a means to select geometries based on spatial
geometries, predicates provide a means to select geometries based on spatial
relationships, and accessors extract information about geometries.
License: Apache License (== 2.0)
Encoding: UTF-8
LazyData: true
Roxygen: list(markdown = TRUE)
RoxygenNote: 7.2.3
SystemRequirements: OpenSSL >= 1.0.1
LinkingTo:
SystemRequirements: OpenSSL >= 1.0.1, Abseil >= 20230802.0
LinkingTo:
Rcpp,
wk
Imports:
Imports:
Rcpp,
wk (>= 0.6.0)
Suggests:
Suggests:
bit64,
testthat (>= 3.0.0),
vctrs
URL: https://r-spatial.github.io/s2/, https://github.com/r-spatial/s2, http://s2geometry.io/
BugReports: https://github.com/r-spatial/s2/issues
Depends:
Depends:
R (>= 3.0.0)
Config/testthat/edition: 3
10 changes: 10 additions & 0 deletions NEWS.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,13 @@
# s2 (development version)

* The Abseil dependency is resolved using pkg-config where possible.
Where this is not possible, a vendored version of Abseil will be built using
CMake (#258).

# s2 1.1.6

* Fix CRAN warning (#254).

# s2 1.1.5

* fix compiler problem on Alpine 3.19.0 (#251)
Expand Down
6 changes: 5 additions & 1 deletion cleanup
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
#!/bin/sh
rm -f src/Makevars configure.log autobrew
rm `find src -name *.o`
rm `find src -name "*.o"`
rm `find src -name "*.a"`
rm `find src -name "*.tmp"`
rm -rf tools/build
rm -rf tools/dist
46 changes: 45 additions & 1 deletion configure
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,51 @@ else
PKG_CFLAGS="$PKG_CFLAGS -DIS_BIG_ENDIAN"
fi

# Check pkg-config for abseil-cpp, but fall back to a cmake build.
# This should possibly be updated to check that the pkg-config detected
# version will actually work; however, the version of abseil that included
# pkg-config files is about the same as the minimum version required here.
export PKG_CONFIG_PATH="${PKG_CONFIG_PATH}:`pwd`/tools/pkgconfig"
echo "** Using PKG_CONFIG_PATH=${PKG_CONFIG_PATH}"

if [ -z "${R_S2_ABSL_HOME}" ] && pkg-config absl_s2 --libs 2>/dev/null; then
echo "** Using abseil-cpp from pkg-config"

PKGCONFIG_CFLAGS=`pkg-config --cflags-only-I absl_s2`
PKGCONFIG_LIBS=`pkg-config --libs absl_s2`
PKG_CFLAGS="${PKGCONFIG_CFLAGS} ${PKG_CFLAGS}"
PKG_LIBS="${PKGCONFIG_LIBS} ${PKG_LIBS}"
else
# Allow developers to set R_S2_ABSL_HOME in .Renviron to avoid building
# Abseil on every package rebuild. This can be built from this directory using
# tools/build_absl.sh path/to/dir (then R_S2_ABSL_HOME=path/to/dir in .Renviron)
if [ -z "$R_S2_ABSL_HOME" ]; then
echo "** Building abseil-cpp using cmake"
CMAKE_INSTALL_PREFIX="`pwd`/tools/dist"
if tools/build_absl.sh "${CMAKE_INSTALL_PREFIX}"; then
echo "** Done!"
else
echo "** CMake build of Abseil failed"
echo "** Abseil can be installed with:"
echo "** - apt-get install libabsl-dev"
echo "** - dnf install abseil-cpp-devel"
echo "** - brew install abseil"
echo "** If a system install of Abseil is not possible, cmake is required to build"
echo "** the internal vendored copy."
exit 1
fi

# Clean up build directory
rm -rf tools/build

R_S2_ABSL_HOME="`pwd`/tools/dist"
fi

ABSL_LIBS="-labsl_cord -labsl_cordz_info -labsl_cord_internal -labsl_cordz_functions -labsl_cordz_handle -labsl_crc_cord_state -labsl_crc32c -labsl_crc_internal -labsl_crc_cpu_detect -labsl_raw_hash_set -labsl_hashtablez_sampler -labsl_exponential_biased -labsl_synchronization -labsl_graphcycles_internal -labsl_kernel_timeout_internal -labsl_stacktrace -labsl_symbolize -labsl_malloc_internal -labsl_time -labsl_civil_time -labsl_time_zone -labsl_hash -labsl_city -labsl_bad_optional_access -labsl_bad_variant_access -labsl_low_level_hash -labsl_str_format_internal -labsl_strings -labsl_string_view -labsl_strings_internal -labsl_int128 -labsl_throw_delegate -labsl_demangle_internal -labsl_base -labsl_spinlock_wait -labsl_debugging_internal -labsl_raw_logging_internal -labsl_log_severity"
PKG_CFLAGS="-I${R_S2_ABSL_HOME}/include ${PKG_CFLAGS}"
PKG_LIBS="-L${R_S2_ABSL_HOME}/lib ${ABSL_LIBS} ${PKG_LIBS}"
fi

# From apache/arrow/r/configure:
# If on Raspberry Pi, need to manually link against latomic
# See: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81358 for similar example
Expand All @@ -117,7 +162,6 @@ fi
echo "Using PKG_LIBS=$PKG_LIBS"
echo "Using PKG_CFLAGS=$PKG_CFLAGS"


# Write to Makevars
sed -e "s|@cflags@|$PKG_CFLAGS|" -e "s|@libs@|$PKG_LIBS|" src/Makevars.in > src/Makevars

Expand Down
21 changes: 21 additions & 0 deletions configure.win
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@

# On R 4.3 and 4.4, Abseil can be resolved using pkg-config. Otherwise,
# build a copy of Abseil-cpp using CMake.
if pkg-config absl_base --libs 2>/dev/null; then
echo "Using Abseil from RTools via pkg-config"
else
# Build the libraries
CMAKE_INSTALL_PREFIX="`pwd`/tools/dist"
WIN_CPPFLAGS="-DABSL_FORCE_WAITER_MODE=4" tools/build_absl.sh "${CMAKE_INSTALL_PREFIX}"

# Remove the build directory (its existence causes a check warning on R 4.2)
rm -rf tools/build
fi

# On R 4.3 and 4.4, OpenSSL can be resolved using pkg-config. Otherwise,
# Use rwinlibs to download a static library for that toolchain.
if pkg-config openssl --libs 2>/dev/null; then
echo "Using openssl from RTools via pkg-config"
else
"${R_HOME}/bin/Rscript.exe" "tools/winlibs.R"
fi
184 changes: 24 additions & 160 deletions data-raw/update-absl.R
Original file line number Diff line number Diff line change
@@ -1,166 +1,30 @@

library(tidyverse)
tag <- "20220623.1"
tag <- "20230802.2"

# download Abseil
source_url <- glue::glue("https://github.com/abseil/abseil-cpp/archive/refs/tags/{tag}.zip")
curl::curl_download(source_url, "data-raw/abseil-cpp-source.zip")
unzip("data-raw/abseil-cpp-source.zip", exdir = "data-raw")


absl_copy <- function(src, dst) {
src_files <- list.files(src, "\\.(cc|h|inc)$", recursive = TRUE) %>%
str_subset("_test(ing)?\\.(cc|h)$", negate = TRUE) %>%
str_subset("test_", negate = TRUE) %>%
str_subset("_benchmark", negate = TRUE)

dst_files <- file.path(dst, src_files)
dst_dirs <- unique(dirname(dst_files))
for (d in sort(dst_dirs)) {
if (!dir.exists(d)) dir.create(d, recursive = TRUE)
}

stopifnot(all(file.copy(file.path(src, src_files), dst_files)))
}

unlink("src/absl", recursive = TRUE)

absl_copy(
glue::glue("data-raw/abseil-cpp-{tag}/absl/container"),
"src/absl/container"
)

absl_copy(
glue::glue("data-raw/abseil-cpp-{tag}/absl/base"),
"src/absl/base"
)

absl_copy(
glue::glue("data-raw/abseil-cpp-{tag}/absl/meta"),
"src/absl/meta"
)

absl_copy(
glue::glue("data-raw/abseil-cpp-{tag}/absl/synchronization"),
"src/absl/synchronization"
)

absl_copy(
glue::glue("data-raw/abseil-cpp-{tag}/absl/time"),
"src/absl/time"
)

absl_copy(
glue::glue("data-raw/abseil-cpp-{tag}/absl/strings"),
"src/absl/strings"
)

absl_copy(
glue::glue("data-raw/abseil-cpp-{tag}/absl/utility"),
"src/absl/utility"
)

absl_copy(
glue::glue("data-raw/abseil-cpp-{tag}/absl/debugging"),
"src/absl/debugging"
)

absl_copy(
glue::glue("data-raw/abseil-cpp-{tag}/absl/memory"),
"src/absl/memory"
)

absl_copy(
glue::glue("data-raw/abseil-cpp-{tag}/absl/types"),
"src/absl/types"
)

absl_copy(
glue::glue("data-raw/abseil-cpp-{tag}/absl/numeric"),
"src/absl/numeric"
)

absl_copy(
glue::glue("data-raw/abseil-cpp-{tag}/absl/algorithm"),
"src/absl/algorithm"
)

absl_copy(
glue::glue("data-raw/abseil-cpp-{tag}/absl/functional"),
"src/absl/functional"
)

absl_copy(
glue::glue("data-raw/abseil-cpp-{tag}/absl/profiling"),
"src/absl/profiling"
)

absl_objects <- list.files("src/absl", ".cc$", recursive = TRUE) %>%
file.path("absl", .) %>%
str_subset("\\.cc$") %>%
str_replace("\\.cc$", ".o") %>%
paste0(collapse = " \\\n ") %>%
paste0("ABSL_LIBS = ", .)

clipr::write_clip(absl_objects)
usethis::edit_file("src/Makevars.win")
usethis::edit_file("src/Makevars.in")

# Edits needed to make CMD check happy

# Pragmas
fix_pragmas <- function(f) {
content <- readr::read_file(f)
content <- stringr::str_replace_all(content, "\n#pragma", "\n// #pragma")
readr::write_file(content, f)
}

fix_pragmas("src/absl/base/internal/invoke.h")
fix_pragmas("src/absl/container/inlined_vector.h")
fix_pragmas("src/absl/container/internal/inlined_vector.h")
fix_pragmas("src/absl/functional/internal/any_invocable.h")
fix_pragmas("src/absl/types/internal/optional.h")
fix_pragmas("src/absl/container/internal/counting_allocator.h")

# Aborts
fix_aborts <- function(f) {
content <- readr::read_file(f)
content <- stringr::str_replace_all(content, fixed("abort()"), "throw std::runtime_error(\"abort()\")")
readr::write_file(content, f)
}

fix_aborts("src/absl/base/internal/raw_logging.cc")
fix_aborts("src/absl/base/internal/sysinfo.cc")
fix_aborts("src/absl/debugging/symbolize_elf.inc")

# Manual updates

# The symbolizer implementation causes some trouble. We don't use this feature here
# and there seems to be a way to turn it off completely. Do this.
usethis::edit_file("src/absl/debugging/symbolize.cc")

# On Windows, R.h defines a macro 'Free', which we have to undefine
usethis::edit_file("src/absl/base/internal/low_level_alloc.h")

# On Windows with rtools35 (i.e., very old GCC with incomplete C++11), a reference
# to std::get_time() causes compilation error. We don't need strptime here, so just
# return nullptr in this function.
usethis::edit_file("src/absl/time/internal/cctz/src/time_zone_format.cc")

# Windows builds have some additional issues with format strings. These are all within
# absl logger functions...just remove the definition of ABSL_RAW_LOG(...).
usethis::edit_file("src/absl/base/internal/raw_logging.h")

# Fix a workaround for older gcc that causes a check warning. The bug that the
# workaround is addressing only applies to old gcc, so only use that bit of code
# for old gcc
usethis::edit_file("src/absl/container/internal/raw_hash_set.h")

# CRAN compiles with -Wpedantic, so we can't use the __int128 intrinsic type
# undefine ABSL_HAVE_INTRINSIC_INT128 here:
usethis::edit_file("src/absl/base/config.h")

# The use of ABSL_HAVE_CPP_ATTRIBUTE() with ABSL_FALLTHROUGH_INTENDED
# here uses C++17 attributes even if -std=c++17 is not set,
# which causes CRAN warnings with -Wpedantic
usethis::edit_file("src/absl/base/attributes.h")
unzip("data-raw/abseil-cpp-source.zip", exdir = "tools/vendor")
file.rename(glue::glue("tools/vendor/abseil-cpp-{tag}"), "tools/vendor/abseil-cpp")

# prune unused components
unlink("tools/vendor/abseil-cpp/.github", recursive = TRUE)
unlink("tools/vendor/abseil-cpp/.clang-format")
unlink("tools/vendor/abseil-cpp/ci", recursive = TRUE)
unlink(list.files("tools/vendor/abseil-cpp", "\\.py$", full.names = TRUE))
unlink(
list.files(
"tools/vendor/abseil-cpp/absl", "_test.cc$",
full.names = TRUE,
recursive = TRUE
)
)
unlink(
list.files(
"tools/vendor/abseil-cpp/absl", "_benchmark.cc$",
full.names = TRUE,
recursive = TRUE
)
)
unlink("tools/vendor/abseil-cpp/absl/time/internal/cctz/testdata", recursive = TRUE)
1 change: 1 addition & 0 deletions src/.gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,4 @@
*.so
*.dll
Makevars
s2/libs2static.a
Loading

0 comments on commit 29c11b4

Please sign in to comment.