Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[dev] build from top of branch-2.1, set -DORC_PACKAGE_KIND= #86

Draft
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

h-vetinari
Copy link
Member

@conda-forge-admin
Copy link
Contributor

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe/meta.yaml) and found it was in an excellent condition.

@h-vetinari
Copy link
Member Author

@wgtmac, same issue with vcpkg as with conan:

-- ZLIB_HOME: 
CMake Error at cmake_modules/FindZLIB.cmake:80 (message):
  Could not find ZLIB in system search paths.
Call Stack (most recent call first):
  cmake_modules/ThirdpartyToolchain.cmake:216 (find_package)
  CMakeLists.txt:185 (INCLUDE)

-DZSTD_HOME=$PREFIX \
-DProtobuf_ROOT=$PREFIX \
-DPROTOBUF_HOME=$PREFIX \
-DORC_PACKAGE_KIND=vcpkg \
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for trying it! How does $PREFIX directory look like? Are libraries like ZLIB and others have their CMake package config file in it?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems that Arrow has special logic for conda so that XXX_ROOT and XXX_SOURCE will be defined automatically:

if(ARROW_DEPENDENCY_SOURCE STREQUAL "CONDA")
  if(MSVC)
    set(ARROW_PACKAGE_PREFIX "$ENV{CONDA_PREFIX}/Library")
  else()
    set(ARROW_PACKAGE_PREFIX $ENV{CONDA_PREFIX})
  endif()
  set(ARROW_ACTUAL_DEPENDENCY_SOURCE "SYSTEM")
  ...
endif()

and

# For each dependency, set dependency source to global default, if unset
foreach(DEPENDENCY ${ARROW_THIRDPARTY_DEPENDENCIES})
  if("${${DEPENDENCY}_SOURCE}" STREQUAL "")
    set(${DEPENDENCY}_SOURCE ${ARROW_ACTUAL_DEPENDENCY_SOURCE})
    # If no ROOT was supplied and we have a global prefix, use it
    if(NOT ${DEPENDENCY}_ROOT AND ARROW_PACKAGE_PREFIX)
      set(${DEPENDENCY}_ROOT ${ARROW_PACKAGE_PREFIX})
    endif()
  endif()
endforeach()

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How does $PREFIX directory look like? Are libraries like ZLIB and others have their CMake package config file in it?

Yes, $PREFIX contains all of the following (using linux as an example):

## Package Plan ##

  environment location: /home/conda/feedstock_root/build_artifacts/orc_1740564737847/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho


The following NEW packages will be INSTALLED:

    _libgcc_mutex: 0.1-conda_forge             conda-forge
    _openmp_mutex: 4.5-2_gnu                   conda-forge
    libabseil:     20240722.0-cxx17_hbbce691_4 conda-forge
    libgcc:        14.2.0-h767d61c_2           conda-forge
    libgomp:       14.2.0-h767d61c_2           conda-forge
    libprotobuf:   5.28.3-h6128344_1           conda-forge
    libstdcxx:     14.2.0-h8f9b012_2           conda-forge
    libzlib:       1.3.1-hb9d3cd8_2            conda-forge
    lz4-c:         1.10.0-h5888daf_1           conda-forge
    snappy:        1.2.1-h8bd8927_1            conda-forge
    zlib:          1.3.1-hb9d3cd8_2            conda-forge
    zstd:          1.5.7-hb8e6e7a_1            conda-forge

The content of each of these packages can be explored online, here's zstd, for example.

@@ -8,13 +8,13 @@ source:
url: https://archive.apache.org/dist/orc/orc-{{ version }}/orc-{{ version }}.tar.gz
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to specify a github url and a commit tag? If true, I can try it by myself.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, the syntax for this is:

source:
  git_url: https://github.com/apache/orc.git
  git_rev: branch-2.1  # or 0ee235c2d92096155aaaebf87c3daf94be46882e, or ...

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! I will try to fix this on my end but I cannot promise the ETA.

What about releasing Apache ORC 2.1.1 with the current branch-2.1? It means that following workarounds are still needed by conda-forge:

  • Add ZLIB_xxx and ZSTD_xxx variables to ORC.
  • Patching ORC adapter test to Arrow.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about releasing Apache ORC 2.1.1 with the current branch-2.1?

Yes, this is preferable to me

It means that following workarounds are still needed [...]

We can manage for now, and we can still get rid of them later.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, thanks for confirmation!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants