Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix build failure due to patch conflict #1851

Merged
merged 1 commit into from
Jul 18, 2024

Conversation

heemin32
Copy link
Collaborator

@heemin32 heemin32 commented Jul 18, 2024

Description

This PR is a continuation of #1833 to fix build failure when patch is applied more than once.

The previous PR resolved an issue for manual build where it uses git am and the Cmake system skips applying patches by comparing patch id. However the build script which runs in CI build system still fails because it uses git apply which does not commit the patch and we cannot use patch-id to determine if patches are already applied or not.

Therefore introduced a new parameter, build.lib.apply_patches to disable patching for the second run of :cmakeJniLib task inside build script.

The alternative is making the build script to use git am by setting git user.name and user.email. This approach will provide more control when setting user.name and user.email is not desirable.

Related Issues

#1842
Part of opensearch-project/opensearch-build#4771

Check List

  • New functionality includes testing.
  • New functionality has been documented.
  • API changes companion pull request created.
  • Commits are signed per the DCO using --signoff.
  • Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@heemin32
Copy link
Collaborator Author

heemin32 commented Jul 18, 2024

BWC test is failing because OS distribution does not have knn 2.16 which will be fixed once this PR is merged.

ryanbogan
ryanbogan previously approved these changes Jul 18, 2024
}
else {
commandLine 'cmake', '.', "-DKNN_PLUGIN_VERSION=${opensearch_version}", "-DSIMD_ENABLED=${simd_enabled}", "-DCOMMIT_LIB_PATCHES=${commit_lib_patches}"
args.add("-G")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you sure on this?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

of course.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops. Let me check one more time.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems the order of -G option doesn't matter. I put it at the end and it works.

list(APPEND PATCH_FILE_LIST "${CMAKE_CURRENT_SOURCE_DIR}/patches/faiss/0002-Enable-precomp-table-to-be-shared-ivfpq.patch")
list(APPEND PATCH_FILE_LIST "${CMAKE_CURRENT_SOURCE_DIR}/patches/faiss/0003-Custom-patch-to-support-range-search-params.patch")
list(APPEND PATCH_FILE_LIST "${CMAKE_CURRENT_SOURCE_DIR}/patches/faiss/0004-Custom-patch-to-support-binary-vector.patch")
if(NOT DEFINED APPLY_LIB_PATCHES OR "${APPLY_LIB_PATCHES}" STREQUAL true)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we just skip everything for patches?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ack

commandLine 'cmake', '.', "-G", "Unix Makefiles", "-DKNN_PLUGIN_VERSION=${opensearch_version}", "-DBLAS_LIBRARIES=$rootDir\\src\\main\\resources\\windowsDependencies\\libopenblas.dll", "-DLAPACK_LIBRARIES=$rootDir\\src\\main\\resources\\windowsDependencies\\libopenblas.dll", "-DSIMD_ENABLED=${simd_enabled}", "-DCOMMIT_LIB_PATCHES=${commit_lib_patches}"
}
else {
commandLine 'cmake', '.', "-DKNN_PLUGIN_VERSION=${opensearch_version}", "-DSIMD_ENABLED=${simd_enabled}", "-DCOMMIT_LIB_PATCHES=${commit_lib_patches}"
Copy link
Member

@martin-gaievski martin-gaievski Jul 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

did we skip -DKNN_PLUGIN_VERSION=${opensearch_version} intentionally?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See line# 317.

@heemin32 heemin32 merged commit 881364f into opensearch-project:main Jul 18, 2024
19 of 56 checks passed
@heemin32 heemin32 deleted the build branch July 18, 2024 17:12
opensearch-trigger-bot bot pushed a commit that referenced this pull request Jul 18, 2024
Signed-off-by: Heemin Kim <[email protected]>
(cherry picked from commit 881364f)
heemin32 added a commit that referenced this pull request Jul 18, 2024
Signed-off-by: Heemin Kim <[email protected]>
(cherry picked from commit 881364f)

Co-authored-by: Heemin Kim <[email protected]>
naveentatikonda pushed a commit to naveentatikonda/k-NN that referenced this pull request Jul 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants