Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch to vectorscan ✨ #3

Merged
merged 2 commits into from
Jan 9, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .vscode/extensions.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
{
"recommendations": [
"bierner.github-markdown-preview",
"yahyabatulu.vscode-markdown-alert",
"ms-azuretools.vscode-docker",
]
}
39 changes: 21 additions & 18 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,14 +1,15 @@
ARG POLICY=manylinux2014
ARG PLATFORM=x86_64
ARG TAG=2023-11-13-f6b0c51
ARG TAG=2024-01-08-eb135ed

ARG DEVTOOLSET_ROOTPATH=/opt/rh/gcc-toolset-12/root
ARG LD_LIBRARY_PATH_ARG=${DEVTOOLSET_ROOTPATH}/usr/lib64:${DEVTOOLSET_ROOTPATH}/usr/lib:${DEVTOOLSET_ROOTPATH}/usr/lib64/dyninst:${DEVTOOLSET_ROOTPATH}/usr/lib/dyninst
ARG PREPEND_PATH=${DEVTOOLSET_ROOTPATH}/usr/bin:

ARG boost_version=1.57.0
ARG build_type=Release
ARG hyperscan_version=v5.4.2
ARG hyperscan_git_source=https://github.com/VectorCamp/vectorscan
ARG hyperscan_git_tag=vectorscan/5.4.11
ARG pcre_version=8.45
ARG ragel_version=6.10

Expand All @@ -25,38 +26,40 @@ RUN wget -qO- https://www.colm.net/files/ragel/ragel-${ragel_version}.tar.gz | t
WORKDIR /tmp/ragel-${ragel_version}
RUN ./configure --prefix=/usr && make -j$(nproc) && make install

FROM base_ragel as base_hyperscan
FROM base_ragel as base_vectorscan
ARG boost_version
ARG hyperscan_version
ARG hyperscan_git_source
ARG hyperscan_git_tag
WORKDIR /tmp
RUN git clone -b ${hyperscan_version} https://github.com/01org/hyperscan.git
RUN git clone -b ${hyperscan_git_tag} ${hyperscan_git_source}
RUN wget -qO- http://downloads.sourceforge.net/project/boost/boost/${boost_version}/boost_$(echo "${boost_version}" | tr . _).tar.bz2 | tar xj
RUN mv boost*/boost hyperscan/include
RUN mv boost*/boost vectorscan/include

FROM base_hyperscan as build_pcre
FROM base_vectorscan as build_pcre
ARG pcre_version
ENV CFLAGS="-fPIC"
WORKDIR /tmp/hyperscan
WORKDIR /tmp/vectorscan
RUN wget -qO- https://sourceforge.net/projects/pcre/files/pcre/${pcre_version}/pcre-${pcre_version}.tar.gz/download | tar xvz
RUN mv pcre-${pcre_version} pcre
WORKDIR /tmp/hyperscan/pcre
WORKDIR /tmp/vectorscan/pcre
RUN ./configure --prefix=/opt/pcre --enable-unicode-properties --enable-utf
RUN make -j$(nproc) && make install
RUN cp -r .libs /opt/pcre/
WORKDIR /tmp/hyperscan
WORKDIR /tmp/vectorscan

FROM build_pcre AS build_hyperscan
FROM build_pcre AS build_vectorscan
ARG build_type
ARG pcre_version
RUN mkdir -p build
WORKDIR /tmp/hyperscan/build
WORKDIR /tmp/vectorscan/build
ENV CFLAGS="-fPIC"
RUN [[ "$POLICY" == 'musllinux_1_1' ]] && \
export CFLAGS="$CFLAGS -march=core2"; \
export CXXFLAGS="$CFLAGS -D_GLIBCXX_USE_CXX11_ABI=0"; \
cmake \
-DCMAKE_INSTALL_PREFIX=/opt/hyperscan \
-DBUILD_STATIC_AND_SHARED=ON \
-DCMAKE_INSTALL_PREFIX=/opt/vectorscan \
-DBUILD_STATIC_LIBS=ON \
-DBUILD_SHARED_LIBS=ON \
-DCMAKE_BUILD_TYPE=${build_type} \
-DCMAKE_C_FLAGS="${CFLAGS}" \
-DCMAKE_CXX_FLAGS="${CXXFLAGS}" \
Expand All @@ -66,12 +69,12 @@ RUN make -j$(nproc) && make install

FROM base
LABEL maintainer="David Gidwani <[email protected]>"
LABEL org.opencontainers.image.description Python manylinux with Intel Hyperscan
LABEL org.opencontainers.image.description Python manylinux with Intel Vectorscan
ARG LD_LIBRARY_PATH_ARG
ARG PREPEND_PATH
ENV LD_LIBRARY_PATH=${LD_LIBRARY_PATH_ARG}
ENV PATH=${PREPEND_PATH}${PATH}
ENV PKG_CONFIG_PATH=/opt/pcre/lib/pkgconfig:/opt/hyperscan/lib/pkgconfig:/opt/hyperscan/lib64/pkgconfig:/usr/local/lib/pkgconfig
ENV PKG_CONFIG_PATH=/opt/pcre/lib/pkgconfig:/opt/vectorscan/lib/pkgconfig:/opt/vectorscan/lib64/pkgconfig:/usr/local/lib/pkgconfig
WORKDIR /opt
COPY --from=build_hyperscan /opt/pcre/ pcre
COPY --from=build_hyperscan /opt/hyperscan/ hyperscan
COPY --from=build_vectorscan /opt/pcre/ pcre
COPY --from=build_vectorscan /opt/vectorscan/ vectorscan
26 changes: 21 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,28 @@
![](https://github.com/darvid/manylinux-hyperscan/workflows/Publish%20Docker%20Image/badge.svg?branch=master)

This project provides an **x86_64 only** [manylinux][1]-based Docker
image with [Hyperscan][2] installed.
image with [Vectorscan][3] (a fork of [Intel Hyperscan][2]) installed.

> [!NOTE]
> As of December 13th, 2022, future tagged versions of this project will
> be indepdent of the upstream Hyperscan version, as this base image
> reflects multiple pre-requisites (most notably new Python versions) and
> not just Hyperscan.

> [!NOTE]
> As of 2024, this image defaults to installing [Vectorscan][3] instead
> of Hyperscan, due to a multiple factors, including the lack of
> multi-arch support as well as Intel's decision to move away from an
> open source license after version 5.4 (see [this issue][4] for
> context). The Dockerfile now includes two build args for configuring
> the git repo URI and ref (tag), allowing users to choose between
> Intel Hyperscan and Vectorscan if needed. However, Intel's Hyperscan
> will no longer be supported moving forward, so build functionality
> and compatibility with [python-hyperscan][5] is not guaranteed.

**Note:** As of December 13th, 2022, future tagged versions of this
project will be indepdent of the upstream Hyperscan version, as this
base image reflects multiple pre-requisites (most notably new Python
versions) and not just Hyperscan.

[1]: https://github.com/pypa/manylinux
[2]: https://github.com/intel/hyperscan
[3]: https://github.com/VectorCamp/vectorscan
[4]: https://github.com/intel/hyperscan/issues/421
[5]: https://github.com/darvid/python-hyperscan