Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide a bundled flag for gdal-sys #517

Open
wants to merge 20 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 51 additions & 6 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -62,15 +62,15 @@ jobs:
- name: Check with Clippy
run: cargo clippy --all-targets -- -D warnings
- name: Check with Clippy (--all-features)
run: cargo clippy --all-targets --all-features -- -D warnings
run: cargo clippy --all-targets --features "default bindgen array" -- -D warnings
- name: Build
run: cargo build
- name: Run tests
run: cargo test
- name: Build (--all-features)
run: cargo build --all-features
run: cargo build --features "default bindgen array"
- name: Run tests (--all-features)
run: cargo test --all-features -- --nocapture
run: cargo test --features "default bindgen array" -- --nocapture

ubuntu_lts:
name: "ci ubuntu-lts"
Expand Down Expand Up @@ -105,18 +105,63 @@ jobs:
- name: Check with Clippy
run: cargo clippy --all-targets -- -D warnings
- name: Check with Clippy (--all-features)
run: cargo clippy --all-targets --all-features -- -D warnings
run: cargo clippy --all-targets --features "default bindgen array" -- -D warnings
- name: Build
run: cargo build
- name: Run tests
run: cargo test
- name: Build (--all-features)
run: cargo build --all-features
run: cargo build --features "default bindgen array"
- name: Run tests (--all-features)
run: cargo test --all-features -- --nocapture
run: cargo test --features "default bindgen array" -- --nocapture
- name: Install valgrind and cargo-valgrind
run: |
sudo apt-get install valgrind -y
cargo install cargo-valgrind
- name: Run --lib tests under valgrind
run: cargo valgrind test --lib
gdal_static:
name: "ci gdal-static"
strategy:
matrix:
os:
- ubuntu-latest
- windows-latest
- macos-latest # x86_64
- macos-14 # aarch64
runs-on: ${{ matrix.os }}
steps:
- name: Checkout code
uses: actions/checkout@v4
with:
submodules: true
- name: Install stable
run: |
rustup install --no-self-update --profile minimal stable
rustup component add rustfmt clippy
# we need to have the sqlite binary in path for building proj from source
- name: Install Sqlite (Windows)
if: runner.os == 'Windows'
run: |
choco install sqlite
echo "C:\ProgramData\chocolatey\lib\SQLite\tools" >> $GITHUB_PATH
sqlite3 --version
# use the minimal driver set for clippy as the other
# drivers do not change the rust code
# enable `driver_sqlite` to force statically linking libsqlite3 for proj
- name: Check with Clippy (bundled)
run: cargo clippy --all-targets --features "gdal-sys/bundled gdal-src gdal-src/driver_sqlite" -- -D warnings
# we only build tests here as we have disabled features
# that are required for running tests
- name: Build bundled gdal (minimal features)
if: runner.os == 'Windows'
# we use cargo test --no-run here because
# tests do not pass due to missing libgeos but we want to have a complete build (including linking)
run: cargo test --features "gdal-sys/bundled gdal-src gdal-src/driver_sqlite" --no-run
- name: Build bundled gdal (minimal features)
if: runner.os != 'Windows'
# we use cargo test --no-run here because
# tests do not pass due to missing libgeos but we want to have a complete build (including linking)
run: cargo test --features "gdal-sys/bundled gdal-src" --no-run
- name: Test bundled (all features)
run: cargo test --features "gdal-sys/bundled gdal-src gdal-src/all_drivers gdal-src/geos_static"
3 changes: 3 additions & 0 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[submodule "gdal-src/source"]
weiznich marked this conversation as resolved.
Show resolved Hide resolved
ChristianBeilschmidt marked this conversation as resolved.
Show resolved Hide resolved
path = gdal-src/source
url = https://github.com/OSGeo/gdal
3 changes: 3 additions & 0 deletions CHANGES.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@

## Unreleased

- Add a `bundled` feature for `gdal-sys` that allows to build and statically link a minimal bundled version of gdal during `cargo build`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should probably mention that it only supports SQLite, GPKG, GeoTIFF, some other less popular formats, and probably no compression. I'm afraid that users will see this and expect it to work for COG, JPEG2000, NetCDF and so on.

Also see https://www.mail-archive.com/[email protected]/msg40172.html.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The following 19 drivers are supported by this configuration according to DriverManager::all():

Virtual Raster
Derived datasets using VRT pixel functions
GeoTIFF
Cloud optimized GeoTIFF generator
Erdas Imagine Images (.img)
In Memory Raster
Geographic Network generic file based model
Geographic Network generic DB based model
ESRI Shapefile
MapInfo File
VRT - Virtual Datasource
Memory
Keyhole Markup Language (KML)
GeoJSON
GeoJSON Sequence
ESRIJSON
TopoJSON
GeoPackage
SQLite / Spatialite


## 0.17

- Added pre-built bindings for GDAL 3.9
Expand Down Expand Up @@ -85,6 +87,7 @@

- <https://github.com/georust/gdal/pull/466>


- Added `{Display|FromStr} for ResampleAlg` and `ResampleAlg::iter`.

- <https://github.com/georust/gdal/pull/462>
Expand Down
6 changes: 5 additions & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ thiserror = "1.0"
libc = "0.2"
geo-types = { version = "0.7.11" }
gdal-sys = { path = "gdal-sys", version = "0.10" }
gdal-src = { path = "gdal-src", optional = true, default-features = false }
ndarray = { version = "0.15", optional = true }
chrono = { version = "0.4.26", default-features = false }
bitflags = "2.4"
Expand All @@ -36,7 +37,7 @@ tempfile = "3.8"
arrow = { version = "52.0", default-features = false, features = ["ffi"] }

[workspace]
members = ["gdal-sys"]
members = ["gdal-src", "gdal-sys"]

# docs.rs-specific configuration
[package.metadata.docs.rs]
Expand All @@ -61,3 +62,6 @@ check-cfg = [
'cfg(minor_ge_8)',
'cfg(minor_ge_9)',
]

[patch.crates-io]
ChristianBeilschmidt marked this conversation as resolved.
Show resolved Hide resolved
proj-sys = { git = "https://github.com/georust/proj", rev = "57c9e80" }
17 changes: 17 additions & 0 deletions DEVELOPMENT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# Updating bundled gdal version for gdal-src

Perform the following steps:

```
git submodule init
git submodule update
cd gdal-src/source
git pull
git checkout v3.8.3 # corresponds to the tag you want to update to
cd ../../
git add gdal-src/source
git commit -m "Update bundled gdal version to 3.8.4"
```

These steps assume that there are no fundamental changes to the gdal build system.

Loading
Loading