defer closing of vector indices

Author: huahaiy

CHANGELOG.md

@@ -1,5 +1,12 @@
 # Change Log
 
+## WIP
+
+### Improved
+
+- [Vector] Defer closing of vector indices until after async executor is
+  shutdown to avoid segfault due to trying to save a closed index.
+
 ## 0.9.21 (2025-03-18)
 
 ### Added
@@ -9,8 +16,9 @@
   indexing is implemented with [usearch](https://github.com/unum-cloud/usearch).
 - [Vector] Corresponding `add-vec`, `remove-vec`, and `search-vec` functions to
   work with vector index. Similar to full-text search, vector search also
-  support domain semantics to allow grouping of vectors into domains, see
-  [doc](doc/vector.md).
+  support domain semantics to allow grouping of vectors into domains.
+- [Doc] [Vector doc](doc/vector.md). **Note**, you may need to install some
+  native dependencies on your system to load Datalevin: OpenMP and Vector Math.
 - [Datalog] New data type `:db.type/vec`, for which a vector index is
   automatically created for them to allow a query function `vec-neighbors` to
   return the datoms with neighboring vector values.

doc/install.md

@@ -56,6 +56,21 @@ For `dep.edn`, this is known to work:
 ```
 Then `clj -A:jvm-base`
 
+### Native Dependencies
+
+For now, Datalevin requires some system libraries to be present, such as `libc`,
+`libomp`, and `libmvec`. These are easy to get:
+
+* Linux needs [OpenMP](https://www.openmp.org/) and [Vectorized
+  Math](https://sourceware.org/glibc/wiki/libmvec) from GCC, e.g. on
+  Debian/Ubuntu, `apt-get install g++-12 gcc-12`
+
+* MacOSX needs the same libraries as the above from Clang, e.g. `brew
+  install libomp llvm`
+
+Otherwise, Datalevin may fail to load and report
+`java.lang.UnsatisfiedLinkError`.
+
 ### Other JVM Languages
 
 Datalevin can be used in other JVM languages than Clojure, such as Java, Scala, Kotlin,

doc/vector.md

@@ -17,12 +17,17 @@ MacOSX on both x86_64 and arm64 CPUs.
 
 ## Native Dependencies
 
-The vector feature expects some native dependencies on your system.
+The vector feature expects some native dependencies on your system:
 
-* Linux on arm64 and x86_64, needs OpenMP and Vectorized Math from GCC,
-  e.g. on latest Debian/Ubuntu, `apt-get install g++-12 gcc-12`
+* Linux needs [OpenMP](https://www.openmp.org/) and [Vectorized
+  Math](https://sourceware.org/glibc/wiki/libmvec) from GCC, e.g. on
+  Debian/Ubuntu, `apt-get install g++-12 gcc-12`
 
-* MacOSX on arm64 and x86_64, needs the same from Clang, e.g. `brew install libomp llvm`
+* MacOSX needs the same libraries as the above from Clang, e.g. `brew
+  install libomp llvm`
+
+Otherwise, Datalevin may fail to load and report
+`java.lang.UnsatisfiedLinkError`.
 
 ## Configurations
 

script/compile-native-intel-mac

@@ -15,13 +15,15 @@ echo $JAVA_HOME
 args_app=(
     "--verbose"
     "--native-compiler-path=/usr/local/opt/llvm/bin/clang"
+    "--native-compiler-options='-I/usr/local/opt/llvm/include -L/usr/local/opt/llvm/lib'"
     "-jar" "target/main.uberjar.jar"
     "-H:Name=dtlv"
 )
 
 args_test0=(
     "--verbose"
     "--native-compiler-path=/usr/local/opt/llvm/bin/clang"
+    "--native-compiler-options='-I/usr/local/opt/llvm/include -L/usr/local/opt/llvm/lib'"
     "-jar" "target/test0.uberjar.jar"
     "-H:Name=dtlv-test0"
 )

src/datalevin/binding/cpp.clj

@@ -556,15 +556,15 @@
   (close-kv [this]
     (when-not (.isClosed env)
       (stop-scheduled-sync scheduled-sync)
-      (doseq [idx (keep @l/vector-indices (u/list-files (.dir this)))]
-        (v/close-vecs idx))
       (swap! l/lmdb-dirs disj (l/dir this))
       (when (zero? (count @l/lmdb-dirs))
         (a/shutdown-executor)
         (u/shutdown-query-thread-pool)
         (u/shutdown-scheduler))
       (.sync env 1)
       (.close env)
+      (doseq [idx (keep @l/vector-indices (u/list-files (.dir this)))]
+        (v/close-vecs idx))
       (when (@info :temp?) (u/delete-files (@info :dir)))
       nil))