Skip to content

Commit c798211

Browse files
huahaiyhuahaiy
committed
prepare to release
1 parent 4048c51 commit c798211

File tree

21 files changed

+256
-184
lines changed

21 files changed

+256
-184
lines changed

‎README.md‎

Lines changed: 20 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -55,14 +55,19 @@ more declarative and composable than SQL, e.g. the automatic implicit joins seem
5555
to be its killer feature. In addition, the recursive rules feature of Datalog
5656
makes it suitable for graph processing and deductive reasoning.
5757

58-
The feature set of Datomic® may not be a good fit for some use cases. One thing that may [confuse some users](https://vvvvalvalval.github.io/posts/2017-07-08-Datomic-this-is-not-the-history-youre-looking-for.html) is its [temporal features](https://docs.datomic.com/cloud/whatis/data-model.html#time-model). To keep things simple and familiar, Datalevin behaves the same way as most other databases: when data are deleted, they are gone. Datalevin also follows the widely accepted principles of ACID, instead of introducing [unusual semantics](https://jepsen.io/analyses/datomic-pro-1.0.7075).
59-
60-
Datalevin started out as a port of
61-
[Datascript](https://github.com/tonsky/datascript) in-memory Datalog database to
62-
[LMDB](https://en.wikipedia.org/wiki/Lightning_Memory-Mapped_Database) for
63-
persistence. We then added a [cost-based query optimizer](doc/query.md),
64-
resulting in query performance [competitive](benchmarks/JOB-bench) against SQL
65-
RDBMS such as PostgreSQL.
58+
The feature set of Datomic® may not be a good fit for some use cases. One thing
59+
that may [confuse some
60+
users](https://vvvvalvalval.github.io/posts/2017-07-08-Datomic-this-is-not-the-history-youre-looking-for.html)
61+
is its [temporal
62+
features](https://docs.datomic.com/cloud/whatis/data-model.html#time-model). To
63+
keep things simple and familiar, Datalevin behaves the same way as most other
64+
databases: when data are deleted, they are gone. Datalevin also follows the
65+
widely accepted principles of ACID, instead of introducing [unusual
66+
semantics](https://jepsen.io/analyses/datomic-pro-1.0.7075).
67+
68+
Datalevin has a [novel cost-based query optimizer](doc/query.md), with a query
69+
performance [competitive](benchmarks/JOB-bench) with SQL RDBMS such as
70+
PostgreSQL.
6671

6772
Datalevin can be used as a library, embedded in applications to manage state, e.g.
6873
used like SQLite; or it can run in a networked
@@ -84,7 +89,7 @@ engine](https://github.com/juji-io/datalevin/blob/master/doc/search.md) that has
8489
[competitive](https://github.com/juji-io/datalevin/tree/master/benchmarks/search-bench)
8590
search performance.
8691

87-
Additional materials:
92+
Additional materials in chronological order:
8893

8994
* [Competing for the JOB with a Triplestore](https://yyhh.org/blog/2024/09/competing-for-the-job-with-a-triplestore/)
9095
* [If I had to Pick One: Datalevin](https://vimsical.notion.site/If-I-Had-To-Pick-One-Datalevin-be5c4b62cda342278a10a5e5cdc2206d)
@@ -112,7 +117,7 @@ Datalevin is aimed to be a versatile database.
112117
### Use as a Datalog store
113118

114119
In addition to [our API doc](https://cljdoc.org/d/datalevin/datalevin),
115-
since Datalevin has almost the same Datalog API as Datascript, which in turn has
120+
Datalevin has almost the same Datalog API as [Datascript](https://github.com/tonsky/datascript), which in turn has
116121
almost the same API as Datomic®, please consult the abundant tutorials, guides
117122
and learning sites available online to learn about the usage of Datomic® flavor
118123
of Datalog.
@@ -256,13 +261,11 @@ for EDN data.
256261

257262
Please refer to the [API
258263
documentation](https://cljdoc.org/d/datalevin/datalevin) for more details.
259-
You may also consult online materials for Datascript or Datomic®, as the Datalog API is
260-
similar.
261264

262265
## :rocket: Status
263266

264267
Datalevin is extensively tested with property-based testing. It is also used
265-
in production at [Juji](https://juji.io).
268+
in production at [Juji](https://juji.io), among other companies.
266269

267270
Running the [benchmark suite adopted from
268271
Datascript](https://github.com/juji-io/datalevin/tree/master/benchmarks/datascript-bench),
@@ -278,9 +281,8 @@ here is how it looks.
278281

279282
In all benchmarked queries, Datalevin is the fastest among the three tested
280283
systems, as Datalevin has a [cost based query optimizer](doc/query.md) while Datascript and
281-
Datomic do not. Datalevin also has a caching layer for index access.
282-
283-
See [here](https://github.com/juji-io/datalevin/tree/master/benchmarks/datascript-bench)
284+
Datomic do not. Datalevin also has a caching layer for index access. See
285+
[here](https://github.com/juji-io/datalevin/tree/master/benchmarks/datascript-bench)
284286
for a detailed analysis of the results.
285287

286288
We also compared Datalevin and PostgreSQL in handling complex queries, using
@@ -294,9 +296,8 @@ the average times:
294296

295297
Datalevin is about 1.3X faster than PostgreSQL on average in running the complex
296298
queries in this benchmark. The gain is mainly in better query execution time due
297-
to higher quality of generated query plans.
298-
299-
For more details, see [here](benchmarks/JOB-bench).
299+
to higher quality of generated query plans. For more details, see
300+
[here](benchmarks/JOB-bench).
300301

301302
## :earth_americas: Roadmap
302303

‎appveyor.yml‎

Lines changed: 10 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -4,11 +4,9 @@ version: "v-{build}"
44

55
branches:
66
only:
7-
- coll
8-
# - master
7+
- master
98

10-
skip_non_tags: false
11-
# skip_non_tags: true
9+
skip_non_tags: true
1210

1311
image: Visual Studio 2022
1412

@@ -54,11 +52,11 @@ build_script:
5452
artifacts:
5553
- path: dtlv-$(APPVEYOR_REPO_TAG_NAME)-windows-amd64.zip
5654

57-
# deploy:
58-
# provider: GitHub
59-
# auth_token:
60-
# secure: QIiuupmpAl2Y5kxVyFJTSDRIj/uKyXXE3PzARE//JY44zXEJ7vbqvhcftKq/dFdf
61-
# artifact: /.*\.zip/
62-
# force_update: true
63-
# on:
64-
# APPVEYOR_REPO_TAG: true
55+
deploy:
56+
provider: GitHub
57+
auth_token:
58+
secure: QIiuupmpAl2Y5kxVyFJTSDRIj/uKyXXE3PzARE//JY44zXEJ7vbqvhcftKq/dFdf
59+
artifact: /.*\.zip/
60+
force_update: true
61+
on:
62+
APPVEYOR_REPO_TAG: true

‎benchmarks/JOB-bench/deps.edn‎

Lines changed: 12 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,18 @@
11
{:paths ["src"]
22
:deps {
3-
datalevin/datalevin {:local/root "../.."}
3+
datalevin/datalevin {:local/root "../.."}
44
;; datalevin/datalevin {:mvn/version "0.9.14"}
5-
org.clojure/clojure {:mvn/version "1.12.0"}
6-
com.cognitect/transit-clj {:mvn/version "1.0.333"}
7-
com.taoensso/nippy {:mvn/version "3.4.2"}
8-
me.lemire.integercompression/JavaFastPFOR {:mvn/version "0.1.12"}
9-
org.clojars.huahaiy/dtlvnative-macosx-arm64 {:mvn/version "0.11.0"}
10-
org.roaringbitmap/RoaringBitmap {:mvn/version "1.0.6"}
11-
org.eclipse.collections/eclipse-collections {:mvn/version "11.1.0"}
5+
org.clojure/clojure {:mvn/version "1.12.0"}
6+
com.cognitect/transit-clj {:mvn/version "1.0.333"}
7+
com.taoensso/nippy {:mvn/version "3.4.2"}
8+
me.lemire.integercompression/JavaFastPFOR {:mvn/version "0.1.12"}
9+
org.clojars.huahaiy/dtlvnative-macosx-arm64 {:mvn/version "0.11.4"}
10+
org.clojars.huahaiy/dtlvnative-macosx-x86_64 {:mvn/version "0.11.4"}
11+
org.clojars.huahaiy/dtlvnative-linux-arm64 {:mvn/version "0.11.4"}
12+
org.clojars.huahaiy/dtlvnative-linux-x86_64 {:mvn/version "0.11.4"}
13+
org.clojars.huahaiy/dtlvnative-windows-x86_64 {:mvn/version "0.11.4"}
14+
org.roaringbitmap/RoaringBitmap {:mvn/version "1.3.0"}
15+
org.eclipse.collections/eclipse-collections {:mvn/version "11.1.0"}
1216
}
1317

1418
:aliases {:dev {:extra-paths ["target/classes"]

‎benchmarks/math-bench/deps.edn‎

Lines changed: 15 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,20 @@
11
{:paths ["src"]
22
:deps {
3-
datalevin/datalevin {:local/root "../.."}
4-
datascript/datascript {:mvn/version "1.6.5"}
5-
com.datomic/peer {:mvn/version "1.0.7075"}
6-
com.cognitect/transit-clj {:mvn/version "1.0.333"}
7-
metosin/jsonista {:mvn/version "0.3.8"}
8-
com.taoensso/nippy {:mvn/version "3.4.0"}
9-
me.lemire.integercompression/JavaFastPFOR {:mvn/version "0.1.12"}
10-
org.clojure/clojure {:mvn/version "1.11.3"}
11-
org.lmdbjava/lmdbjava {:mvn/version "0.9.0"}
12-
org.roaringbitmap/RoaringBitmap {:mvn/version "1.0.6"}
13-
org.eclipse.collections/eclipse-collections {:mvn/version "11.1.0"}
3+
datalevin/datalevin {:local/root "../.."}
4+
datascript/datascript {:mvn/version "1.7.4"}
5+
com.datomic/peer {:mvn/version "1.0.7075"}
6+
com.cognitect/transit-clj {:mvn/version "1.0.333"}
7+
metosin/jsonista {:mvn/version "0.3.8"}
8+
com.taoensso/nippy {:mvn/version "3.4.2"}
9+
me.lemire.integercompression/JavaFastPFOR {:mvn/version "0.1.12"}
10+
org.clojure/clojure {:mvn/version "1.12.0"}
11+
org.roaringbitmap/RoaringBitmap {:mvn/version "1.3.0"}
12+
org.clojars.huahaiy/dtlvnative-macosx-arm64 {:mvn/version "0.11.4"}
13+
org.clojars.huahaiy/dtlvnative-macosx-x86_64 {:mvn/version "0.11.4"}
14+
org.clojars.huahaiy/dtlvnative-linux-arm64 {:mvn/version "0.11.4"}
15+
org.clojars.huahaiy/dtlvnative-linux-x86_64 {:mvn/version "0.11.4"}
16+
org.clojars.huahaiy/dtlvnative-windows-x86_64 {:mvn/version "0.11.4"}
17+
org.eclipse.collections/eclipse-collections {:mvn/version "11.1.0"}
1418
}
1519

1620
:aliases {:dev {

‎benchmarks/search-bench/deps.edn‎

Lines changed: 22 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -1,22 +1,27 @@
11
{:paths ["src"]
22
:deps {
3-
com.cognitect/transit-clj {:mvn/version "1.0.333"}
4-
com.fasterxml.jackson.core/jackson-core {:mvn/version "2.17.0"}
5-
com.fasterxml.jackson.core/jackson-databind {:mvn/version "2.17.0"}
6-
com.taoensso/nippy {:mvn/version "3.4.0-RC3"}
7-
;; datalevin/datalevin {:local/root "../.."}
8-
datalevin/datalevin {:mvn/version "0.9.5"}
9-
me.lemire.integercompression/JavaFastPFOR {:mvn/version "0.1.12"}
10-
org.clojure/clojure {:mvn/version "1.11.2"}
11-
org.lmdbjava/lmdbjava {:mvn/version "0.9.0"}
12-
org.roaringbitmap/RoaringBitmap {:mvn/version "1.0.6"}
13-
org.eclipse.collections/eclipse-collections {:mvn/version "11.1.0"}
14-
org.apache.lucene/lucene-core {:mvn/version "8.10.1"}
15-
org.apache.lucene/lucene-analyzers-common {:mvn/version "8.10.1"}
16-
org.apache.lucene/lucene-queryparser {:mvn/version "8.10.1"}
17-
org.apache.lucene/lucene-queries {:mvn/version "8.10.1"}
18-
org.apache.lucene/lucene-highlighter {:mvn/version "8.10.1"}
19-
org.apache.lucene/lucene-codecs {:mvn/version "8.10.1"}
3+
com.cognitect/transit-clj {:mvn/version "1.0.333"}
4+
com.fasterxml.jackson.core/jackson-core {:mvn/version "2.17.0"}
5+
com.fasterxml.jackson.core/jackson-databind {:mvn/version "2.17.0"}
6+
com.taoensso/nippy {:mvn/version "3.4.2"}
7+
datalevin/datalevin {:local/root "../.."}
8+
org.clojars.huahaiy/dtlvnative-macosx-arm64 {:mvn/version "0.11.4"}
9+
org.clojars.huahaiy/dtlvnative-macosx-x86_64 {:mvn/version "0.11.4"}
10+
org.clojars.huahaiy/dtlvnative-linux-arm64 {:mvn/version "0.11.4"}
11+
org.clojars.huahaiy/dtlvnative-linux-x86_64 {:mvn/version "0.11.4"}
12+
org.clojars.huahaiy/dtlvnative-windows-x86_64 {:mvn/version "0.11.4"}
13+
;; datalevin/datalevin {:mvn/version "0.9.5"}
14+
me.lemire.integercompression/JavaFastPFOR {:mvn/version "0.1.12"}
15+
org.clojure/clojure {:mvn/version "1.12.0"}
16+
org.lmdbjava/lmdbjava {:mvn/version "0.9.0"}
17+
org.roaringbitmap/RoaringBitmap {:mvn/version "1.3.0"}
18+
org.eclipse.collections/eclipse-collections {:mvn/version "11.1.0"}
19+
org.apache.lucene/lucene-core {:mvn/version "8.10.1"}
20+
org.apache.lucene/lucene-analyzers-common {:mvn/version "8.10.1"}
21+
org.apache.lucene/lucene-queryparser {:mvn/version "8.10.1"}
22+
org.apache.lucene/lucene-queries {:mvn/version "8.10.1"}
23+
org.apache.lucene/lucene-highlighter {:mvn/version "8.10.1"}
24+
org.apache.lucene/lucene-codecs {:mvn/version "8.10.1"}
2025
}
2126

2227
:aliases {:datalevin {:exec-fn datalevin.bench/-main

‎examples/simple-deps/deps.edn‎

Lines changed: 15 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,19 @@
11
{:paths ["src"]
2-
:deps {io.github.juji-io/datalevin
3-
#_ {:git/tag "0.6.21"
4-
:git/sha "41d1915eb67b1606aab4807058ff24bb41fdb8e8"}
5-
#_ {:mvn/version "0.9.5"}
2+
:deps {datalevin/datalevin
3+
#_ {:git/tag "0.6.21"
4+
:git/sha "41d1915eb67b1606aab4807058ff24bb41fdb8e8"}
5+
#_ {:mvn/version "0.9.5"}
66
{:local/root "../.."}
7-
com.cognitect/transit-clj {:mvn/version "1.0.333"}
8-
io.airlift/aircompressor {:mvn/version "0.26"}}
7+
com.cognitect/transit-clj {:mvn/version "1.0.333"}
8+
com.taoensso/nippy {:mvn/version "3.4.2"}
9+
me.lemire.integercompression/JavaFastPFOR {:mvn/version "0.1.12"}
10+
org.clojars.huahaiy/dtlvnative-macosx-arm64 {:mvn/version "0.11.4"}
11+
org.clojars.huahaiy/dtlvnative-macosx-x86_64 {:mvn/version "0.11.4"}
12+
org.clojars.huahaiy/dtlvnative-linux-arm64 {:mvn/version "0.11.4"}
13+
org.clojars.huahaiy/dtlvnative-linux-x86_64 {:mvn/version "0.11.4"}
14+
org.clojars.huahaiy/dtlvnative-windows-x86_64 {:mvn/version "0.11.4"}
15+
org.roaringbitmap/RoaringBitmap {:mvn/version "1.3.0"}
16+
org.eclipse.collections/eclipse-collections {:mvn/version "11.1.0"}
17+
}
918
:aliases {:dev {:jvm-opts ["--add-opens=java.base/java.nio=ALL-UNNAMED"
1019
"--add-opens=java.base/sun.nio.ch=ALL-UNNAMED"]}}}

‎release.clj‎

Lines changed: 14 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,8 @@
44

55
(def new-v (first *command-line-args*))
66

7-
(assert (re-matches #"\d+\.\d+\.\d+" (or new-v "")) "Use ./release.clj <new-version>")
7+
(assert (re-matches #"\d+\.\d+\.\d+" (or new-v ""))
8+
"Use ./release.clj <new-version>")
89
(println "Releasing version" new-v)
910

1011
(require '[clojure.string :as str])
@@ -23,7 +24,8 @@
2324

2425
(defn sh [& args]
2526
(apply println "Running" (if (empty? *env*) "" (str :env " " *env*)) args)
26-
(let [res (apply sh/sh (concat args [:env (merge (into {} (System/getenv)) *env*)]))]
27+
(let [res (apply sh/sh
28+
(concat args [:env (merge (into {} (System/getenv)) *env*)]))]
2729
(if (== 0 (:exit res))
2830
(do
2931
(println (:out res))
@@ -32,27 +34,23 @@
3234
(println "Process" args "exited with code" (:exit res))
3335
(println (:out res))
3436
(println (:err res))
35-
(throw (ex-info (str "Process" args "exited with code" (:exit res)) res))))))
37+
(throw (ex-info
38+
(str "Process" args "exited with code" (:exit res)) res))))))
3639

3740
(defn update-version []
3841
(println "\n\n[ Updating version number ]\n")
3942
(let [old-v (current-version)
4043
old->new #(str/replace % old-v new-v)]
4144
(update-file "CHANGELOG.md"
42-
#(str/replace % "# WIP"
43-
(str "# " new-v " ("
44-
(.toString (LocalDate/now))
45-
")")))
45+
#(str/replace % "# WIP" (str "# " new-v " ("
46+
(.toString (LocalDate/now))
47+
")")))
4648
(update-file "project.clj" #(str/replace-first % old-v new-v))
4749
(update-file "test-jar/deps.edn" old->new)
4850
(update-file "test-jar/test-uber.sh" old->new)
49-
(update-file "test-jar/project.clj" old->new)
5051
(update-file "doc/install.md" old->new)
5152
(update-file "doc/dtlv.md" old->new)
5253
(update-file "src/datalevin/constants.clj" old->new)
53-
(update-file "native/project.clj" old->new)
54-
(update-file "native/test-jar/deps.edn" old->new)
55-
(update-file "native/README.md" old->new)
5654
(update-file "README.md" old->new)))
5755

5856
(defn make-commit []
@@ -62,13 +60,9 @@
6260
"project.clj"
6361
"test-jar/deps.edn"
6462
"test-jar/test-uber.sh"
65-
"test-jar/project.clj"
6663
"doc/install.md"
6764
"doc/dtlv.md"
6865
"src/datalevin/constants.clj"
69-
"native/project.clj"
70-
"native/test-jar/deps.edn"
71-
"native/README.md"
7266
"README.md")
7367

7468
(sh "git" "commit" "-m" (str "Version " new-v))
@@ -91,13 +85,13 @@
9185
(sh "test-jar/test-uber.sh")
9286

9387
(println "\n\n[ Testing native jar ]\n")
94-
(sh "script/jar" :dir "native")
95-
(sh "./test.sh" :dir "native/test-jar")
88+
(sh "test-jar/test-native.sh")
9689

9790
(println "\n\n[ Running native tests ]\n")
98-
(sh "script/compile-local" :dir "native")
99-
(sh "native/dtlv-test0")
100-
(sh "native/dtlv-test1")
91+
(sh "script/compile-native")
92+
(sh "./dtlv-test0")
93+
(sh "script/compile-native-test1")
94+
(sh "./dtlv-test1")
10195
)
10296

10397
(defn- str->json [s]
@@ -138,7 +132,6 @@
138132
(make-commit)
139133
(github-release)
140134
(sh "./deploy" :dir "script")
141-
(sh "script/deploy" :dir "native")
142135
(System/exit 0)
143136
)
144137

‎resources/META-INF/native-image/datalevin/datalevin/native-image.properties‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,5 +4,5 @@ Args=-H:+UnlockExperimentalVMOptions \
44
--enable-url-protocols=https \
55
--features=clj_easy.graal_build_time.InitClojureClasses \
66
--initialize-at-build-time=me.lemire.integercompression \
7-
--initialize-at-run-time=org.bytedeco.javacpp.presets.javacpp,datalevin.dtlvnative,datalevin.cpp \
7+
--initialize-at-run-time=org.bytedeco.javacpp.presets.javacpp,datalevin.dtlvnative,datalevin.cpp,org.eclipse.collections.impl.set.sorted.mutable \
88
-H:ReachabilityMetadataResources=${.}/reachability-metadata.json

‎resources/META-INF/native-image/datalevin/datalevin/reachability-metadata.json‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,15 @@
1414
{
1515
"type": "org.joda.time.DateTime"
1616
},
17+
{
18+
"type": "org.eclipse.collections.impl.set.sorted.mutable.TreeSortedSet",
19+
"allDeclaredConstructors" : true,
20+
"allPublicConstructors" : true,
21+
"allDeclaredMethods" : true,
22+
"allPublicMethods" : true,
23+
"allPublicFields" : true,
24+
"allDeclaredFields" : true
25+
},
1726
{
1827
"type": "java.lang.UnsatisfiedLinkError"
1928
},

‎script/deploy‎

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2,9 +2,6 @@
22

33
set -eou pipefail
44

5-
export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
6-
export PATH=$JAVA_HOME/bin:$PATH
7-
85
cd ..
96

107
lein clean

0 commit comments

Comments
 (0)