Skip to content

Commit

Permalink
Remove all the references to MDBX
Browse files Browse the repository at this point in the history
  • Loading branch information
@Kerollmops
Kerollmops committed Mar 1, 2022
1 parent 45e651c commit 87b423a
Show file tree
Hide file tree
Showing 10 changed files with 11 additions and 501 deletions.
29 changes: 7 additions & 22 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,27 +1,15 @@
# heed
A fully typed [LMDB]/[MDBX] wrapper with minimum overhead, uses zerocopy internally.
A fully typed [LMDB](https://en.wikipedia.org/wiki/Lightning_Memory-Mapped_Database) wrapper with minimum overhead, uses zerocopy internally.

[![Build Status](https://dev.azure.com/renaultcle/heed/_apis/build/status/Kerollmops.heed?branchName=master)](https://dev.azure.com/renaultcle/heed/_build/latest?definitionId=1&branchName=master)
[![Dependency Status](https://deps.rs/repo/github/Kerollmops/heed/status.svg)](https://deps.rs/repo/github/Kerollmops/heed)
[![Heed Doc](https://docs.rs/heed/badge.svg)](https://docs.rs/heed)
[![Crates.io](https://img.shields.io/crates/v/heed.svg)](https://crates.io/crates/heed)
[![License](https://img.shields.io/badge/license-MIT-green)](#LICENSE)
[![Crates.io](https://img.shields.io/crates/v/heed)](https://crates.io/crates/heed)
[![Docs](https://docs.rs/heed/badge.svg)](https://docs.rs/heed)
[![dependency status](https://deps.rs/repo/github/meilisearch/heed/status.svg)](https://deps.rs/repo/github/meilisearch/heed)
[![Build](https://github.com/meilisearch/heed/actions/workflows/test.yml/badge.svg)](https://github.com/meilisearch/heed/actions/workflows/test.yml)

![the opposite of heed](https://thesaurus.plus/img/antonyms/153/heed.png)

This library is able to serialize all kind of types, not just bytes slices, even Serde types are supported.

## Using MDBX instead of LMDB

Heed can also [be used with MDBX][MDBX], this is a compatible backend, you can activate it by using the `mdbx` feature. When MDBX is used the generated databases cannot be read by using the `lmdb` feature, those two kv-stores are not compatible and vice-versa.

Notice that some specific features will only be accessible by using the `mdbx` cargo feature.
Environment creation flags depends on the backend you chose.

To test that heed works with MDBX you can run this command:

```bash
cargo test --features 'mdbx serde-json' --no-default-features
```
This library is able to serialize all kind of types, not just bytes slices, even _Serde_ types are supported.

## Example Usage

Expand Down Expand Up @@ -56,6 +44,3 @@ assert_eq!(ret, Some(5));
```

You want to see more about all the possibilities? Go check out [the examples](heed/examples/).

[LMDB]: https://en.wikipedia.org/wiki/Lightning_Memory-Mapped_Database
[MDBX]: https://github.com/erthink/libmdbx
12 changes: 2 additions & 10 deletions heed/Cargo.toml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,7 @@ byteorder = { version = "1.3.4", default-features = false }
heed-traits = { version = "0.7.0", path = "../heed-traits" }
heed-types = { version = "0.7.2", path = "../heed-types" }
libc = "0.2.80"
lmdb-rkv-sys = { git = "https://github.com/meilisearch/lmdb-rs", optional = true }
mdbx-sys = { version = "0.7.1", optional = true }
lmdb-rkv-sys = { git = "https://github.com/meilisearch/lmdb-rs" }
once_cell = "1.5.2"
page_size = "0.4.2"
serde = { version = "1.0.118", features = ["derive"], optional = true }
Expand All @@ -33,20 +32,13 @@ url = "2.2.0"
[features]
# The `serde` feature makes some types serializable,
# like the `EnvOpenOptions` struct.
default = ["lmdb", "serde", "serde-bincode", "serde-json"]
default = ["serde", "serde-bincode", "serde-json"]

# The NO_TLS flag is automatically set on Env opening and
# RoTxn implements the Sync trait. This allow the user to reference
# a read-only transaction from multiple threads at the same time.
sync-read-txn = []

# Choose between using the MDBX key-value store or LMDB
# MDBX is a fork of LMDB: https://github.com/erthink/libmdbx
#
# You cannot use both features at the same time, you have to choose!
lmdb = ["lmdb-rkv-sys"]
mdbx = ["mdbx-sys"]

# Enable the serde en/decoders for bincode or serde_json
serde-bincode = ["heed-types/serde", "heed-types/bincode"]
serde-json = ["heed-types/serde", "heed-types/serde_json"]
Expand Down
121 changes: 0 additions & 121 deletions heed/src/db/polymorph.rs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -113,127 +113,6 @@ impl PolyDatabase {
PolyDatabase { env_ident, dbi }
}

/// Retrieve the sequence of a database.
///
/// This function allows to retrieve the unique positive integer of this database.
///
/// ```
/// # use std::fs;
/// # use std::path::Path;
/// # use heed::EnvOpenOptions;
/// use heed::Database;
/// use heed::types::*;
///
/// # fn main() -> Result<(), Box<dyn std::error::Error>> {
/// # fs::create_dir_all(Path::new("target").join("zerocopy.mdb"))?;
/// # let env = EnvOpenOptions::new()
/// # .map_size(10 * 1024 * 1024) // 10MB
/// # .max_dbs(3000)
/// # .open(Path::new("target").join("zerocopy.mdb"))?;
/// let db = env.create_poly_database(Some("use-sequence-1"))?;
///
/// // The sequence starts at zero
/// let rtxn = env.read_txn()?;
/// let ret = db.sequence(&rtxn)?;
/// assert_eq!(ret, 0);
/// rtxn.abort()?;
///
/// let mut wtxn = env.write_txn()?;
/// # db.clear(&mut wtxn)?;
/// let incr = db.increase_sequence(&mut wtxn, 32)?;
/// assert_eq!(incr, Some(0));
/// let incr = db.increase_sequence(&mut wtxn, 28)?;
/// assert_eq!(incr, Some(32));
/// wtxn.commit()?;
///
/// let rtxn = env.read_txn()?;
/// let ret = db.sequence(&rtxn)?;
/// assert_eq!(ret, 60);
///
/// # Ok(()) }
/// ```
#[cfg(all(feature = "mdbx", not(feature = "lmdb")))]
pub fn sequence<T>(&self, txn: &RoTxn<T>) -> Result<u64> {
assert_eq!(self.env_ident, txn.env.env_mut_ptr() as usize);

let mut value = mem::MaybeUninit::uninit();

let result = unsafe {
mdb_result(ffi::mdbx_dbi_sequence(
txn.txn,
self.dbi,
value.as_mut_ptr(),
0, // increment must be 0 for read-only transactions
))
};

match result {
Ok(()) => unsafe { Ok(value.assume_init()) },
Err(e) => Err(e.into()),
}
}

/// Increment the sequence of a database.
///
/// This function allows to create a linear sequence of a unique positive integer
/// for this database. Sequence changes become visible outside the current write
/// transaction after it is committed, and discarded on abort.
///
/// Returns `Some` with the previous value and `None` if increasing the value
/// resulted in an overflow an therefore cannot be executed.
///
/// ```
/// # use std::fs;
/// # use std::path::Path;
/// # use heed::EnvOpenOptions;
/// use heed::Database;
/// use heed::types::*;
///
/// # fn main() -> Result<(), Box<dyn std::error::Error>> {
/// # fs::create_dir_all(Path::new("target").join("zerocopy.mdb"))?;
/// # let env = EnvOpenOptions::new()
/// # .map_size(10 * 1024 * 1024) // 10MB
/// # .max_dbs(3000)
/// # .open(Path::new("target").join("zerocopy.mdb"))?;
/// let db = env.create_poly_database(Some("use-sequence-2"))?;
///
/// let mut wtxn = env.write_txn()?;
/// let incr = db.increase_sequence(&mut wtxn, 32)?;
/// assert_eq!(incr, Some(0));
/// let incr = db.increase_sequence(&mut wtxn, 28)?;
/// assert_eq!(incr, Some(32));
/// wtxn.commit()?;
///
/// let rtxn = env.read_txn()?;
/// let ret = db.sequence(&rtxn)?;
/// assert_eq!(ret, 60);
///
/// # Ok(()) }
/// ```
#[cfg(all(feature = "mdbx", not(feature = "lmdb")))]
pub fn increase_sequence<T>(&self, txn: &mut RwTxn<T>, increment: u64) -> Result<Option<u64>> {
assert_eq!(self.env_ident, txn.txn.env.env_mut_ptr() as usize);

use crate::mdb::error::Error;

let mut value = mem::MaybeUninit::uninit();

let result = unsafe {
mdb_result(ffi::mdbx_dbi_sequence(
txn.txn.txn,
self.dbi,
value.as_mut_ptr(),
increment,
))
};

match result {
Ok(()) => unsafe { Ok(Some(value.assume_init())) },
Err(Error::Other(c)) if c == i32::max_value() => Ok(None), // MDBX_RESULT_TRUE
Err(e) => Err(e.into()),
}
}

/// Retrieves the value associated with a key.
///
/// If the key does not exist, then `None` is returned.
Expand Down
23 changes: 0 additions & 23 deletions heed/src/db/uniform.rs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -127,29 +127,6 @@ impl<KC, DC> Database<KC, DC> {
}
}

/// Retrieve the sequence of a database.
///
/// This function allows to retrieve the unique positive integer of this database.
/// You can see an example usage on the `PolyDatabase::sequence` method documentation.
#[cfg(all(feature = "mdbx", not(feature = "lmdb")))]
pub fn sequence<T>(&self, txn: &RoTxn<T>) -> Result<u64> {
self.dyndb.sequence(txn)
}

/// Increment the sequence of a database.
///
/// This function allows to create a linear sequence of a unique positive integer
/// for this database. Sequence changes become visible outside the current write
/// transaction after it is committed, and discarded on abort.
/// You can see an example usage on the `PolyDatabase::increase_sequence` method documentation.
///
/// Returns `Some` with the previous value and `None` if increasing the value
/// resulted in an overflow an therefore cannot be executed.
#[cfg(all(feature = "mdbx", not(feature = "lmdb")))]
pub fn increase_sequence<T>(&self, txn: &mut RwTxn<T>, increment: u64) -> Result<Option<u64>> {
self.dyndb.increase_sequence(txn, increment)
}

/// Retrieves the value associated with a key.
///
/// If the key does not exist, then `None` is returned.
Expand Down
8 changes: 0 additions & 8 deletions heed/src/env.rs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -460,20 +460,12 @@ impl Env {
Ok(())
}

#[cfg(all(feature = "lmdb", not(feature = "mdbx")))]
pub fn force_sync(&self) -> Result<()> {
unsafe { mdb_result(ffi::mdb_env_sync(self.0.env, 1))? }

Ok(())
}

#[cfg(all(feature = "mdbx", not(feature = "lmdb")))]
pub fn force_sync(&self) -> Result<()> {
unsafe { mdb_result(ffi::mdb_env_sync(self.0.env))? }

Ok(())
}

/// Returns the canonicalized path where this env lives.
pub fn path(&self) -> &Path {
&self.0.path
Expand Down
Loading

0 comments on commit 87b423a

Please sign in to comment.