Skip to content

Public APIs (1.0 Beta)

chiyoung edited this page Nov 6, 2014 · 3 revisions

Public APIs (1.0 Beta)

The examples of ForestDB APIs described in this page are in the C language. The basic indexing unit of ForestDB is a document that consists of key, metadata, and body, each of which is an arbitrary length* sequence of uninterpreted bytes. Documents are ordered by key (defaulting to a simple lexicographic sort, like memcmp) and can be iterated in that order.

* The current implementation limits the length of keys to 3840 bytes, but this will be relaxed in the future.

Creating & Opening A Database

Each ForestDB instance has a unique name, that is used as a prefix of path-names for corresponding DB files. The following example shows how to create and open a DB instance. Note that the default configuration creates a new DB file if it does not exist.

#include <assert.h>
#include "libforestdb/forestdb.h"

fdb_handle *db;
fdb_status status;
fdb_config config;

config = fdb_get_default_config();
status = fdb_open(&db, "/tmp/db_filename", &config);
assert(status == FDB_RESULT_SUCCESS);
...

Almost all ForestDB API calls return their results as fdb_status type. The return value will be FDB_RESULT_SUCCESS if the call succeeded. Otherwise, the value has an error number defined in fdb_errors.h.

ForestDB provides several configuration options via a variable defined as the fdb_config type. You can get the default configuration by calling fdb_get_default_config API, and the details are described in fdb_types.h.

Note that an ForestDB handle (fdb_handle) cannot be used simultaneously by two or more threads, which means that each thread should have its own handles. As some languages (e.g., Golang) abstract threading, we recommend them to use the pooling for readers and writers.

Adjusting Block Cache Size

ForestDB’s block cache is an alternative to the OS page cache, whose default size is 128 MB. You can change the size in bytes, or disable the cache by setting the size to zero:

...
fdb_config config;

config = fdb_get_default_config();
config.buffercache_size = 1073741824; // 1 GB
...
...
fdb_config config;

config = fdb_get_default_config();
config.buffercache_size = 0; // disable
...

The block cache is globally shared across all active DB instances, thus buffercache_size will be ignored if the block cache is already initialized by the previously called fdb_open API.

Closing A Database

If you want to close a DB handle, call fdb_close as follows:

fdb_handle *db;
fdb_status status;
...

status = fdb_close(db);
assert(status == FDB_RESULT_SUCCESS);

Note that global resources such as the block cache and file management data still reside in memory for fast restart, even after closing the DB handle. If you want to completely terminate the ForestDB library, call fdb_shutdown as follows:

fdb_status status;
...

status = fdb_shutdown();
assert(status == FDB_RESULT_SUCCESS);

Creating A Document

As we mentioned, ForestDB performs indexing on a per-document basis. To create an in-memory document, call fdb_doc_create. (This does not add the document to the database; it just allocates memory for the fdb_doc structure and its contents.)

...
fdb_doc *doc;
fdb_status status;

status = fdb_doc_create(&doc, "foo", 3, "00", 2, "bar", 3);
assert(status == FDB_RESULT_SUCCESS);
...

Note that any field can be empty:

fdb_doc *doc1, *doc2, *doc3;
fdb_status status;

status = fdb_doc_create(&doc1, "foo", 3, NULL, 0, NULL, 0);
status = fdb_doc_create(&doc2, NULL, 0, NULL, 0, NULL, 0);
status = fdb_doc_create(&doc3, "bar", 3, NULL, 0, "baz", 3);

Changing A Document's Contents

If you want to change the metadata or body of a document in memory, call fdb_doc_update. (Again, this does not affect the database; it just updates the in-memory values stored in the fdb_doc.)

...
fdb_doc *doc;
fdb_status status;

status = fdb_doc_create(&doc, "foo", 3, "00", 2, "bar", 3);
assert(status == FDB_RESULT_SUCCESS);
status = fdb_doc_update(&doc, "11", 2, "baz", 3);
assert(status == FDB_RESULT_SUCCESS);
...

This example modifies the metadata and body of the document doc from 00 and bar to 11 and baz, respectively.

Destroying A Document

To free an fdb_doc structure and the data it stores, just call fdb_doc_free:

fdb_doc *doc;
fdb_status status;
...

status = fdb_doc_free(doc);
assert(status == FDB_RESULT_SUCCESS);

Inserting A Document Into A Database

The following example shows how to insert a document into a DB instance by calling fdb_set:

fdb_handle *db;
fdb_doc *doc;
fdb_status status;
...

status = fdb_doc_create(&doc, "foo", 3, "00", 2, "bar", 3);
status = fdb_set(db, doc);
status = fdb_commit(db, FDB_COMMIT_NORMAL);
status = fdb_doc_free(doc);
...

If a document with the same key already exists, ForestDB overwrites it with the new document.

After a successful fdb_set the fdb_doc can be freed if desired, since its contents have now been copied into the database.

After making one or more changes to the database, you must invoke fdb_commit to commit the updates to disk. Without this, the document would be lost after closing the database handle or upon a system failure. fdb_commit writes any pending in-memory data, updates the tree structures, adds a new header at the end of the file, and calls fsync to flush all dirty data into the disk.

Asynchronous Write

If you want to write data to disk asynchronously, ForestDB supports an option for asynchronous commit mode in fdb_config. The following example shows how to write a document and commit the update asynchronously. Note that fdb_commit should be called even though the DB instance is opened with asynchronous option.

fdb_handle *db;
fdb_doc *doc;
fdb_status status;
fdb_config config;

config = fdb_get_default_config();
config.durability_opt = FDB_DRB_ASYNC;
status = fdb_open(&db, "/tmp/db_filename", &config);
...

status = fdb_doc_create(&doc, "foo", 3, "00", 2, "bar", 3);
status = fdb_set(db, doc);
status = fdb_commit(db, FDB_COMMIT_NORMAL);
status = fdb_doc_free(doc);
...

Writing A Batch of Documents

To commit more than one document update at once, call fdb_set for each document and then invoke fdb_commit at the end:

fdb_handle *db;
fdb_doc *doc1, *doc2, *doc3;
fdb_status status;
...

status = fdb_set(db, doc1);
status = fdb_set(db, doc2);
status = fdb_set(db, doc3);
status = fdb_commit(db, FDB_COMMIT_NORMAL);
...

Bulk Insert

If you want to add large numbers of documents at once, the wal_flush_before_commit option in fdb_config is useful. This option periodically flushes WAL entries into the on-disk tree before fdb_commit API is called, in order to restrict the overall memory consumption by WAL indexes. fdb_commit only has to be invoked once at the end of the bulk load.

fdb_handle *db;
fdb_status status;
fdb_config config;

config = fdb_get_default_config();
config.wal_flush_before_commit = true;
status = fdb_open(&db, "/tmp/db_filename", &config);

... (do bulk write) ...

status = fdb_commit(db, FDB_COMMIT_NORMAL);
...

Document Compression

Documents can be stored as a compressed form when they are written in the disk. ForestDB provides a compression option in fdb_config, that compresses and decompresses the body of documents using the Snappy library. Snappy does not provide optimal compression, but it's very fast. You can enable the option as follows:

fdb_handle *db;
fdb_status status;
fdb_config config;

config = fdb_get_default_config();
config.compress_document_body = true;
status = fdb_open(&db, "/tmp/db_filename", &config);
...

Retrieving A Document From A Database

ForestDB provides two kinds of retrieval methods: retrieval by key and retrieval by sequence number. If you want to get a document using its key, create a document with empty meta and body fields, and call fdb_get:

fdb_handle *db;
fdb_doc *doc;
fdb_status status;
...

status = fdb_doc_create(&doc, "foo", 3, NULL, 0, NULL, 0);
status = fdb_get(db, doc);
...

status = fdb_doc_free(doc);
...

If the target document is found, then the meta and body fields of doc will be filled with those of the corresponding document.

A document can also be retrieved by its sequence number, which is a value assigned to it whenever it's created or updated, taken from a monotonically-increasing sequence counter maintained by the database. It is a 64-bit integer defined as the fdb_seqnum_t type, and available as the seqnum field of an fdb_doc after it's been saved by fdb_set.

To get a document by its sequence number, create an empty document, assign a sequence number to it, and call fdb_get_byseq:

fdb_handle *db;
fdb_doc *doc1, *doc2;
fdb_status status;
fdb_seqnum_t seqnum;
...

status = fdb_doc_create(&doc1, "foo", 3, "00", 2, "bar", 3);
status = fdb_set(db, doc1);
seqnum = doc1->seqnum;
status = fdb_commit(db, FDB_COMMIT_NORMAL);
status = fdb_doc_free(doc1);
...

status = fdb_doc_create(&doc2, NULL, 0, NULL, 0, NULL, 0);
doc2->seqnum = seqnum;
status = fdb_get_byseq(db, doc2);
...

status = fdb_doc_free(doc2);
...

Reading Metadata of A Document

To read a document's metadata only, without its body, ForestDB provides two operations: fdb_get_metaonly and fdb_get_metaonly_byseq. Their usage is same as those of fdb_get and fdb_get_byseq:

fdb_handle *db;
fdb_doc *doc;
fdb_status status;
...

status = fdb_doc_create(&doc, "foo", 3, NULL, 0, NULL, 0);
status = fdb_get_metaonly(db, doc);
...
fdb_handle *db;
fdb_doc *doc;
fdb_status status;
fdb_seqnum_t seqnum;
...

status = fdb_doc_create(&doc, NULL, 0, NULL, 0, NULL, 0);
doc->seqnum = seqnum;
status = fdb_get_metaonly_byseq(db, doc);
...

In the case of fdb_get_metaonly_byseq, both key and metadata are returned by calling the API.

After the call returns, the length of the body is available in the bodylen field, even though the body field is NULL.

Afterwards, the document's body can be read very efficiently by calling fdb_get_byoffset, as described in the next section.

Retrieving A Document Using Disk Offset

Documents also can be retrieved by their disk offset, which is literally the byte offset from the start of the database file to the position where the document is stored. This is very efficient because it bypasses the tree entirely, requiring only a simple read operation.

Each document contains an offset field (a 64-bit unsigned integer) which is set by the fdb_set, fdb_get, fdb_get_byseq, fdb_get_metaonly, and fdb_get_metaonly_byseq functions. To get a document using disk offset, create an empty document, assign a disk offset to it, and call fdb_get_byoffset:

#include <stdint.h>
...

fdb_handle *db;
fdb_doc *doc1, *doc2;
fdb_status status;
uint64_t offset;
...

status = fdb_doc_create(&doc1, "foo", 3, "00", 2, "bar", 3);
status = fdb_set(db, doc1);
offset = doc1->offset;
...

status = fdb_doc_create(&doc2, NULL, 0, NULL, 0, NULL, 0);
doc2->offset = offset;
status = fdb_get_byoffset(db, doc2);
...

Deleting A Document From A Database

Deleting a document is similar to updating it. The following example shows how to delete a document:

fdb_handle *db;
fdb_doc *doc;
fdb_status status;
...

status = fdb_doc_create(&doc, "foo", 3, NULL, 0, NULL, 0);
status = fdb_del(db, doc);
status = fdb_commit(db, FDB_COMMIT_NORMAL);
status = fdb_doc_free(doc);
...

Adjusting Purging Interval

Note that documents deleted using fdb_del are not physically removed at that time (since the database file is append-only). ForestDB lazily purges those documents upon the next compaction after the period configured in fdb_config. Although the deleted documents cannot be retrieved by fdb_get or fdb_get_byseq, you can still read their metadata by calling fdb_get_metaonly or fdb_get_metaonly_byseq until they are purged from the DB.

You can change the purging interval defined in fdb_config as follows:

fdb_handle *db;
fdb_status status;
fdb_config config;

config = fdb_get_default_config();
config.purging_interval = 3600;
status = fdb_open(&db, "/tmp/db_filename", &config);
...

The unit of the purging interval is in seconds. If set to zero (which is the default), deleted documents are immediately purged at the next compaction.

WAL Flush Threshold

Document updates in ForestDB are first added to the write-ahead log (WAL), and later reflected into the HB+trie. WAL entries are automatically flushed when fdb_commit is called, or when the total number of documents in the WAL exceeds a configured threshold in fdb_config. The default value is 4,096 documents, and you can change the threshold as follows:

fdb_handle *db;
fdb_status status;
fdb_config config;

config = fdb_get_default_config();
config.wal_threshold = 65536;
status = fdb_open(&db, "/tmp/db_filename", &config);
...

Manually Flushing WAL Entries

If you want to manually flush WAL entries before the threshold is reached, call fdb_commit with the FDB_COMMIT_MANUAL_WAL_FLUSH option, as follows:

fdb_handle *db;
fdb_status status;
...

status = fdb_commit(db, FDB_COMMIT_MANUAL_WAL_FLUSH);
...

Simplified Key-Value Operations

ForestDB provides simplified key-value operations, for those who do not want to use document-related features such as sequence number and metadata. There are three functions for key-value operations: fdb_set_kv, fdb_get_kv, and fdb_del_kv.

To insert a key-value pair, call fdb_set_kv:

fdb_handle *db;
fdb_status status;
...

status = fdb_set_kv(db, "foo", 3, "bar", 3);
status = fdb_commit(db, FDB_COMMIT_NORMAL);
...

By calling fdb_get_kv, you can get a value corresponding to a key:

fdb_handle *db;
fdb_status status;
void *value;
size_t value_len;
...

status = fdb_get_kv(db, "foo", 3, &value, &value_len);
...

free(value);

You should free the value returned from fdb_get_kv API by calling free.

Key-value pairs can be deleted using fdb_del_kv API:

fdb_handle *db;
fdb_status status;
...

status = fdb_del_kv(db, "foo", 3);
status = fdb_commit(db, FDB_COMMIT_NORMAL);
...

Since these key-value pairs are internally converted to and stored as documents, the simplified functions are compatible with the original APIs such as fdb_set and fdb_get. The following example gets the body of a document inserted using fdb_set, by calling fdb_get_kv.

fdb_handle *db;
fdb_doc *doc;
fdb_status status;
void *value;
size_t value_len;
...

status = fdb_doc_create(&doc1, "foo", 3, "00", 2, "bar", 3);
status = fdb_set(db, doc1);
status = fdb_commit(db, FDB_COMMIT_NORMAL);
...

status = fdb_get_kv(db, "foo", 3, &value, &value_len);
...

Using Iterators

An iterator traverses a DB instance in order by key or sequence number. It can also traverse a smaller range of keys or sequences. An index operates on a snapshot of the database, so any mutations made after the iterator is created have no effect on the documents returned by the iterator.

fdb_iterator_init and fdb_iterator_sequence_init create an iterator for keys and sequence numbers, respectively:

fdb_handle *db,
fdb_iterator *itr_key, *itr_seq;
fdb_status status;
...

status = fdb_iterator_init(db, &itr_key,
                           NULL, 0, NULL, 0,
                           FDB_ITR_NONE);
assert(status == FDB_RESULT_SUCCESS);
...

status = fdb_iterator_sequence_init(db, &itr_seq,
                                    0, 0, FDB_ITR_NONE);
assert(status == FDB_RESULT_SUCCESS);
...

You can assign a specific range to the iterator, by passing start and end values of the range. The following example creates an iterator traversing from key aaa to key zzz (inclusive):

...
status = fdb_iterator_init(db, &itr_key,
                           "aaa", 3, "zzz", 3,
                           FDB_ITR_NONE);
...

If you pass NULL as the start key, the iterator starts at the first (smallest) key. If the end key is NULL, the iterator terminates after the last (largest) key. If the start key does not exist in the DB instance, the iterator begins from the smallest key that is greater than the start key. If the end key does not exist in the DB instance, the iterator ends at the greatest key that is smaller than the end key.

The last parameter to fdb_iterator_init is a set of option flags. The available options are to return only document metadata not bodies (FDB_ITR_METAONLY), and to omit deleted documents from the iteration (FDB_ITR_NO_DELETES).

fdb_iterator_sequence_init follows the same semantics. The example below shows how to create an iterator for the range of sequence number from 10 to 20:

...
status = fdb_iterator_sequence_init(db, &itr_seq,
                                    10, 20, FDB_ITR_NONE);
...

Once an iterator is created, you can get the documents from the iterator by calling fdb_iterator_next, regardless of the type of the iterator. fdb_iterator_next returns FDB_RESULT_SUCCESS if the next document for the iterator exists; in that case its doc parameter will point to an fdb_doc filled in with the contents of the next document. (As usual, this document structure should be freed when you're done using it.)

fdb_iterator_next returns FDB_RESULT_ITERATOR_FAIL when it reaches the end of the range. When finished, you should close the iterator by calling fdb_iterator_close.

fdb_handle *db,
fdb_doc *doc;
fdb_iterator *itr_key;
fdb_status status;
...

status = fdb_iterator_init(db, &itr_key,
                           NULL, 0, NULL, 0,
                           FDB_ITR_NONE);
while(fdb_iterator_next(itr_key, &doc) == FDB_RESULT_SUCCESS) {
    ...
    fdb_doc_free(doc);
}
status = fdb_iterator_close(itr_key);
...
fdb_handle *db,
fdb_doc *doc;
fdb_iterator *itr_seq;
fdb_status status;
...

status = fdb_iterator_sequence_init(db, &itr_seq,
                                    0, 0, FDB_ITR_NONE);
while(fdb_iterator_next(itr_seq, &doc) == FDB_RESULT_SUCCESS) {
    ...
    fdb_doc_free(doc);
}
status = fdb_iterator_close(itr_seq);
...

If you want to move an iterator forward to get a document for a specific key, call fdb_iterator_seek:

fdb_iterator *itr_key;
fdb_status status;
fdb_doc doc;
...

status = fdb_iterator_seek(itr_key, "car", 3);
status = fdb_iterator_next(itr_key, &doc);
...

Afterwards, the next call to fdb_iterator_next will return the document of the key that you sought to.

If the given key does not exist, fdb_iterator_seek moves to the smallest key greater than the given key.

If you want to move an iterator backward, call fdb_iterator_prev:

status = fdb_iterator_prev(itr_key, &doc);
...

Iterator can move in both direction, and can seek in either direction.

Getting Information of A DB Instance

You can get some information about a DB instance using fdb_get_dbinfo:

fdb_handle *db;
fdb_info info;
fdb_status status;
...

status = fdb_get_dbinfo(db, &info);

The return type is a fdb_info struct that contains the following information:

  • filename: The filename of the DB instance.
  • new_filename: The name of the new file, if the DB instance is being compacted.
  • last_seqnum: The last sequence number of the DB instance.
  • doc_count: The approximate total number of documents stored in the DB instance.
  • space_used: The approximate space occupied by live data in the DB instance.
  • file_size: The total size of the DB file.

Creating A Snapshot

ForestDB can create a snapshot of a specific DB instance, using a sequence number that corresponds to one of the commits that have been persisted in the same database instance. A snapshot is a read-only immutable DB handle: subsequent changes to the database will have no effect on the snapshot.

The following example shows how to create a snapshot:

fdb_handle *db, *snapshot,
fdb_status status;
fdb_info info;
fdb_seqnum_t seqnum;
...

status = fdb_commit(db, FDB_COMMIT_NORMAL);
status = fdb_get_dbinfo(db, &info);
seqnum = info.last_seqnum;
status = fdb_snapshot_open(db, &snapshot, seqnum);
...

status = fdb_close(snapshot);

A snapshot is treated as a read-only DB handle, thus it must be closed with fdb_close when it is not needed anymore.

Rollback

You can rollback a DB instance to a specific point represented by a sequence number that corresponds to a commit. fdb_rollback reverts a DB instance as follows:

fdb_handle *db,
fdb_status status;
fdb_info info;
fdb_seqnum_t seqnum;
...

status = fdb_commit(db, FDB_COMMIT_NORMAL);
status = fdb_get_dbinfo(db, &info);
seqnum = info.last_seqnum;
...

status = fdb_rollback(&db, seqnum);
...

Note that all other mutations of the DB instance are blocked while the rollback operation is being performed.

Using Transactions

ForestDB supports transactional features which makes it possible to atomically update multiple documents. You can begin a transaction by calling fdb_begin_transaction, and commit dirty updates by calling fdb_end_transaction. Here's a simple example that atomically updates doc1 and doc2:

fdb_handle *db;
fdb_doc *doc1, *doc2;
fdb_status status;
...

status = fdb_begin_transaction(db, FDB_ISOLATION_READ_COMMITTED);
status = fdb_set(db, doc1);
status = fdb_set(db, doc2);
status = fdb_end_transaction(db, FDB_COMMIT_NORMAL);
...

There are two isolation levels supported by the current version of ForestDB:

  • FDB_ISOLATION_READ_COMMITTED: Read committed isolation level. Dirty updates (i.e., uncommitted updates) by other transactions are not visible, although updates committed by other transactions will be visible.
  • FDB_ISOLATION_READ_UNCOMMITTED: Read uncommitted isolation level. Dirty updates by other transactions are visible.

If you want to revert all dirty updates after fdb_begin_transaction, invoke fdb_abort_transaction instead of fdb_end_transaction:

fdb_handle *db;
fdb_doc *doc1, *doc2;
fdb_status status;
...

status = fdb_begin_transaction(db, FDB_ISOLATION_READ_COMMITTED);
status = fdb_set(db, doc1);
status = fdb_set(db, doc2);
status = fdb_abort_transaction(db);
...

Multi-Threaded Accesses

Since ForestDB is based on an MVCC model, concurrent read and update operations do not block each other. All ForestDB operations are thread-safe, but you should open a separate handle on a DB file for each thread. The fdb_handle structure is not thread-safe, and ForestDB does not guarantee thread-safety if multiple threads share one DB handle. The following example shows how to concurrently access to a DB instance:

  • Thread 1
fdb_handle *db_thread1;
fdb_status status;
fdb_config config;
...

status = fdb_open(&db_thread1, "/tmp/db_filename", &config);
...

status = fdb_close(db_thread1);
  • Thread 2
fdb_handle *db_thread2;
fdb_status status;
fdb_config config;
...

status = fdb_open(&db_thread2, "/tmp/db_filename", &config);
...

status = fdb_close(db_thread2);

The threads access the same DB instance /tmp/db_filename using their own DB handles (db_thread1 and db_thread2), and they do not need to grab a lock on the shared DB instance when they invoke ForestDB operations.

Note: Multiple handles on the same database file share the same underlying file caches, so the memory overhead per extra handle is low.

Compaction

As ForestDB uses append-only style disk logging, the overall space occupied by a DB instance grows with more mutations. This fragmented DB instance should be compacted to reclaim the space used by stale data. You can manually compact a DB instance by calling fdb_compact:

fdb_handle *db;
fdb_status status;
fdb_config config;
...

status = fdb_open(&db, "/tmp/db_filename", &config);
...

status = fdb_compact(db, "/tmp/db_filename2");
...

After compaction, the previous file /tmp/db_filename is removed and new file /tmp/db_filename2 is created. All DB handles referring to the DB instance automatically change their information to point to the new file, thus it is not necessary to close the existing DB handles and open the new file.

Note that you do not need to manually compact the DB instance in auto-compaction mode, but manual compaction is possible by calling fdb_compact with a NULL filename as follows:

fdb_handle *db;
fdb_status status;
fdb_config config;
...

config = fdb_get_default_config();
config.compaction_mode = FDB_COMPACTION_AUTO;
status = fdb_open(&db, "/tmp/db_filename", &config);
...

status = fdb_compact(db, NULL);
...

Switching Compaction mode

DB instances created under auto-compaction mode cannot be opened under manual-compaction mode, and vice versa. To open a DB instance based on different compaction mode, it has to switch its compaction mode using fdb_switch_compaction_mode API:

fdb_handle *db;
fdb_status status;
fdb_config config;
...

config = fdb_get_default_config();
config.compaction_mode = FDB_COMPACTION_AUTO;
status = fdb_open(&db, "/tmp/db_filename", &config);
...

status = fdb_switch_compaction_mode(db, FDB_COMPACTION_MANUAL, 0);
status = fdb_close(db);

config.compaction_mode = FDB_COMPACTION_MANUAL;
status = fdb_open(&db, "/tmp/db_filename", &config);
...

Changing Compaction Threshold

fdb_switch_compaction_mode also can be used for changing the compaction threshold for the given DB instance whose compaction mode is currently auto-compaction. The following example changes the compaction threshold of the DB instance from 30% to 50%:

fdb_handle *db;
fdb_status status;
fdb_config config;
...

config = fdb_get_default_config();
config.compaction_mode = FDB_COMPACTION_AUTO;
config.compaction_threshold = 30;
status = fdb_open(&db, "/tmp/db_filename", &config);
...

status = fdb_switch_compaction_mode(db, FDB_COMPACTION_AUTO, 50);
...

The compaction threshold is calculated as stale_data_size / total_file_size. For example, when the compaction threshold and live data size are 30% and 70 MB, respectively, compaction is triggered when the DB file size gets larger than 100 MB, since the stale data size (i.e., 100 MB - 70 MB = 30 MB) exceeds 30% of the file size.

Note that compaction will not be executed if you set the compaction threshold to zero or 100%.

Using A Custom Comparison Function

ForestDB supports pluggable custom comparison function for both fixed-size primitive type keys and variable-length keys. A custom comparison functions is given pointers to two keys, key1 and key2 for example, and must return zero if the keys are equal, a negative value if key1 < key2, or a positive value if key1 > key2.

The following code represents an example that uses a custom comparison function for floating-point numbers represented as the double primitive type:

  • Custom comparison function for double type:
int _cmp_double(void *key1_ptr, void *key2_ptr)
{
    double key1, key2;
    key1 = *(double *)key1_ptr;
    key2 = *(double *)key2_ptr;

    if (key1 < key2) return -1;
    else if (key1 > key2) return 1;
    else return 0;
}
fdb_handle *db;
fdb_doc *doc;
fdb_status status;
fdb_config config;
double key;
void *value;
size_t value_len;
...

config = fdb_get_default_config();
config.cmp_fixed = _cmp_double;
status = fdb_open_cmp_fixed(&db, "/tmp/db_filename", &config);
...

status = fdb_doc_create(&doc,
                        (void *)&key, sizeof(double),
                        NULL, 0, value, value_len);
status = fdb_set(db, doc);
status = fdb_doc_free(doc);
...

Note: This example is not endian-safe, so a database created on a little-endian CPU would not be readable on a big-endian CPU! Any real usage of custom key types should be careful to represent keys with a consistent byte ordering.

The next example shows how to define and use a custom comparison function for variable-length keys:

  • Custom comparison function that compares only last four bytes in each key:
int _cmp_substring(void *key1, size_t keylen1,
                   void *key2, size_t keylen2)
{
    if (keylen1 < 4 || keylen2 < 4) return 0;

    return memcmp((char*)key1 + keylen1 - 4,
                  (char*)key2 + keylen2 - 4, 4);
}
fdb_handle *db;
fdb_status status;
fdb_config config;
...

config = fdb_get_default_config();
config.cmp_variable = _cmp_substring;
status = fdb_open_cmp_variable(&db, "/tmp/db_filename", &config);
...

Destroying database file(s)

If you wish to clean up all the files associated with a ForestDB instance, you can call the fdb_destroy api as follows..

fdb_status status;
fdb_config config;
...

config = fdb_get_default_config();
status = fdb_destroy("/tmp/db_filename", &config);
...

The above routine can be useful to clean up all ForestDB files including meta data files for an auto compaction setting. If the destroyed file is currently being compacted by the auto compaction daemon which is still running, then fdb_destroy will return FDB_RESULT_IN_USE_BY_COMPACTOR, and the operation would need to be retried after the file compaction completes. In case of manual compaction a best-effort is done to wipe out all known files associated with the ForestDB path provided.

Clone this wiki locally