Thoughts: motivations, what it's not

Motivations

Project Mentat is specifically intended to address deficiencies in existing embedded storage systems:

• Schema evolution is fiddly and error-prone, but it's a frequent event because schemas change alongside product features.
• Schema and data sharing between modules is risky and difficult. (E.g., Feature A depends on Feature B's schema, and will break — perhaps undetectably — if it changes.)
• Unit schema/data ownership — that each database holds one module's data, and has one schema, and is managed by a single domain-specific code module — makes it unnatural to combine data across modules. We can't arbitrarily encrypt your bookmarks like we can your passwords. We can't query for sites in your history that have saved passwords in the login manager, etc.
• Traditional single-schema systems (as opposed to CQRS) don't cope well with tensions between writers and different readers. E.g., storing frecency in-line with history makes Places hard to change and limits possible approaches to improve performance. Event-shaped data is a powerful concept, but existing approaches require developers to do all the work, so they don’t.
• Join-heavy queries are extremely hard to write correctly in SQL. (Non-SQL alternatives have their own problems around scaling, querying, durability, and concurrent access.)
• Full-text indexing is inconvenient and requires specialized knowledge.
• Change observing is an afterthought, either through Gecko notifications driven by API code itself, or having to be expressed as database triggers written by 'wizards', which are unable to have side-effects outside of the database.
• Syncing and data replication are ignored, or ad hoc domain-specific versions are built after the fact.

In short: if you start with SQLite and try to solve schema sharing, easy FTS, add some helpers for CQRS, and the rest, you end up building a hacky version of Mentat.

What it's not

• Not SQL. Queries and data are expressed via data structures.
• Not NoSQL, as commonly understood. Mentat doesn't expose SQL, so technically "NoSQL" is correct, but Mentat is relational (though not tabular), with rich queries, rich types, and immediate consistency. Most NoSQL stores are understood to sacrifice relational data models and weaken consistency guarantees in order to address scale and distributed storage.
• Not an ORM. Data and results are data-shaped, not object-shaped.
• Not just a graph store. Queries aren't expressed through paths, though paths can be part of them.
• Not a document store.
• Not schemaless. Schemas are an important part of thinking about a domain and characterizing storage and querying.