Executing SQL synchronously via JSI

[scarlac]

A few other discussions have touched on this, but in transitioning from Redux(-persist) to WatermelonDB, it would be extremely helpful to be able to temporarily execute raw SQL queries synchronously. This allows a gradual transition to proper model-based/Q-based WatermelonDB queries.

This is roughly what I've come to so far:

// Where the database is set up and initialized:
export const adapter = new SQLiteAdapter({ ... });

// Where the synchronous query is needed:
const _db = adapter._dispatcher._db;
const res = _db.unsafeQueryRaw(`SELECT id, bar FROM foo`, []);
console.log('sql', res);

Any guidance on how to do it more smoothly / catch any pitfalls? So far I've been trying to reverse engineer it all but the amount of adapter code makes it unclear what code I'm actually calling and the need for decoding/encoding parameters, results, caching, etc.

I found that simply calling _db.query(...) was an issue because the result gets cached so the second the the query runs, the resultset turns into a JSI cache "id" instead of the actual data.
(Ideally a public API would be great, but I understand the reasoning from another issue where this was requested)

Some specific questions to get started:

• How do I get the cached result back?
• Is there a way to access the adapter's _db without using the private fields like this?

Any guidance would be helpful.

[scarlac]

I'm sharing my solution, as it may be handy for others doing a transition. It assumes the availability of:

1. const adapter = ... referring to the SQLiteJSI adapter
2. const schema = ... referring to the appSchema definition

/**
 * Execute a raw SQL 'SELECT' query to the SQLite database.
 * You must always start with 'SELECT *' (all columns) to allow hydration (relation data => object)
 * SELECTs are safe, but UPDATE, DELETE, and INSERTs must follow up with a call to rawClearCache()!
 * otherwise data goes stale in other places
 * @param table Table name, used as part of the cache key (keyed as tableName@rowId)
 **/
export function rawQuery<T = any>(table: string, sql: string, args: any[] = []) {
  const _db = (adapter as any)._dispatcher._db;

  // Call adapters/sqlite/makeDispatcher/index.native's SqliteJsiDispatcher's "this._db" directly
  // (it calls directly into native using JSI)
  // Call @nozbe/watermelondb/adapters/sqlite's adapter (aka dispatcher) directly
  // WatermelonDB uses a queue and a 'dispatcher'. The db adapter dispatches to the queue,
  // and then the adapter later picks up each query, processes it, and calls the callback
  const res = _db.queryAsArray(
    table,
    sql,
    args,
  );
  const decoded = decodeQueryResult(res);
  const sanitizedResults = sanitizeQueryResult(decoded, schema.tables[table]);

  const collection: any = database.collections.get(table);
  const out = collection._cache.recordsFromQueryResult(sanitizedResults);

  return out as T[];
}

And combined with a cache clearing hack, it can be helpful:

export function rawClearCache(table: string, ids: string[]) {
  const _db = (adapter as any)._dispatcher._db;

  // Clear native cache by faking a "batch update" that does nothing
  const CACHE_BEHAVIOR = -1; // -1 means 'delete' from cache
  const ops: any[] = [];
  ids.forEach(id => {
    ops.push([CACHE_BEHAVIOR, table, `UPDATE "${table}" SET id = ? WHERE 1 = 0`, [[id]]]);
  });
  _db.batchJSON(JSON.stringify(ops));

  // Clear JS object cache
  const collection: any = database.collections.get(table);
  collection._cache.unsafeClear();
}

[radex]

@scarlac

db.get('table')._unsafeFetchRaw(query, results => { ... })

is the simplest (but undocumented and without any guarantees of API stability) way of synchronously getting results for a raw sql query

[scarlac]

A callback would be asynchronous though?

[radex]

No, synchronous under JSI