Eliminate eviction process; use same connection

[JakeSiFive]

This change eliminates the eviction process which is now quite apparently an over optimization IMO. With this new change there will be small pauses in wake here and there while the eviction tasks are occurring but on the plus side we now have a single DB connection, guarded by a file system lock, per cache.

Additionally I spotted a couple cases where I was improperly using the database on two separate threads which is a big no no for SQLite and may have been causing some of our issues. I've removed those at additional cost (but it should be a small cost).

With these changes things should be both easier to debug, and possibly more stable.

And remember kids, friends don't let friends write multi-threaded C++ code when Rust is an option ;)

[colinschmidt]

This looks correct to me.
Did we see how long the pauses are when cleaning a large cache? Is it in seconds, ms, us?

[JakeSiFive]

Did we see how long the pauses are when cleaning a large cache? Is it in seconds, ms, us?

I didn't check but it should be in the low ms with TTL. For TTL it does a query over the jobs to find everything that has expired but there's an index on that time so it should be fast. It then goes and unlinks just the inodes associated with those jobs. The P95 might be hundreds of milliseconds and the P99 might be several seconds (try rm -rf ing a cache and see how long it takes!) The current LRU implementation which is not tested or confirmed to have been made to work by this change would be longer more abrupt pauses.

These pauses can be mostly eliminated by having a queue for things to free that's processed by a separate thread but I've learned my lesson about premature optimization on this project so I'll only do that if its required 😄

[colinschmidt]

LGTM assuming the cache doesn't frequently get stuck for long periods during evictions. Where frequently and long is something like 99% are less than 10 seconds.

[JakeSiFive]

I'll measure to confirm but I'll go ahead and land this for testing purposes