On Cache Invalidation - Why is it hard? | /en/2018/06/cache-invalidation/

[yihui]

On Cache Invalidation - Why is it hard?

https://yihui.org/en/2018/06/cache-invalidation/

[giscus-bot]

Guest *Nicholas Tierney* @ 2018-06-26 08:09:47 originally posted:

Thanks for writing that up, Yihui! I definitely understand why this is hard now, and it is nice to get an insight into cool ways to watch data with things like:

```{r import-data, cache=TRUE, cache.extra=tools::md5sum('my-precious.csv')}  
d = read.csv('my-precious.csv')  
```

[giscus-bot]

Guest *Rainer M Krug* @ 2018-08-08 07:47:58 originally posted:

Thanks for this writeup - it explains a lot and I learned about the cache.extra argument - very useful!

There are other tools with likely more complicated approaches of caching results (e.g. drake). It could be useful, to think about using e.g. drake for this ask, especially as it would allow easy parallel processing?

[giscus-bot]

Guest *sandy* @ 2020-07-22 17:07:00 originally posted:

Can't believe I found this while I am working. Great article.
From Badminton fellow.

[refeed]

this is interesting

[rosedo]

I agree with everything in the article.

There will be times when you feel the need to cache something but knowing when to invalidate the cache IS hard. You most likely aren't using a cutting-edge database that alerts you when some data has changed. You may not be coding a single-page app that can fetch data at the last moment and therefore you're fetching it too often (and potentially too early to make matters worse).

There are ways to make invalidating cache easier though, like invalidating a whole cache if any random tiny thing is updated anywhere, or by giving the cache a short lifespan with an expiration date.

Some battles are not worth fighting, even if you win.