Race condition in cached function

[lenaqr]

(originally learning-unlimited/ESP-Website#1463)

If a cached function takes a long time to compute, there is a possible race condition where a database update happens during the evaluation of the cached function, and a stale result gets stored in the cache. This is likely the cause of the bug where the catalog displays a class's enrollment as (e.g.) 19/20 when it is actually full.

Example where this might happen:

• user 1 requests the catalog, which gets num_students for each class
• user 2 registers for a class, updating num_students, and invalidating the (previous) cached catalog
• user 1's catalog finishes rendering, and is stored in the cache

A separate, but related issue happens when multiple users simultaneously request the same cached function, causing it to be evaluated a bunch of times and possibly slowing the site (see explanation of the "thundering herd problem" here: https://github.com/ericflo/django-newcache)

Some ideas/links that may be useful:

• store the timestamp when the cache was last touched (updated or invalidated), check against the timestamp?
• store a token when starting to compute a cached function, check if it's still there?
• http://bluxte.net/musings/2009/10/28/simple-distributed-lock-memcached
• http://stackoverflow.com/questions/7917604/django-memcache-compare-and-set
• #1462 may help in the meantime before this is fixed.