Stale-while-error functionallity

[TBeijen]

We use cacheback a lot for async 'always fast' fetching of data that is requested regularly. Example: Our API exposes weather info. This data is fetched from a 3rd party vendor API. Occasionally that API has quirks for a certain (usually short) period, which (within certain boundaries of course) we want to hide by returning stale data.

If I understand correctly, once a request is made outside of lifetime but within cache_ttl, the entry stored in cache is replaced by one with ttl of timeout.

So, if request somehow fails, after 'timeout', the cached entry is gone as well, and errors will be visible.

I'd like to elegantly implement a stale-while-error mechanism in Cacheback.

Reference: Similar functionality in Fastly (basically varnish-as-a-service) and Nginx

Haven't looked into implementation but some first thoughts:

• Similar to the fetch() method that needs to be implemented on a Job subclass, provide a method that can be implemented where errors can be handled. handle_error_while_stale() for example.
• If not using a custom Job class, some way to provide a callable that decorates fetch(), catches errors and re-raises something like AllowedWhileStaleError

Do you see any value in this?

[stephrdev]

When a cached item is end of life, the current value will be cached again for refresh_timeout seconds. An async task is issued which will update the value and cache for lifetime seconds. If the refresh fails, it will be retried every refresh_timeout seconds until successful executed. I think this should solve your issue.

For example: lifetime = 300 (5m), refresh_timeout = 30 -> cache weather for 5m, after the lifetime remember the data for another 30 seconds and issue refresh task. If the task fails, cacheback will retry to refresh async after 30 seconds, and again, and again. If the async tasks finishes, the data will be cached for 5m again.

[TBeijen]

Ok, I'm gonna look into that. Based on findings I might issue a PR that describes error-scenario's in docs.

[nara-l]

@TBeijen did you write the PR for error-scenarios? Also, is there already an implementation to fetch stale data if celery fails? cc: @stephrdev

[stephrdev]

If Celery is down, the same happens as desribed earlier (stale cache kept and extended until a new cache version was generated).

[nara-l]

@stephrdev thank you for the quick response. Is refresh_timout same as timeout ?