Rest API and Caching

[Mike.Sheen]

The other thing we need to do, which is why headers including cache control information is important, is to make sure our caching works well with upstream proxy caches.

Ideally you'd want to be able to stand up a proxy in front of your Jiwa REST API, and have it bear the brunt of requests, but pass through requests not cached by the proxy cache, and the REST API downstream to appropriately handle the request and if need be inform the upstream proxy cache in some way of any responses it should be now caching.

dawg_cache.jpg (51.9 KiB) Viewed 2339 times

[SBarnes]

How is jiwa-stateful going to work with all of this?

[Mike.Sheen]

How is jiwa-stateful going to work with all of this?

I worked that out today, actually. In essence the jiwa-stateful requests skip the cache, because it has to or things get weird. A SaveRequest which is stateful however, will push into the cache the new DTO, but every other stateful operation skips the cache and behaves as before.

[Mike.Sheen]

The other thing I discovered today was that any persistent cache (such as Redis) needs to flush session information when the REST API service starts.

Otherwise, when you restart the REST API, the Redis instance might still have cache keys for sessionIds not yet expired, and should a client make a request using one of those sessionIds then you get a horrible 500 error because the REST API gets a null out of the JiwaSessionDictionary.

So, the Redis and Memcache plugins have now an extra 4 lines of code in there to flush all session/auth keys when starting up.

I also discovered the eviction policies of Redis and Memcache - ballooning memory requirements are managed quite nicely if you set an eviction policy of Least Recently Used (LRU) or Least Frequently Used (LFU) - when the cache is close to using the configured max memory, when LRU or LFU policies are employed, they automagically evict other cache entries to make room for new cache entries, which I thought was pretty neat.

[SBarnes]

I can definitely tell you from having used Redis for session management that it doesn't support any sort of session end event which backs up what you are saying.

So, the Redis and Memcache plugins have now an extra 4 lines of code in there to flush all session/auth keys when starting up.

Would it be worthwhile having some sort of api call to support this, that way a plugin could make the call either on a schedule basis to help maintain the cache, or something could be added to the user interface say in system settings to let the user make this happen, I did a similar thing in system settings to restart the api and reread the system setting in the extension for our friends who went live in February.

[Mike.Sheen]

Would it be worthwhile having some sort of api call to support this, that way a plugin could make the call either on a schedule basis to help maintain the cache, or something could be added to the user interface say in system settings to let the user make this happen, I did a similar thing in system settings to restart the api and reread the system setting in the extension for our friends who went live in February.

I am planning to have APIs to manage cache entries, and as a stretch goal have a user interface in the Jiwa application to consume these APIs to manage the cache entries (show a filterable list, with the option to delete an entry).

Having said that, the session cache keys do have an expiry so they'll automatically be evicted from the cache when they do expire, and we clean up the Jiwa side of things (orphaned Manager instances in the JiwaSessionDictionary) by looking for all JiwaSessionDictionary entries which don't appear in the caches session cache entries - we do that each time a request is made.

Previously I was lamenting the fact that DTO's cached as a result of requests made using a Debtor API Key would have to have their key salted with the Debtor API Key, lest we return a DTO previously cached as a result of a non-Debtor API Key request. As it turns out, despite actually implementing that and getting it working just fine, I found it was all for nought as even if the DTO cached is the full DTO, when a request is made using a Debtor API Key to retrieve the DTO, the response filters we already have in place will sanitise the DTO and remove the naughty bits anyway.

So - time was wasted putting all that logic in worrying about unwanted information disclosure, and time was then wasted undoing all that - but at least it's back to being a cleaner solution.

I've done the cache invalidation / refresh / delete when users of the Jiwa application make changes to a record - it piggybacks off what we did with webhooks - so we have business logic handlers for the SaveEnd of all relevant business logic objects and that will call an internal route of the API to update or delete (if they were deleting a record) any cached entries... meaning if caching is enabled, using the Jiwa application will keep the cache up-to-date. We only update cache entries when changes are made from the Jiwa application IF and ONLY IF there was already a cache entry. Cache entries are only added when consumers of the API make requests which results in a DTO being stored in the cache. This strategy is to keep the cache from eventually just being a cache of the entire Jiwa database - we only cache what we need.

I'm about to start on tackling AutoQuery caching - that should be quite a challenge.

[SBarnes]

I don't know if you remember but while back we had been talking about the Auto query annotation [QueryDb(QueryTerm.Or)] which allows for any of the conditions to be true rather than all and you were going to look at generating the necessary classes as part of the build process probably through T4 I would assume.

I am not looking to make it happen as part of this as it probably is still in the back logs but I thought it worth mentioning it again given you said you were going on to look at caching for Auto Query next in case it may impact things.

[Mike.Sheen]

I don't know if you remember but while back we had been talking about the Auto query annotation [QueryDb(QueryTerm.Or)] which allows for any of the conditions to be true rather than all and you were going to look at generating the necessary classes as part of the build process probably through T4 I would assume.

Yes, that's still in our backlog: DEV-7057

I am not looking to make it happen as part of this as it probably is still in the back logs but I thought it worth mentioning it again given you said you were going on to look at caching for Auto Query next in case it may impact things.

I don't think it will have a significant impact in how we handle caching AutoQuery routes... but I've been known to be wrong before!

[SBarnes]

Ok we have some of these types of routes anyway so we should be able to see if it breaks them.

Two other things I had also thought of (excuse the constant questions):

1. Where in the pipeline will the caching occurring?

2. What's going to be the case of something that returns a file like a report/pdf is it going to be possible to cache that, which I am not sure is a good idea given you could bloat a cache fairly quickly with big response sizes?

[Mike.Sheen]

1. Where in the pipeline will the caching occurring?

Good question! At the moment it's within each service method - a little bit of code in the GETs to wrap existing code with a function call and pass the existing code as a lamda - as shown in this post. For POST, PATCH and DELETE operations it's a line of code at the end to pass the DTO to a method which updates the cache if the route is registered to be cached.

However, I may be able to move that into request and response filters... so that would mean absolutely no code changes needed to existing service methods - it would all be abstracted away and work transparently. I'll get to exploring that once I've covered off all the different types of requests we typically have - I don't want to go and modify the hundreds of existing service methods to explicitly support caching so making it global request and response filters is the holy grail I'm going for.

2. What's going to be the case of something that returns a file like a report/pdf is it going to be possible to cache that, which I am not sure is a good idea given you could bloat a cache fairly quickly with big response sizes?

You can opt-in to caching for any request, so that will work for report PDF generation also (it's just a byte array) - if cache bloat is a concern the eviction policies and the optional cache entry expiry time will help with that, and you can also just not cache requests which return PDF's. If using the in-memory cache then cache bloat is going to be a real concern, because it will be easy to exceed the memory allowed for the process - but other cache providers are be able to handle large cache sizes and have good eviction policies to prevent exhaustion of the available memory.

On cache providers, besides MemCache and Redis we might have some other options built-in - such as Azure Table Storage and AWS DynamoDB - ServiceStack have those packages available so we might as well ship with support for those as well.

excuse the constant questions

No problem! - it's always good to have such questions early on in the development process, rather than when the thing is baked.