Rest API and Caching

[Mike.Sheen]

With regards to my whinge about [CompressResponse] being mutually exclusive to caching above, there is an alternative but it means the opt-in approach gets eliminated and clients instead can control if they want a cached response or not.

Cached responses returned by ToOptimizedResultUsingCache include in the response header the Last-Modified - which is the date of the cache entry (eg: Fri, 15 Nov 2019 00:55:05 GMT).

If clients include in their request header If-Modified-Since the server will return a 304 and an empty response if the If-Modified-Since is before the date of the cached entry, otherwise the server skips the cache, returns a 200 and updates the cache entry with the new dto and current time.

So by specifying a time in the future in the If-Modified-Since request header, clients can always skip the cache.

[SBarnes]

My only question is the fundamental one of who is in charge?

Traditionally I would have viewed the server controls the caching and not the client as it removes the complexity from the client needing to know it's even in place, also what happens if a client can't support caching in terms of what you have described?

It would be nice to get the best of both worlds i.e. both alternatives available but from what you are describing I don't think that might be possible.

Is there any way to simply approach this using something like your .net core solution, namely have something sit in front of the api and remove the functionality from the api and the danger of bloating the api but in some way having what sits in front more easily configurable to what gets cached or not rather than hard coded i.e. there might be an api route that the caching service uses to ask the api what it should cache or not cache at start up and a web hook could update this?

[Mike.Sheen]

As was once said by Phil Karlton:

There are only two hard things in Computer Science: cache invalidation and naming things

In my exploration of caching in the last few days, I've come across issues with caching which makes me appreciate this on a whole new level - including the "who is in charge?" question you posed.

For simplicity - let's assume the server is totally in charge - even then we have dilemmas.

For instance - we have a route to retrieve inventory items, which is filterable - GET https://api.jiwa.com.au/Queries/Invento ... No&Take=25

This would return 25 part numbers starting with the letter 'a'.

So, anyone calling that route will either get a cached result or a fresh result. Let's assume the cache expiry is set to 1 hour, so the first visit is fresh, thereafter it is cached until it expires in 1 hour and after that a fresh result is returned.

We also have our route to retrieve a single inventory item, GET https://api.jiwa.com.au/Inventory/{InventoryID} - the same rule applies with caching - so the first visit is fresh, thereafter it is cached until it expires in 1 hour and after that a fresh result is returned.

If someone in Jiwa updates the description for that product, the REST API plugin hooks into the Inventory SaveEnd event and updates the cache entry for that item (if it was cached) with the new value. No problem there - anyone requesting a GET on the https://api.jiwa.com.au/Inventory/{InventoryID} route using that ID will get the new value from the cache.

But what about the GET https://api.jiwa.com.au/Queries/Invento ... No&Take=25 route? It doesn't know about the change. And it would no longer have a valid response - the description for that item might be incorrect.

This is a trivial example, and product descriptions being stale by at worst 1 hour might not be worth worrying about, but one can easily imagine situations where this might become a problem - a product deleted for instance might still be able to be added to a webstore cart as the user could select it from a list returned by the InventoryItemList, but POSTing to the /SalesOrders route would throw an exception.

I'm just thinking out loud here - just expressing some of the issues we're grappling with.

[Mike.Sheen]

Is there any way to simply approach this using something like your .net core solution, namely have something sit in front of the api and remove the functionality from the api and the danger of bloating the api but in some way having what sits in front more easily configurable to what gets cached or not rather than hard coded i.e. there might be an api route that the caching service uses to ask the api what it should cache or not cache at start up and a web hook could update this?

Perhaps... We're still coming to terms with the scope of the problem, everything is on the table.

[Mike.Sheen]

Another issue I've come across is a security aspect.

When a customer API key is used, we would need to salt the key of the cache entry with the customer API key otherwise there would be unwanted information disclosure.

For example, the GET /Inventory/{InventoryID} gets intercepted by a request filter to strip out things like cost prices or customer specific prices for customers other than the one making the request.

That won't happen if a request is made by a non customer authorised request - so the cache of the full inventory item is stored with the key urn:inventory:{InventoryID}. If a customer then makes the same request, the cached entry is returned with no intervention by the filter to remove unwanted information.

This means longer keys (adding nanoseconds to cache lookups) and also a bigger cache.

FUN!

[SBarnes]

One other thing which may or may not need to be a consideration which also arises with session management is session affinity or sticky sessions, namely what if this needs to scale beyond one cache and one instance of the api to support the volume of traffic?

Of course under a cloud scenario you can use a load balancer to manage the traffic such as with Azure.

Again more FUN, this all of course gets into as you are no doubt aware the whole supporting of state or stateless and scalability argument whilst trying to maintain performance.

[Mike.Sheen]

One other thing which may or may not need to be a consideration which also arises with session management is session affinity or sticky sessions, namely what if this needs to scale beyond one cache and one instance of the api to support the volume of traffic?

Of course under a cloud scenario you can use a load balancer to manage the traffic such as with Azure.

Again more FUN, this all of course gets into as you are no doubt aware the whole supporting of state or stateless and scalability argument whilst trying to maintain performance.

Scaled out cache instances, distributed caches, CDN caches, et al and managing all that whilst providing integrity is an interesting problem. I think I can dream up 10x as many problems as there are proposed solutions. I guess it all comes down to finding a balance, and providing a way of detecting and managing issues.

Caching is - and always has been - a slight of hand trick to mask deficiencies. In a perfect universe we could not need caches.

[Mike.Sheen]

At this point I'm going to move forward with manual cache management.

It adds some noise to each service method, but I'll add some helper / extension methods to reduce that down as much as I can.

The philosophy is opt-in explicitly via custom plugin - existing behaviour is unchanged without a custom plugin.

All the custom plugin does is add to a static dictionary the route and optional expiry TimeSpan (example in comments below).

And example of the how the GET of /Debtors/{DebtorID} would now look:

Code: Select all

[Authenticate]
public JiwaFinancials.Jiwa.JiwaServiceModel.Debtors.Debtor Get(DebtorGETRequest request)
{
   JiwaApplication.Manager manager = this.SessionAs<JiwaAuthUserSession>().Manager; // This is going to change once I get rid of JiwaAuthUserSession
   JiwaFinancials.Jiwa.JiwaServiceModel.Debtors.Debtor debtorDTO = null;
   
   // if this route is registered to be cached, retrieve the cache entry for the route
   // Caching is opt-in only.
   // It's up to custom plugins to add to the CachedRoutes static dictionary the routes they want to cache responses for, along with an optional expiry
   //
   // Example to cache the GET of /Debtors for max 60 seconds:
   // public class RESTAPIPlugin : System.MarshalByRefObject, JiwaFinancials.Jiwa.JiwaApplication.IJiwaRESTAPIPlugin
   // {
   //     public void Configure(JiwaFinancials.Jiwa.JiwaApplication.Plugin.Plugin Plugin, ServiceStack.ServiceStackHost AppHost, Funq.Container Container, JiwaApplication.Manager JiwaApplicationManager)
   //     {
   //         RESTAPIPlugin.CachedRoutes.Add("/Debtors", TimeSpan.FromSeconds(60));
   //     }
   // }
   // The CacheDuration (Timespan passed as argument 2) is optional - null means it will not expire.
   // We don't need to expire items in the cache, the cache provider does that for us automatically.         
      
   Nullable<TimeSpan> cacheDuration = null;
   string cacheKey = null;   
   RESTAPIPlugin.CachedRoutes.TryGetValue(base.Request.GetRoute().Path, out cacheDuration);
   
   if (cacheDuration != null && cacheDuration.Value != null && cacheDuration.Value != TimeSpan.FromMilliseconds(0))
   {
      cacheKey = UrnId.Create<JiwaFinancials.Jiwa.JiwaServiceModel.Debtors.Debtor>(request.DebtorID);   
      // if this is a request using a debtor API key, we need to salt the cache key with the API Key otherwise we might accidentally return a cached entry which came from
      // a staff member or something which will have stuff in the DTO we don't want our customers to be exposed to.
      if (manager.Database.APIKey_Type == "Debtor")
         cacheKey += manager.Database.APIKey_KeyValue;
      
      debtorDTO = base.Cache.Get<JiwaFinancials.Jiwa.JiwaServiceModel.Debtors.Debtor>(cacheKey);
      
      if (debtorDTO != null)
         return debtorDTO;      // Return the cached value.
   }
   
   // route either is not cached, or cache entry for the record is not present - go and retrieve the record normally
   JiwaFinancials.Jiwa.JiwaDebtors.Debtor Debtor = Helper.Service.GetMaintenanceBusinessLogic<JiwaFinancials.Jiwa.JiwaDebtors.Debtor>(manager, this, request.DebtorID,
      (d, m, s) =>
      {
         d.Read(request.DebtorID);
         if (Helper.Service.IsStateful(s))
         {
            m.ObjectDictionary.Add(d.DebtorID, d);
         }
      });
   
   debtorDTO = Debtor.DTO_Serialise();
   
   if (cacheDuration != null)
   {
      // if we got to here then we want to cache this route, and we previously had no cache entry - so we set one now.
      base.Cache.Set<JiwaFinancials.Jiwa.JiwaServiceModel.Debtors.Debtor>(cacheKey, debtorDTO, DateTime.Now.ToUniversalTime().Add(cacheDuration.Value));
   }
   
   return debtorDTO;                             
}

For reference, this is how the above looked with no caching:

Code: Select all

[Authenticate]
public JiwaFinancials.Jiwa.JiwaServiceModel.Debtors.Debtor Get(DebtorGETRequest request)
{
   JiwaApplication.Manager manager = this.SessionAs<JiwaAuthUserSession>().Manager;
   JiwaFinancials.Jiwa.JiwaDebtors.Debtor Debtor = Helper.Service.GetMaintenanceBusinessLogic<JiwaFinancials.Jiwa.JiwaDebtors.Debtor>(manager, this, request.DebtorID,
      (d, m, s) =>
      {
         d.Read(request.DebtorID);
         if (Helper.Service.IsStateful(s))
         {
            m.ObjectDictionary.Add(d.DebtorID, d);
         }
      });

   return Debtor.DTO_Serialise();
}


As for Redis, I can tell you if you are running it as a local service on a server it is fairly reliable but under Azure you can get some nasty timeouts with it when used for session management and it can be expensive to run.

Another change I'm making is getting rid of our custom JiwaAuthUserSession class which extends ServiceStack's AuthUserSession by adding a property to store the Manager instance associated with the authenticated user. As the Manager is not serialisable, I'm pretty certain that would have caused issues in anything besides an in-memory cache.
We'll see if that solves any problems relating to session caching that you saw, but refactoring that so we no longer associate a Manager instance with a session by storing it in the session object which is cached will be a good thing regardless.

[SBarnes]

How will you therefore get a manager in a service?

[Mike.Sheen]

How will you therefore get a manager in a service?

Static dictionary of the RESTAPIPlugin class - example:

Code: Select all

JiwaFinancials.Jiwa.JiwaApplication.Manager manager = null;         
RESTAPIPlugin.JiwaSessionDictionary.TryGetValue(this.GetSession().Id, out manager);