Rest API and Caching

[Mike.Sheen]

Oh - and we'll provide guidance on what changes you'll need to make (if any) if you copied our patterns for your own custom plugins - it should be a simple search and replace, that's a goal I've set in order to minimise the work required by those who have extended the ReST API in any way.

[SBarnes]

You read my mind as I was thinking how many changes I would have to make given one of our extensions of the api is about 9,000 lines of code. It would actually be good if there was a helper function where you could replace 1 line with 1 line, then a global search and replace would probably do it.

Will the api continue support deployment to IIS (Azure Web App) if you are going for in memory caching, I suppose it should still work if session affinity is enabled if the web app scaled out?

[Mike.Sheen]

You read my mind as I was thinking how many changes I would have to make given one of our extensions of the api is about 9,000 lines of code. It would actually be good if there was a helper function where you could replace 1 line with 1 line, then a global search and replace would probably do it.

Yep - that's exactly what I'm going for.

Even if it turns out to be a multi-line search and replace, there are tools which make that pretty easy - one I used today to get things into a working state was a plugin for Notepad++ called ToolBucket - I copy pasted all the REST API plugin code into Notepad++ and was able to replace several lines of our code with several new lines of code - it was quite painless even with several tens of thousands of lines of code we have (540 code block replacements from memory).

Will the api continue support deployment to IIS (Azure Web App) if you are going for in memory caching, I suppose it should still work if session affinity is enabled if the web app scaled out?

Yes, caching will work with IIS or self-hosted exactly the same, so no code changes needed in our plugin or your custom ones extending our offering in regards to IIS vs self hosted. We will support any caching provider conforming to the ServiceStack model - so in-memory, memcache or Redis should all work without any problem.

Session affinity will certainly be required - as long as we have our current architecture of having a Manager per session - which I don't see changing unless we redo all our layers to handle truly stateless.

[SBarnes]

Notepad++ is always a great tool which we install in nearly every client site, apparently it is base on scintilla so I don't know if you have seen this https://github.com/jacobslusser/ScintillaNET

[Mike.Sheen]

Ok, so the pattern has been mostly bedded down - an example of this for the GET on /Debtors/{DebtorID} is as follows:

In our standard REST API plugin, we've changed the service methods for GET, POST, PATCH and DELETE to use a static method which pretty much just accepts as a lambda the existing code - so the new Get method of the DebtorGETRequest looks like this:

Code: Select all

[Authenticate]
public JiwaFinancials.Jiwa.JiwaServiceModel.Debtors.Debtor Get(DebtorGETRequest request)
{
   JiwaApplication.Manager manager = this.GetManager();
   
   JiwaServiceModel.Debtors.Debtor DTO = Helper.Service.GetCachedDTOFromMaintenanceBusinessLogic<JiwaServiceModel.Debtors.Debtor>(manager, this, request.DebtorID,
   (a, b, c) =>
   {
      JiwaDebtors.Debtor Debtor = Helper.Service.GetMaintenanceBusinessLogic<JiwaDebtors.Debtor>(manager, this, request.DebtorID,
            (d, m, s) =>
            {
               d.Read(request.DebtorID);
               if (Helper.Service.IsStateful(s))
               {
                  manager.ObjectDictionary.Add(d.DebtorID, d);
               }
            });               
      return Debtor.DTO_Serialise();
   });
   
   return DTO;                                      
}

The first line shows the new way we get the Manager instance for the session:

Code: Select all

JiwaApplication.Manager manager = this.GetManager();

This is a single line replacement of the existing:

Code: Select all

JiwaApplication.Manager manager = this.SessionAs<JiwaAuthUserSession>().Manager;

The call to GetCachedDTOFromMaintenanceBusinessLogic is what transparently caches the DTO result. It accepts a lambda function which is pretty close to the code which was already there - so if you have any custom services it should be a simple "paint by numbers" exercise to get your own services caching responses.

Right now it's called GetCachedDTOFromMaintenanceBusinessLogic because I'm not sure if I need yet separate methods for Post, Patch and so on, or if I need a separate method for the different business logic types (ListMaintenance, MultiListMaintenance, Configuration and so on). So the name of the method may change - I'm hoping I can actually just use the same static helper method for all operations of all business logic types - if that is the case then it will definitely change.

The custom plugin to demonstrate how to opt-in to caching on various routes with an optional expiry timespan we'll be shipping as a standard (but disabled) plugin - it's entire code is fairly compact and easy to grok:

Code: Select all

using System;

namespace JiwaFinancials.Jiwa.JiwaServiceModel
{
    public class RESTAPIPlugin : System.MarshalByRefObject, JiwaFinancials.Jiwa.JiwaApplication.IJiwaRESTAPIPlugin
   {
      public void Configure(JiwaFinancials.Jiwa.JiwaApplication.Plugin.Plugin Plugin, ServiceStack.ServiceStackHost AppHost, Funq.Container Container, JiwaApplication.Manager JiwaApplicationManager)
        {         
         // Get the cached routes dictionary from the IOC container
         CachedRoutesDictionary cachedRoutes = Container.TryResolve<CachedRoutesDictionary>();   
         
         //cachedRoutes.Routes.Add("/Debtors/{DebtorID}", TimeSpan.FromDays(1));   // 1 Day expiry
         cachedRoutes.Routes.Add("/Debtors/{DebtorID}", TimeSpan.Zero);         // No expiry
      }      
   }
}

This one is named "REST API In-Memory Cache" - we plan to ship another 2 - one for Redis and one for Memcache - the only difference to the in-memory plugin will be the setup of a caching provider in the IOC container (I think - will know for sure when it's done).

So, I've tried to keep the impact down to an absolute minimum - the only change imposed that isn't optional for your custom services is to use the new extension method this.GetManager(); to get the manager instead of this.SessionAs<JiwaAuthUserSession>().Manager;. It's a pity we didn't originally make an extension method for getting the manager from the session... otherwise there would be no work for you to do!

The caching is optional to implement - but if you want the users of your custom services to have the option of caching DTOs around your services then there is a little bit of repetitive boilerplate code to apply - it'd be a complex search and replace regex to try and do, so I think a manual effort will yield the safest results.

So far this work has only been plugin changes - it doesn't require any software changes, so I still think we will be able to provide a plugin which will work with 7.2.0 or 7.2.1.

[SBarnes]

Well on the huge customisation that extended the web api that went live at the beginning of the year

Code: Select all

JiwaApplication.Manager manager = this.SessionAs<JiwaAuthUserSession>().Manager;

may not be a problem as there are only five occurrences so it looks like that bullet gets dodged.

Fortunately the original system used a lot of stored procedures for database access so our original starting point was to use the T4 template for ORMLite to generate the necessary code and then we wrapped these in a service to allow the api calls. These stored procedures now only write to proprietary tables and any access to the Jiwa tables use the Jiwa business logic objects.

If you want when the caching is baked we can test it on this same client database in our development environment given it's a large database in size and has been customised with a level of complexity that if something is going to break things this will surely do it.

[Mike.Sheen]

Well on the huge customisation that extended the web api that went live at the beginning of the year

Code: Select all

JiwaApplication.Manager manager = this.SessionAs<JiwaAuthUserSession>().Manager;

may not be a problem as there are only five occurrences so it looks like that bullet gets dodged.

I'm going to call that a win!

If you want when the caching is baked we can test it on this same client database in our development environment given it's a large database in size and has been customised with a level of complexity that if something is going to break things this will surely do it.

That would be great. We still have work to do before it's done - such as:

• Modify all service methods to wrap caching around existing code
• Add response headers so client can tell if the response was cached and if so when (if at all) it expires
• Add routes to manually invalidate parts or all of the cache using wildcards (eg: /Cache/Invalidate?urnStartsWith="{urn:Debtor")
• Consider if we want the plugin that adds routes to cache to be able to inject logic rules when to invalidate (eg: a POST to /Widget should invalidate all caching of GETs for Widget related queries).
• Add tests to our test application to test caching

That last one is going to be rather big - we're pretty much going to double the size of our test app. On that, I feel our test strategy is rather hard to manage. It's a manual process of adding the test methods, and whilst we use the JsonServiceClient which makes the code fairly compact, but it's easy to forget or miss tests when writing them - and we currently have to do multiple for each route/service method - client creds, apikey, creds + stateful, api-key + stateful and now with introducing caching it's adding multiple more dimensions (in-memory, memcache, redis).

I've love to hear any suggestions on good ways to manage tests. We're currently using NUnit, in a large console app, broken into classes for each area of concern - eg: SalesOrders, Debtors, Inventory and so on.

[Mike.Sheen]

we plan to ship another 2 - one for Redis and one for Memcache - the only difference to the in-memory plugin will be the setup of a caching provider in the IOC container (I think - will know for sure when it's done).

Well that was really all it took - the Redis caching plugin is literally just 2 lines of code extra when compared to the in-memory caching plugin:

Code: Select all

using System;
using ServiceStack.Redis;

namespace JiwaFinancials.Jiwa.JiwaServiceModel
{
    public class RESTAPIPlugin : System.MarshalByRefObject, JiwaFinancials.Jiwa.JiwaApplication.IJiwaRESTAPIPlugin
   {
      public void Configure(JiwaFinancials.Jiwa.JiwaApplication.Plugin.Plugin Plugin, ServiceStack.ServiceStackHost AppHost, Funq.Container Container, JiwaApplication.Manager JiwaApplicationManager)
        {   
         // Register Redis as the cache provider
         Container.Register<IRedisClientsManager>(c => new RedisManagerPool("localhost:6379"));
         Container.Register(c => c.Resolve<IRedisClientsManager>().GetCacheClient());
         
         // Get the cached routes dictionary from the IOC container
         CachedRoutesDictionary cachedRoutes = Container.TryResolve<CachedRoutesDictionary>();   
         
         //cachedRoutes.Routes.Add("/Debtors/{DebtorID}", TimeSpan.FromDays(1));   // 1 Day expiry
         cachedRoutes.Routes.Add("/Debtors/{DebtorID}", TimeSpan.Zero);         // No expiry                        
      }      
   }
}

I used Debian 9 running in WSL and deployed a Redis instance there, and inspecting the cache after performing some requests I could see things in the Redis cache.

A small gotcha was our global request filter in the REST API plugin had to be modified slightly for disposing of Manager instances when the sessions had expired - the Redis cache provider was throwing exceptions when querying the cache provider if there were no sessions at all in the cache, but other than that it worked great.

Scott's been calling my documenting here of our journey towards caching "the ramblings of a madman" - I don't expect responses to any of these posts I've been making of late - It's more for my benefit really so I have some sort of explanation in the future of why decisions were made one way or another, and sometimes the very act of writing stuff down gives me other ideas or potential problems to explore... In case anyone felt obligated to respond to these "ramblings".

[SBarnes]

tell him it's the ramblings of mad men and he's willing to join in if he feels left out, it is the silly season after all.

Of course the other question is how this will all perform once its put under some load and I suppose anything that does an auto query and some paging would definitely make the cache grow fairly quickly which might be a good test or will auto query not cache?

[Mike.Sheen]

and he's willing welcome to join in if he feels left out

[crickets chirping]

Of course the other question is how this will all perform once its put under some load

It really depends on the nature of the load imposed. The timings I've observed so far see an uncached GET of /Debtors/{DebtorID} to be around 570ms. When a cached response is returned it's 17ms. The load is greatly reduced on both the SQL Server and the REST API host server whenever there is a cache hit - so as long as we can find a sensible approach towards when to invalidate a cache entry it should be a huge win.

I suppose anything that does an auto query and some paging would definitely make the cache grow fairly quickly which might be a good test or will auto query not cache?

Definitely AutoQuery routes will be cached - but the jury is out as to how effective that will be - on largely static routes (like a list of inventory categories) this will work well as it's not too big and can be easily invalidated when a category is changed / added / removed. For other routes, such as a list of sales orders, this will be invalidated pretty darn quick and will not really benefit that much from caching... except when it does - like when only a few orders an hour are placed, but a client is using a sales order AutoQuery several times a minute - huge cache hit potential there.

We'll still have at least some benefit - if only fleeting - for the busy areas.