Rest API and Caching

Discussions relating to the REST API of Jiwa 7.

Re: Rest API and Caching

Postby SBarnes » Sun Dec 08, 2019 10:12 am

However, I may be able to move that into request and response filters... so that would mean absolutely no code changes needed to existing service methods - it would all be abstracted away and work transparently. I'll get to exploring that once I've covered off all the different types of requests we typically have - I don't want to go and modify the hundreds of existing service methods to explicitly support caching so making it global request and response filters is the holy grail I'm going for.


The holy grail would be good, but my question was more aimed at will it cache before or after any compression as this would obviously affect the cache size and whether or not a response is then going to need to need compressing on subsequent requests?

If using the in-memory cache then cache bloat is going to be a real concern, because it will be easy to exceed the memory allowed for the process - but other cache providers are be able to handle large cache sizes and have good eviction policies to prevent exhaustion of the available memory.


Maybe is would be good if there was a way to define the maximum size of a response that goes into the cache?

I am also assuming from what you said previously that with luck controlling or turning on a given cache will be done in a similar way to how email is currently controlled, where there is a separate plugin for 365 and one for smtp?
Regards
Stuart Barnes
SBarnes
Shihan
Shihan
 
Posts: 1617
Joined: Fri Aug 15, 2008 3:27 pm
Topics Solved: 175

Re: Rest API and Caching

Postby Mike.Sheen » Sun Dec 08, 2019 11:52 am

SBarnes wrote:my question was more aimed at will it cache before or after any compression as this would obviously affect the cache size and whether or not a response is then going to need to need compressing on subsequent requests?


DTO's at the moment are stored uncompressed, regardless of the compression headers in the request, and on the way out it's compressed as per the request header.

This way it doesn't matter what compression is requested by the client, they always get what they asked for. It could be expensive to always store compressed DTO's and then have to decompress that if the client requested no compression, or different compression (brotli vs gzip vs deflate - we might have to decompress the DTO from say deflate then compress back to brotli, for example - if that was what was requested). We could have discrete cache entries for each type of compression (none, brotli, gzip, deflate) - but then you're increasing cache size (depending on the number of variations on compression requested for the same resource) and we'd also have multiple cache DTO's to update or remove when we need to.

So, to keep things simple it's uncompressed for now. If this emerges as a problem we can revisit then.

SBarnes wrote:Maybe is would be good if there was a way to define the maximum size of a response that goes into the cache?


Sure - that should be an easy change - when defining the routes to compress, pass in an optional parameter for the max byte size, and if the DTO is bigger than that, don't cache. Complexity is introduced when updating a cache entry - the DTO might have initially been under a size limit, but a subsequent PATCH pushes it over - then we'd have to have the smarts to remove the existing cache entry.

SBarnes wrote:I am also assuming from what you said previously that with luck controlling or turning on a given cache will be done in a similar way to how email is currently controlled, where there is a separate plugin for 365 and one for smtp?

Yes, I currently have 3 plugins for cache options - one for In-Memory, one for Redis, and one for MemCache - just enable the one you want - just like the emailing plugins. They are mutually exclusive, and you'll find if you enable two or more just the last plugin ordered by name (or execution order) will be used because it will override the registering of the cache provider in the Funq IOC container.
Mike Sheen
Chief Software Engineer
Jiwa Financials

If I do answer your question to your satisfaction, please mark it as the post solving the topic so others with the same issue can readily identify the solution
User avatar
Mike.Sheen
Overflow Error
Overflow Error
 
Posts: 2440
Joined: Tue Feb 12, 2008 11:12 am
Location: Perth, Republic of Western Australia
Topics Solved: 755

Re: Rest API and Caching

Postby SBarnes » Mon Dec 09, 2019 6:23 pm

Yes, I currently have 3 plugins for cache options - one for In-Memory, one for Redis, and one for MemCache - just enable the one you want - just like the emailing plugins. They are mutually exclusive, and you'll find if you enable two or more just the last plugin ordered by name (or execution order) will be used because it will override the registering of the cache provider in the Funq IOC container.


I suppose that means that things could be extensible in that you could introduce other cache providers down the track?
Regards
Stuart Barnes
SBarnes
Shihan
Shihan
 
Posts: 1617
Joined: Fri Aug 15, 2008 3:27 pm
Topics Solved: 175

Re: Rest API and Caching

Postby Mike.Sheen » Wed Dec 11, 2019 6:51 pm

SBarnes wrote:I suppose that means that things could be extensible in that you could introduce other cache providers down the track?


Exactly...

On cache providers, besides MemCache and Redis we might have some other options built-in - such as Azure Table Storage and AWS DynamoDB - ServiceStack have those packages available so we might as well ship with support for those as well


To support Azure Table Storage and AWS DynamoDB we'll just have another plugin for each, and you just enable the one you want as your caching provider... and as you mentioned, in the future when some other shiny caching provider is wanted it should just be a plugin we can provide / ship which will support that provider.
Mike Sheen
Chief Software Engineer
Jiwa Financials

If I do answer your question to your satisfaction, please mark it as the post solving the topic so others with the same issue can readily identify the solution
User avatar
Mike.Sheen
Overflow Error
Overflow Error
 
Posts: 2440
Joined: Tue Feb 12, 2008 11:12 am
Location: Perth, Republic of Western Australia
Topics Solved: 755

Re: Rest API and Caching

Postby Mike.Sheen » Wed Dec 11, 2019 7:50 pm

So I've been backwards and forwards trying a few different approaches for the most sensible way to opt-in to caching.

What I've arrived at is registering a DTO to be cached - and not a request or a route. I did try the latter two, but it didn't feel right because of the nature of how we respond to GET, POST, PUT, PATCH and DELETE requests for the same DTO.

To illustrate, the Debtor DTO (JiwaFinancials.Jiwa.JiwaServiceModel.Debtors.Debtor) is returned on a by the DebtorGETRequest which is mapped to a GET on /Debtors/{DebtorID}

The initial instinct might to be simply add to the cache the DTO with a key such as urn:/Debtors/{DebtorID}:000000001B0000001V - the urn structure there is urn:{Path}:{DTOId}

However, a POST to /Debtors, should add to the cache the DTO for the new debtor. Which would mean adding a lot of opinions and complexity about how operations are related to a specific DTO and that meant adding a lot of noise to service methods to work that out.

And it gets worse... a GET on /Debtors/{DebtorID}:000000001B0000001V/ContactNames returns the contact names for the given debtor - but we don't need a separate cache entry for urn:/Debtors/{DebtorID}/ContactNames:000000001B0000001V as we might already have that Debtor DTO cached - we just want to return part of the already cached DTO.

By instead registering a DTO to be cached, and not a request, response or route we minimise our number of cache entries for the same resource, or parts of the same resource.

So, to register the JiwaFinancials.Jiwa.JiwaServiceModel.Debtors.Debtor DTO, we simply call:
Code: Select all
dtoCacheDictionary.Register<JiwaFinancials.Jiwa.JiwaServiceModel.Debtors.Debtor>();


And this will automatically cache the a debtor DTO for the ALL the routes involving a debtor DTO:
  • GET /Debtors/{DebtorID}
  • POST /Debtors
  • PATCH /Debtors/{DebtorID}
  • DELETE /Debtors/{DebtorID}
  • GET /Debtors/{DebtorID}/ContactNames
  • GET /Debtors/{DebtorID}/ContactNames/{ContactNameID}
  • POST /Debtors/{DebtorID}/ContactNames
  • PATCH /Debtors/{DebtorID}/ContactNames/{ContactNameID}
  • DELETE /Debtors/{DebtorID}/ContactNames/{ContactNameID}
  • GET /Debtors/{DebtorID}/Notes
  • GET /Debtors/{DebtorID}/Notes/{NoteID}
  • POST /Debtors/{DebtorID}/Notes
  • PATCH /Debtors/{DebtorID}/Notes/{NoteID}
  • DELETE /Debtors/{DebtorID}/Notes/{NoteID}

...and many more. As the service methods behind all those routes all do the same thing - read a debtor and return either part or all of the DTO - it makes sense to have them operate on the same cached DTO.

Adding an expiry time to the DTO is just an optional parameter:

Code: Select all
dtoCacheDictionary.Register<JiwaFinancials.Jiwa.JiwaServiceModel.Debtors.Debtor>(TimeSpan.FromMintes(10));


And specifying other DTO's to invalidate (delete) if the debtor DTO is updated or deleted is an array of other DTO type names to remove when it invalidates:

Code: Select all
dtoCacheDictionary.Register<JiwaFinancials.Jiwa.JiwaServiceModel.Debtors.Debtor>(TimeSpan.Zero,
   new string[]
   {
      typeof(QueryResponse<Tables.v_Jiwa_Debtor_List>).ToString()
   });


So when a debtor is modified or deleted, the above shows we want the AutoQuery for v_Jiwa_Debtor_List (a list of debtors) to be removed from the cache.

All AutoQuery operations are able to be opted into for caching by just registering the return DTO:

Code: Select all
dtoCacheDictionary.Register<QueryResponse<Tables.v_Jiwa_Debtor_List>>();


It does mean it's not that intuitive to know what DTO type to specify - you need to know what DTO Type a request returns or updates, but it does mean there's a lot less registering of cache interest going on, smaller caches and less overall ceremony and noise.

One thing which isn't obvious, is that POST, PATCH, PUT and DELETE operations automatically update any cached DTO's - we don't remove the cache entry if present, but update or create it. Typically caching works by a GET always being the operation to cache a DTO, but we also keep the cached DTO's fresh with the other operations also as they have generated a DTO already anyway.

The other thing we do which is unconventional is we update any cached entries (or remove them) whenever our business logic objects save - and that covers everything even outside the API - including the Jiwa desktop application and any 3rd party applications using our .NET API. This is done by special internal API calls from the client application to the API.

Before I sweep through and apply the necessary changes to the rest of the service methods to accommodate caching, it might be worth knowing what are the top things anyone would want caching for - I can do those first and provide a test plugin for alpha testing to iron out any wrinkles.

So, what do you think I should be looking at as a priority? (I'm primarily asking Stuart, as he requested this feature - but anyone can chime in!)
Mike Sheen
Chief Software Engineer
Jiwa Financials

If I do answer your question to your satisfaction, please mark it as the post solving the topic so others with the same issue can readily identify the solution
User avatar
Mike.Sheen
Overflow Error
Overflow Error
 
Posts: 2440
Joined: Tue Feb 12, 2008 11:12 am
Location: Perth, Republic of Western Australia
Topics Solved: 755

Re: Rest API and Caching

Postby SBarnes » Wed Dec 11, 2019 8:14 pm

I will give it further though but on the surface of looks a good idea that I can't see a problem with it, I suppose it may also present the option of building an interface to use reflection to get the list of objects to decide to register further down the track.

The other thing we do which is unconventional is we update any cached entries (or remove them) whenever our business logic objects save - and that covers everything even outside the API - including the Jiwa desktop application and any 3rd party applications using our .NET API. This is done by special internal API calls from the client application to the API.


Now I am going to suggest something totally off the wall have you considered the ability of the client to use the cached DTO rather than go back to the database to keep the load off SQL?
Regards
Stuart Barnes
SBarnes
Shihan
Shihan
 
Posts: 1617
Joined: Fri Aug 15, 2008 3:27 pm
Topics Solved: 175

Re: Rest API and Caching

Postby Mike.Sheen » Wed Dec 11, 2019 11:00 pm

SBarnes wrote:Now I am going to suggest something totally off the wall have you considered the ability of the client to use the cached DTO rather than go back to the database to keep the load off SQL?


Yes, and that flirtation was very brief, if when you say client you mean consumers of the .NET API.

For consumers of the REST API, In theory they do have that ability - thanks to the Cache Aware Service Clients ServiceStack provides - as long as we deliver the right response headers the cache aware clients get optimal performance by using their own cached responses transparently. I've not tested that, so don't bank on it just yet.

For other clients - like those using the .NET API, which includes the Jiwa Desktop application - that is not an option. It would mean a significant amount of effort, risk and pain (for both customers/re-sellers and ourselves) which would be better spent on moving to a pure .NET Core code base and offering a true web based platform.

It's an attractive idea - and I'd love to experiment and toy with it, but in reality we'd just be dancing around where we really want to be and delaying the paradigm shift we need to make.
Mike Sheen
Chief Software Engineer
Jiwa Financials

If I do answer your question to your satisfaction, please mark it as the post solving the topic so others with the same issue can readily identify the solution
User avatar
Mike.Sheen
Overflow Error
Overflow Error
 
Posts: 2440
Joined: Tue Feb 12, 2008 11:12 am
Location: Perth, Republic of Western Australia
Topics Solved: 755

Re: Rest API and Caching

Postby SBarnes » Wed Dec 11, 2019 11:10 pm

By client I mean't Jiwa itself and by paradigm shift I am assuming you would be referring to a web based client and by the .net core reference I would assume you are referring to a more microservice like architecture?
Regards
Stuart Barnes
SBarnes
Shihan
Shihan
 
Posts: 1617
Joined: Fri Aug 15, 2008 3:27 pm
Topics Solved: 175

Re: Rest API and Caching

Postby Mike.Sheen » Thu Dec 12, 2019 6:56 pm

SBarnes wrote:By client I mean't Jiwa itself


Yep - I figured that - but Jiwa the desktop application is just a consumer of our .NET API, so whether it's Jiwa or a 3rd party application, they're all in the same boat.

SBarnes wrote:and by paradigm shift I am assuming you would be referring to a web based client and by the .net core reference I would assume you are referring to a more microservice like architecture?


Definitely a web based client, probably written in .NET Core 3.1 or later. Server side also .NET Core 3.1. We'd use server side Blazor, and might augment the client to use wasm Blazor for some bits to improve user experience (I hate that term).

Microservices? Not really. It's not possible to provide guaranteed consistency when using microservices, so that rules it out for a lot of what we do. For some things microservices will be ideal, but they'd just be trivial non-essential aspects.

What I mean by impossible to provide guaranteed consistency can be illustrated with this story:

Let's say we develop a microservice for general ledger journals.
We develop another for invoices.
When we want to post an invoice (process in todays Jiwa parlance), we'd be doing that as an operation within the invoices microservice. As part of posting an invoice, we'd ideally want to leverage the general ledger journals microservice to do the journal post.
Our invoices microservice needs to be in a blocking or synchronous state waiting to see if the general ledger journal microservice completes without error, which is not desirable, but even worse if the general ledger journal microservice completes and commits, and the invoices microservice fails on commit and rolls back - then we have a journal which shouldn't be there.
Strategies like two phase commits attempt to address the problem - but there is no guarantees it will work 100% of the time. There is no transaction coordinator which can work across service boundaries (that I know of... yet).

As soon as you try to enrol a service outside of another in a transaction, the potential for inconsistency arises - it's not atomic and never can be.

As attractive as microservices, serverless computing and all that is, I can't see reliable systems using it. I mentioned earlier that trivial non-essential aspects might be able to use microservices - things like an activity feed which provides a list what people in your organisation are doing ("Stuart just posted invoice 100493", "Mike created a new Cash Book Receipt 70745") would be fine as microservices - because they don't need to be enrolled into transactions, and even if they were they can fail without it have much consequence.
Mike Sheen
Chief Software Engineer
Jiwa Financials

If I do answer your question to your satisfaction, please mark it as the post solving the topic so others with the same issue can readily identify the solution
User avatar
Mike.Sheen
Overflow Error
Overflow Error
 
Posts: 2440
Joined: Tue Feb 12, 2008 11:12 am
Location: Perth, Republic of Western Australia
Topics Solved: 755

Re: Rest API and Caching

Postby SBarnes » Fri Dec 13, 2019 8:59 am

On the microservices I don't necessarily disagree, it adds a lot of overhead to introduce unlimited scalablity and decoupled components and you would really need a strong messaging system to make it work.

As for the web client to be honest I've only looked at Blazor briefly, most of the component manufacturers appear to still be trying to get their heads around it. I also saw a post regarding ServiceStack that said they won't support it until the client side of Blazor is released.

There is then of course the question of mobile which would I suppose mean looking at some sort of PWA with Blazor or Xamarin hosting Blazor. It amazes me that people would consider trying to use an ERP on smaller screens but everyone seems to ask for it.

Without knowing much about Blazor it, it would be handy from a Jiwa point of view if the web client supported the the similar types of customisation and addition of screens etc that the windows client does at present, I've seen this in other areas and systems I know of but whether that is feasible I don't know.
Regards
Stuart Barnes
SBarnes
Shihan
Shihan
 
Posts: 1617
Joined: Fri Aug 15, 2008 3:27 pm
Topics Solved: 175

PreviousNext

Return to REST API

Who is online

Users browsing this forum: No registered users and 1 guest