Rest API and Caching

Discussions relating to the REST API of Jiwa 7.

Re: Rest API and Caching

Postby Mike.Sheen » Fri Dec 13, 2019 6:35 pm

Back onto the topic of the REST API and caching...

I wasn't able add a method to wrap existing code in service methods and have that being all you needed to do in order to add caching support... it still required a little refactoring.

However, I did come up with a method to completely replace the existing code which uses a new helper method.

An example is a GET request for a debtor record. The same method can be used for all our maintenance business logic you want to read and serialise out:

Code: Select all
[Authenticate]
public JiwaFinancials.Jiwa.JiwaServiceModel.Debtors.Debtor Get(DebtorGETRequest request)
{         
   return Helper.Service.GetDTO<JiwaServiceModel.Debtors.Debtor, JiwaDebtors.Debtor>(this, base.Response, request.DebtorID, (dto) => { return (DateTime)dto.LastSavedDateTime; } );
}   


This method simply accepts the generic parameters, the value parameters and it will return any cached DTO if the DTO is registered to be cached, and if there is a DTO present - if not it then it creates the business logic, performs a read and if the DTO is registered to be cached it then adds the DTO to the cache, and adds an optional response header for the Last-Modified (that's the last parameter - a nullable lamda to specify what date to use).

Initially I was worried that this was a level of abstraction which perhaps hid too much, but after comparing with what we had previously and seeing that same dozen or so lines of code repeated over and over in the various GET methods - with just a few type names and values changing - that worry has faded a little.

Similar methods are there for POST, PATCH, DELETE operations and so on - plus overloaded for the different types of business logic (list maintenance, multi-list maintenance, configuration).

It doesn't mean you have to use these helper methods in your own service methods, even if you want to support caching in there - we still have a fairly reasonable way of wrapping that with a few minor tweaks needed to existing code.

It does mean a lot less effort on my part if we decide add more response headers or make logic tweaks - such as adding an ETag - the number of code changes is reduced making that much more manageable.
Mike Sheen
Chief Software Engineer
Jiwa Financials

If I do answer your question to your satisfaction, please mark it as the post solving the topic so others with the same issue can readily identify the solution
User avatar
Mike.Sheen
Overflow Error
Overflow Error
 
Posts: 2444
Joined: Tue Feb 12, 2008 11:12 am
Location: Perth, Republic of Western Australia
Topics Solved: 756

Re: Rest API and Caching

Postby SBarnes » Tue Dec 17, 2019 7:44 pm

Is there an ETA on when this will be finalised, with all the back and forth it's getting hard to keep track, without seeing it in its entirety?
Regards
Stuart Barnes
SBarnes
Shihan
Shihan
 
Posts: 1618
Joined: Fri Aug 15, 2008 3:27 pm
Topics Solved: 175

Re: Rest API and Caching

Postby Mike.Sheen » Wed Dec 18, 2019 11:32 am

SBarnes wrote:Is there an ETA on when this will be finalised, with all the back and forth it's getting hard to keep track, without seeing it in its entirety?


Long ways off yet. I can only spend an hour or so every few days on it... the patterns are almost bedded down, then it's a case of going through all the service methods (600 or so) and refactoring their code to use the new cache aware helper methods.

Once we're at that stage it will be able to be provided for preview / testing.

And then I'll need to write the tests to incorporate caching into the matrix.

So, months away from being finalised yet.

I did ask in this post:
Before I sweep through and apply the necessary changes to the rest of the service methods to accommodate caching, it might be worth knowing what are the top things anyone would want caching for - I can do those first and provide a test plugin for alpha testing to iron out any wrinkles.

So, what do you think I should be looking at as a priority?


I can provide an early preview available with what you'd be most interesting in testing caching with, but I'm now out of time this year to get you something other than what I currently have which is the Debtors services.
Mike Sheen
Chief Software Engineer
Jiwa Financials

If I do answer your question to your satisfaction, please mark it as the post solving the topic so others with the same issue can readily identify the solution
User avatar
Mike.Sheen
Overflow Error
Overflow Error
 
Posts: 2444
Joined: Tue Feb 12, 2008 11:12 am
Location: Perth, Republic of Western Australia
Topics Solved: 756

Re: Rest API and Caching

Postby SBarnes » Wed Dec 18, 2019 11:54 am

If we were looking at routes to cache first, given the rather large customer customisation system we may use for testing the following routes between the web store to api and a windows app to api that sends through orders from their head office I would look at these

Get Debtor
Patch Debtor
Get Inventory
Get Inventory SOH
Post Sales Order
Get Sales Order
Am I alive (Can't remember it's real name, don't know if you'd cache this anyway)

I can email you the Web API Extension Plugin we built if you like , if you want something with custom routes galore and auto queries galore , but I am not sure it's of any use without the database that backs it, but that is in Azure SQL including the test one as you know given you helped set it up.

Let me know if this will help.
Regards
Stuart Barnes
SBarnes
Shihan
Shihan
 
Posts: 1618
Joined: Fri Aug 15, 2008 3:27 pm
Topics Solved: 175

Re: Rest API and Caching

Postby Mike.Sheen » Wed Dec 18, 2019 6:57 pm

SBarnes wrote:Get Debtor
Patch Debtor
Get Inventory
Get Inventory SOH
Post Sales Order
Get Sales Order
Am I alive (Can't remember it's real name, don't know if you'd cache this anyway)


Ok great - I suspected all these would be wanted, and some are already done - so I can definitely do the balance of those listed before everything else we need to do before we declare it complete, and provide preview plugins beforehand.

SBarnes wrote:I can email you the Web API Extension Plugin we built if you like , if you want something with custom routes galore and auto queries galore , but I am not sure it's of any use without the database that backs it, but that is in Azure SQL including the test one as you know given you helped set it up.

Let me know if this will help.


By all means email me the plugin - I can refactor at least few of your custom services (even without a database) so you have a before and after comparison to work out what you would need to do to make them all cache aware services. AutoQuery routes I believe won't need any effort to make cache aware - I was able to do that transparently - at least in my testing so far.

I'm on leave in 20 hours from now, so I don't think I can get you anything before I return next year, but I may find time to tinker in moments of idleness - so it wouldn't hurt to send me that custom Web API Extension Plugin before my return.
Mike Sheen
Chief Software Engineer
Jiwa Financials

If I do answer your question to your satisfaction, please mark it as the post solving the topic so others with the same issue can readily identify the solution
User avatar
Mike.Sheen
Overflow Error
Overflow Error
 
Posts: 2444
Joined: Tue Feb 12, 2008 11:12 am
Location: Perth, Republic of Western Australia
Topics Solved: 756

Re: Rest API and Caching

Postby Mike.Sheen » Sat Mar 07, 2020 1:09 pm

Just came across an issue which means 7.2.1 SR4 will be needed for caching to work - DEV-8141, which isn't the end of the world, but it's a shame we couldn't do it without a change to the software.

Things are progressing nicely - just doing the grunt work of refactoring the 400 or so affected service methods now.
Mike Sheen
Chief Software Engineer
Jiwa Financials

If I do answer your question to your satisfaction, please mark it as the post solving the topic so others with the same issue can readily identify the solution
User avatar
Mike.Sheen
Overflow Error
Overflow Error
 
Posts: 2444
Joined: Tue Feb 12, 2008 11:12 am
Location: Perth, Republic of Western Australia
Topics Solved: 756

Re: Rest API and Caching

Postby SBarnes » Sat Mar 07, 2020 4:21 pm

It could be worse it could have involved a database change as well.

Whilst on the web api I have a question that is not related to caching, which is, is there a way in terms of annotations to require a route to explicitly require https just as you can at the moment require authentication with [Authenticate] and can the self hosted service be made to bind to multiple ports or do you need the iis version to achieve that?
Regards
Stuart Barnes
SBarnes
Shihan
Shihan
 
Posts: 1618
Joined: Fri Aug 15, 2008 3:27 pm
Topics Solved: 175

Re: Rest API and Caching

Postby Mike.Sheen » Sat Mar 07, 2020 6:09 pm

SBarnes wrote:is there a way in terms of annotations to require a route to explicitly require https just as you can at the moment require authentication with [Authenticate]


Not currently, no - you would need two different Jiwa REST API services, one handling the HTTPS routes, and another handling the HTTP routes - and we'd need to modify our service to have config option in the .exe.config to specify which plugin is to be used for the service - so you'd need to split the plugin in two as well - one plugin for HTTPS and one for HTTP, and the different services configured to use the respective plugin.

That's a fairly unusual requirement - what's the use case?

SBarnes wrote:and can the self hosted service be made to bind to multiple ports or do you need the iis version to achieve that?


No, you can't make it bind to multiple ports, but any proxy can be put in front of the API to forward / redirect to another port - including IIS - you can even use IIS to forward to the self hosted API fairly easily.
Mike Sheen
Chief Software Engineer
Jiwa Financials

If I do answer your question to your satisfaction, please mark it as the post solving the topic so others with the same issue can readily identify the solution
User avatar
Mike.Sheen
Overflow Error
Overflow Error
 
Posts: 2444
Joined: Tue Feb 12, 2008 11:12 am
Location: Perth, Republic of Western Australia
Topics Solved: 756

Re: Rest API and Caching

Postby SBarnes » Sat Mar 07, 2020 6:39 pm

The use case is fairly simple certain external systems that will do webhooks will only do https, now you could turn the whole api onto doing https but what if something needs to stay on http?

Also I am not sure that given all routes in the database become available that doing that is such a good idea either, two services and the config specifying what to include or exclude would work but it may be just easier rather than use the api self hosted to use something like https://github.com/aspnet/AspNetWebHooks as I've had a play around with it and they are fairly simple to make work.
Regards
Stuart Barnes
SBarnes
Shihan
Shihan
 
Posts: 1618
Joined: Fri Aug 15, 2008 3:27 pm
Topics Solved: 175

Re: Rest API and Caching

Postby Mike.Sheen » Sat Mar 07, 2020 8:13 pm

SBarnes wrote:but what if something needs to stay on http?


Then that something can't participate on the internet.

We allow non HTTPS only for the ease of development and testing, but really in this day and age if anything can't do HTTPS then they might as well pack up their things and go home.
Mike Sheen
Chief Software Engineer
Jiwa Financials

If I do answer your question to your satisfaction, please mark it as the post solving the topic so others with the same issue can readily identify the solution
User avatar
Mike.Sheen
Overflow Error
Overflow Error
 
Posts: 2444
Joined: Tue Feb 12, 2008 11:12 am
Location: Perth, Republic of Western Australia
Topics Solved: 756

PreviousNext

Return to REST API

Who is online

Users browsing this forum: No registered users and 1 guest

cron