Rate Limits  Topic is solved

Discussions relating to the REST API of Jiwa 7.

Rate Limits

Postby SBarnes » Tue Nov 12, 2019 10:57 am

Does the web api currently support rate limits and if not how could this be implemented?
Regards
Stuart Barnes
SBarnes
Shihan
Shihan
 
Posts: 1617
Joined: Fri Aug 15, 2008 3:27 pm
Topics Solved: 175

Re: Rate Limits

Postby Mike.Sheen » Tue Nov 12, 2019 11:29 am

You can use request filters and the built-in session cache management to do this.

Attached is an example which uses a global request filter to impose a limit of 30 requests in a 10 second interval for all routes for any session.

When the limit is exceeded, a 429 is returned.

All responses, even if not limited, add header information telling the consumer how many requests they have left and when the limit expires.

EDIT: Removed extraneous document attachment to plugin.
Attachments
Plugin API Rate Limit.xml
(28.88 KiB) Downloaded 137 times
Mike Sheen
Chief Software Engineer
Jiwa Financials

If I do answer your question to your satisfaction, please mark it as the post solving the topic so others with the same issue can readily identify the solution
User avatar
Mike.Sheen
Overflow Error
Overflow Error
 
Posts: 2440
Joined: Tue Feb 12, 2008 11:12 am
Location: Perth, Republic of Western Australia
Topics Solved: 755

Re: Rate Limits

Postby Mike.Sheen » Tue Nov 12, 2019 1:13 pm

Updated to limit by IP address when there is no session info (e.g.: routes not requiring authentication such as the /auth route).

This has also been added as a standard (but disabled) plugin in 07.02.03 - DEV-7900
Attachments
Plugin API Rate Limit.xml
(29.01 KiB) Downloaded 137 times
Mike Sheen
Chief Software Engineer
Jiwa Financials

If I do answer your question to your satisfaction, please mark it as the post solving the topic so others with the same issue can readily identify the solution
User avatar
Mike.Sheen
Overflow Error
Overflow Error
 
Posts: 2440
Joined: Tue Feb 12, 2008 11:12 am
Location: Perth, Republic of Western Australia
Topics Solved: 755

Re: Rate Limits

Postby SBarnes » Tue Nov 12, 2019 2:11 pm

Thanks Mike as always,

Could it be possible to have an ip exclusion from rates range as well only so that if you are running a web store of the same api you don't wack yourself with the rate limits?
Regards
Stuart Barnes
SBarnes
Shihan
Shihan
 
Posts: 1617
Joined: Fri Aug 15, 2008 3:27 pm
Topics Solved: 175

Re: Rate Limits  Topic is solved

Postby Mike.Sheen » Tue Nov 12, 2019 6:08 pm

SBarnes wrote:Could it be possible to have an ip exclusion from rates range as well only so that if you are running a web store of the same api you don't wack yourself with the rate limits?


Sure - was something like the attached what you had in mind?

Basically we just allow an array of CIDR addresses to be provided, and if the caller of the API has their IP address within any of those blocks we skip the rate limit check altogether.

Code: Select all
public void RateLimitRequestFilter(IRequest req, IResponse res, object dto)
{
   // check if we've exceeded the rate limit for the users sessionId or IP Address and throw a 429      
   string[] exclusionIPAddresses = {"14.202.205.230/32", "189.40.31.0/24"};  // array of CIDR addressing scheme addresses to skip the rate limit checks for
   
   if (req.RemoteIp.InCIDRExclusions(exclusionIPAddresses))
      return;
   ....
Attachments
Plugin API Rate Limit.xml
(30.97 KiB) Downloaded 135 times
Mike Sheen
Chief Software Engineer
Jiwa Financials

If I do answer your question to your satisfaction, please mark it as the post solving the topic so others with the same issue can readily identify the solution
User avatar
Mike.Sheen
Overflow Error
Overflow Error
 
Posts: 2440
Joined: Tue Feb 12, 2008 11:12 am
Location: Perth, Republic of Western Australia
Topics Solved: 755

Re: Rate Limits

Postby SBarnes » Tue Nov 12, 2019 6:21 pm

Looks good, thanks Mike
Regards
Stuart Barnes
SBarnes
Shihan
Shihan
 
Posts: 1617
Joined: Fri Aug 15, 2008 3:27 pm
Topics Solved: 175

Re: Rate Limits

Postby Mike.Sheen » Tue Nov 12, 2019 6:51 pm

SBarnes wrote:Looks good, thanks Mike


No worries - I should caution you that I've not tested this whitelisting aspect thoroughly yet - so don't deploy to a production environment until you've tested it yourself or we release 07.02.03 (which will have this as a standard plugin and we will have tested it properly by then).
Mike Sheen
Chief Software Engineer
Jiwa Financials

If I do answer your question to your satisfaction, please mark it as the post solving the topic so others with the same issue can readily identify the solution
User avatar
Mike.Sheen
Overflow Error
Overflow Error
 
Posts: 2440
Joined: Tue Feb 12, 2008 11:12 am
Location: Perth, Republic of Western Australia
Topics Solved: 755

Re: Rate Limits

Postby SBarnes » Wed Nov 13, 2019 7:39 am

Hi Mike,

I have updated it now include the system settings, as well as the following code to overcome the load/compile issue we were having before. Basically the exclusion range is a comma separated string.

NOTE: I haven't tested this as yet.


Code: Select all
Plugin.PluginReferenceCollection.Read();
AppDomain.CurrentDomain.AssemblyResolve += delegate(object sender, System.ResolveEventArgs args) { return AssemblyResolve(sender, args, Plugin); };
Attachments
Plugin API Rate Limit.xml
Update to include System settings
(32.68 KiB) Downloaded 131 times
Regards
Stuart Barnes
SBarnes
Shihan
Shihan
 
Posts: 1617
Joined: Fri Aug 15, 2008 3:27 pm
Topics Solved: 175

Re: Rate Limits

Postby Mike.Sheen » Wed Nov 13, 2019 6:03 pm

SBarnes wrote:I have updated it now include the system settings, as well as the following code to overcome the load/compile issue we were having before. Basically the exclusion range is a comma separated string.


Thanks! I've incorporated your changes into the plugin we will ship with 07.02.03.
Mike Sheen
Chief Software Engineer
Jiwa Financials

If I do answer your question to your satisfaction, please mark it as the post solving the topic so others with the same issue can readily identify the solution
User avatar
Mike.Sheen
Overflow Error
Overflow Error
 
Posts: 2440
Joined: Tue Feb 12, 2008 11:12 am
Location: Perth, Republic of Western Australia
Topics Solved: 755


Return to REST API

Who is online

Users browsing this forum: No registered users and 1 guest

cron