Decoding System Design of Rate Limiter: 2

Hello and welcome back reader to this 2nd instalment of our rate limiter system design, in this section, we will fully uncover the remaining types of rate limiter algorithms that were not discussed in the first part, if you are reading this without reading the first one I recommend you first to visit this article.

Until now we explored the two types of rate limiters, namely token buckets and leaking buckets, and moving forward with this article we will go uncover other three types of rate limiters as listed below.

1. Fixed window counter

2. Sliding window log

3. Sliding window counter

Fixed window counter:

As the name suggests, the fixed window counter algorithm solves the problems which are faced by the previous two algorithms by dividing the given timeline into fixed-sized windows and assigning the counter for each window the algorithm details steps as follows

1. The fixed window counter algorithm divides the given timeline into fixed-sized windows, where each window is assigned a counter.

2. Each request in that given window increments the counter by one

3. Once the counter reaches the pre-defined threshold value, new requests are dropped or rejected till the new window starts.

By following the step mentioned in the above algorithm you can design a better rate limiter which is based on the request counts in the given time frame, the representation for the same is shown below.

From the above representation, we can see that each window is of 1 sec and the maximum allowed request per window is 3 requests, when this counter of 3 requests per window increases the required threshold, the further requests are blocked or dropped as shown in the above representation.

But fixed window counter has drawbacks, such as when there are large number of requests are passed through the edge of the window, now there is chances of more request being passed through the rate limiter which is greater then threshold value.

Sliding window log algorithm:

The sliding window log fixes the major issue, which was faced by the fixed window by maintaining the log, of the particular request timestamp the algorithms are as follows

1. The algorithms keep track of the request timestamps. Timestamp data is usually kept in a cache such as sorted sets of Redis.

2. When a new request comes in, remove all the outdated timestamps. Outdated timestamps are defined as those older than the start of the current time window.

3. Add timestamp of the new request to the log.

4. If the log size is the same or lower than the allowed count the request is accepted or it will get rejected.

This type of rate-limiter is very accurate as it stores the log of each request timestamp, due to this storage sometimes it consumes a lot of memory as the request might get rejected but its log is still stored.

Sliding window counter:

Sliding window counter is a hybrid and a combination version of fixed window counter and sliding window log. This algorithm uses both steps as shown in the below figure.

Let us go in-depth about the above representation, from the above diagram let's assume our rate limiter has a maximum of 9 requests per minute and there are 6 request in the previous minute and 3 requests in the current minute. For the request that occupies 30% of the current minute, the number of requests in the rolling window is calculated using the following formula credits@ByteByteGo

1. Requests in the current window + requests in the previous window* overlap % of the rolling window and previous window

2. Using this formula we get 3+6*0.7 = 7.2 requests per minute, this result can be bounded up or down depending on our use cases for the particular rate limiter

Since our rate limiter allows a maximum of 9 request per minute this request will be passed and if one more request comes then it will be rejected or stored in the queue for further processing.

Wrap up:

So in this part, we uncovered the remaining three types of rate limiters that we usually use as substitutions to token buckets and leaking buckets. In our real-world problem, the rate limiter is handled by the API gateway services provided by different service providers. The primary function of the rate limiter remains the same to block unnecessary requests or the traffic request coming from the same source or IP. Depending on the type of rate limiter used we can block the unnecessary requests for e.g. using the sliding window counter algorithm for rate limiter will allow 0.003% of unnecessary requests out of 400 million requests.

I have learnt this on the internet and some of the references, I have mentioned below do checkout the link for in-depth understanding.