Concurrency Limiter Middleware

What are Concurrency limiters?

Concurrency limiters are programming concepts that aid in managing concurrent access to a resource. It implements a rate limiter that limits the number of concurrently running requests. It safeguards apps and APIs from malicious attacks and excessive use.

Concurrency limiters are vital in programming, particularly in web development, where they defend against malicious attacks and guarantee that resources are used efficiently.

What changed in .NET 8?

In .NET 8, the ConcurrencyLimiterMiddleware and its associated methods and classes were tagged as outdated.

If you need rate-limiting capabilities, use the newer and more capable .NET 7 middleware (RateLimiterApplicationBuilderExtensions.UseRateLimiter). The rate-limiting API from .NET 7 and beyond contains a concurrency limiter and numerous other techniques you can use in your application.

Use the following code snippet, for example, to add rate limitation to a short-circuit route.

Add the code to the Program.cs.

app.UseRateLimiter(new RateLimiterOptions()
    .AddConcurrencyLimiter("only-one-at-a-time-market", (options) =>
    {
        options.PermitLimit = 2;
        options.QueueLimit = 15;
        options.QueueProcessingOrder = QueueProcessingOrder.OldestFirst;
    }));

app.MapGet("/market", async (context) =>
{
    await Task.Delay(500);
    context.Response.Redirect("https://contoso.com/market?redir=");
}).RequireRateLimiting("only-one-at-a-time-market");

app.UseRateLimiter(new RateLimiterOptions()
    .AddConcurrencyLimiter("only-one-at-a-time-crm", (options) =>
    {
        options.PermitLimit = 6;
        options.QueueLimit = 35;
        options.QueueProcessingOrder = QueueProcessingOrder.NewestFirst;
    }));

app.MapGet("/crm", async (context) =>
{
    await Task.Delay(100);
    context.Response.Redirect("https://contoso.com/crm?redir=");
}).RequireRateLimiting("only-one-at-a-time-crm");

app.UseRateLimiter(new RateLimiterOptions()
    .AddConcurrencyLimiter("only-one-at-a-time-sales", (options) =>
    {
        options.PermitLimit = 4;
        options.QueueLimit = 25;
        options.QueueProcessingOrder = QueueProcessingOrder.OldestFirst;
    }));

app.MapGet("/sales", async (context) =>
{
    await Task.Delay(300);
    context.Response.Redirect("https://contoso.com/sales?redir=");
}).RequireRateLimiting("only-one-at-a-time-sales");

To begin using this code, please ensure you complete the necessary configuration steps first.

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddRateLimiter(o => o
    .AddFixedWindowLimiter(policyName: "fixed", options =>
    {
        // configuration
    }));

Add app.UseRateLimiter() after builder.Build().

var app = builder.Build();
app.UseRateLimiter();

To add rate-limiting middleware to the application pipeline, use the app.useRateLimiter method.

The rate limiter is configured using the RateLimiterOptions argument. AddConcurrencyLimiter adds a concurrency limiter to the rate limiter with the policy name “only-one-at-a-time-market/crm/sales.” The PermitLimit property specifies the maximum number of concurrent requests allowed, whereas the QueueLimit property specifies the maximum number of queued requests. The QueueProcessingOrder parameter sets how requests from the queue are handled.

The app.MapGet method maps the /market, /crm e /sales route to a delegate, which then asynchronously waits [x]milliseconds before redirecting the request to “https://contoso.com/[key]?redir= The RequireRateLimiting method is used to compel rate limiting for the ‘/[key]’ route with the policy “name only-one-at-a-time-[key].”

The rate-limiting API is a new feature in.NET 7 that allows you to protect a resource from being overloaded by restricting the number of times it can be accessed.

Four rate-limiting algorithms included in .NET 8

1. Concurrency limit: This restricts the number of concurrent requests that can access a resource. If your limit is 10, only 10 requests can access a resource simultaneously, and the 11th request will be denied. When a request is completed, the number of permissible requests grows to one; when another request is completed, the number climbs to two, and so on.

public class ConcurrencyLimiterMiddleware
{
    private readonly RequestDelegate _next;
    private static SemaphoreSlim _semaphore = new SemaphoreSlim(10); // 10 concurrent requests

    public ConcurrencyLimiterMiddleware(RequestDelegate next)
    {
        _next = next;
    }

    public async Task InvokeAsync(HttpContext context)
    {
        if (_semaphore.CurrentCount == 0)
        {
            context.Response.StatusCode = 429; // Too Many Requests
            return;
        }

        await _semaphore.WaitAsync();

        try
        {
            await _next(context);
        }
        finally
        {
            _semaphore.Release();
        }
    }
}

2. Token bucket limit: This restricts the number of requests depending on a set number of allowable requests. Consider a bucket filled to the brim with tokens. When a request is received, it takes a token and stores it indefinitely. Someone adds a certain number of tokens back to the bucket after some constant period, never adding more than the bucket can contain. When a request comes in and the bucket is empty, the request is refused access to the resource.

public class TokenBucketLimiterMiddleware
{
    private readonly RequestDelegate _next;
    private static int _tokenCount = 100; // Total tokens
    private static object _lock = new object();

    public TokenBucketLimiterMiddleware(RequestDelegate next)
    {
        _next = next;
    }

    public async Task InvokeAsync(HttpContext context)
    {
        lock (_lock)
        {
            if (_tokenCount == 0)
            {
                context.Response.StatusCode = 429; // Too Many Requests
                return;
            }

            _tokenCount--; // Take a token
        }

        // Periodically refill the bucket
        // (This would typically be done with a background task)

        await _next(context);
    }
}

3. Fixed window limit: Makes use of the concept of a window, which will also be used in the following method. The window is the period before we go on to the next window where our limit is applied. Moving to the next window in the fixed window case entails resetting the limit to its beginning point.

public class FixedWindowLimiterMiddleware
{
    private readonly RequestDelegate _next;
    private static int _requestCount = 0;
    private static DateTime _windowStart = DateTime.UtcNow;
    private static readonly TimeSpan _windowSize = TimeSpan.FromMinutes(1); // 1 minute window
    private static readonly int _maxRequests = 100;

    public FixedWindowLimiterMiddleware(RequestDelegate next)
    {
        _next = next;
    }

    public async Task InvokeAsync(HttpContext context)
    {
        if ((DateTime.UtcNow - _windowStart) > _windowSize)
        {
            _requestCount = 0;
            _windowStart = DateTime.UtcNow;
        }

        if (_requestCount >= _maxRequests)
        {
            context.Response.StatusCode = 429; // Too Many Requests
            return;
        }

        _requestCount++;

        await _next(context);
    }
}

4. Sliding window limit: This approach is similar to the fixed window algorithm, but instead of resetting the limit to its initial position, the window slides forward by a defined length of time.

public class SlidingWindowLimiterMiddleware
{
    private readonly RequestDelegate _next;
    private static Queue<DateTime> _requestTimestamps = new Queue<DateTime>();
    private static readonly TimeSpan _windowSize = TimeSpan.FromMinutes(1); // 1 minute window
    private static readonly int _maxRequests = 100;

    public SlidingWindowLimiterMiddleware(RequestDelegate next)
    {
        _next = next;
    }

    public async Task InvokeAsync(HttpContext context)
    {
        while (_requestTimestamps.Any() && (DateTime.UtcNow - _requestTimestamps.Peek()) > _windowSize)
        {
            _requestTimestamps.Dequeue();
        }

        if (_requestTimestamps.Count >= _maxRequests)
        {
            context.Response.StatusCode = 429; // Too Many Requests
            return;
        }

        _requestTimestamps.Enqueue(DateTime.UtcNow);

        await _next(context);
    }
}

Could you kindly show me how to use this in Program.cs? Please select just one limiter. 😊

app.UseMiddleware<ConcurrencyLimiterMiddleware>();

app.UseMiddleware<TokenBucketLimiterMiddleware>();

app.UseMiddleware<FixedWindowLimiterMiddleware>();

app.UseMiddleware<SlidingWindowLimiterMiddleware>();

Observations: Place these middleware examples before any middleware that processes requests, such as app.Use() or app.Mvc().UseEndpoints().

Check that the middleware is registered in the proper sequence. In ASP.NET Core, middleware is executed in the order it is added to the pipeline.

Consider leveraging known libraries or ASP.NET Core’s built-in rate limitation and concurrency management features for more sophisticated applications, particularly in distributed systems. These are exemplary examples that may require changes for production use.

Remember that each middleware has its state and behavior that may or may not be shared across many instances of your application. You may require a distributed cache or a similar method to share state in distributed applications.

Conclusion

The article introduces.NET 7’s rate-limiting API. This functionality helps developers avoid resource overload. It limits resource requests to do this.

Later in the text, the four.NET 7 rate-limiting techniques are discussed. These include concurrency, token buckets, and fixed and sliding window limits. The article gives code samples of these methods to help understand.

The post also mentions a major.NET 8 update. It says ConcurrencyLimiterMiddleware, and its methods and classes are outdated. Instead, developers should utilize RateLimiterApplicationBuilderExtensions.Improve efficiency with RateLimiter.

The paper concludes with crucial recommendations for developers working on complicated programs, primarily distributed systems. It suggests using popular libraries or ASP.NET Core’s rate limiting and concurrency control. It also emphasizes the relevance of middleware order and state in the application process.