Rate Limiting / Throttling

Rate Limiting / Throttling

The problem: demand on the public order API exceeds what you can serve, and unbounded traffic, a rush, a scraper, a buggy partner integration hammering the menu endpoint, turns a busy system into a fallen one.

Rate limiting caps how much work you accept; throttling caps how much you send. Both protect a finite resource by refusing or deferring excess rather than collapsing under it (Azure Cloud Design Patterns, Throttling). .NET ships this in the box with System.Threading.RateLimiting, so you rarely roll your own. On a multi-tenant marketplace you usually key the limiter by tenant_id, so one busy restaurant can't crowd the API for the rest.

var limiter = new SlidingWindowRateLimiter(new SlidingWindowRateLimiterOptions
{
    PermitLimit = 100,
    Window = TimeSpan.FromSeconds(1),
    SegmentsPerWindow = 4,
    QueueLimit = 20
});

using var lease = await limiter.AcquireAsync(permitCount: 1);
if (!lease.IsAcquired) return Results.StatusCode(429);

What it buys you in production: a predictable ceiling. You'd rather reject the excess with a clean 429 than let every order request degrade into a timeout. It also keeps you a good citizen against the payment gateway or maps API, both of which enforce quotas of their own.

Skip-if: your traffic is comfortably below any limit and there's no shared resource to protect. A limiter on an internal courier-status endpoint that sees ten requests a minute is ceremony.

Bulkhead

The problem: one slow dependency consumes every thread, connection, or task slot in the pool, and a failure in one corner of the system starves all the others.

The name comes from a ship's hull (Nygard, Release It!): seal it into compartments so one flooded section doesn't sink the vessel. In software you isolate resources per dependency. The surge-pricing service is the obvious candidate: it calls out to demand and weather models, it's slow under load, and it's exactly when load is high that you can least afford it to drag order placement down with it. Give surge pricing its own bounded pool, and when it backs up it exhausts its compartment, not the threads placing orders.

var surgePricing = new ResiliencePipelineBuilder()
    .AddConcurrencyLimiter(permitLimit: 10, queueLimit: 5)
    .Build();
// Surge calls beyond 10 concurrent wait in a queue of 5, then reject.

What it buys you in production: fault isolation. Order placement keeps serving even while the surge-pricing model is timing out, because the two never share a pool. Worst case, an order goes through at the base price instead of failing.

Skip-if: you have one dependency and one workload. Compartments only help when there's something to wall off; a single-purpose courier-location worker doesn't need them.