Load Scheduler

See Also

Load Scheduler Reference

The Load Scheduler is used to throttle request rates dynamically during high load, therefore protecting services from overloads and cascading failures. It uses a local token bucket for estimating the allowed token rate. The fill rate of the token bucket gets adjusted by the controller based on the specified policy. Since this component builds upon the Scheduler, it allows defining workloads along with their priority and tokens. The scheduler employs weighted fair queuing of requests to achieve graceful degradation of applications.

This diagram illustrates the working of a load scheduler.

The Load Scheduler's throttling behavior is controlled by the signal at its load_multiplier input port. As the policy circuit adjusts the signal at the load multiplier port, it gets translated to the token refill rate at the agents. At each agent, the adjusted token rate is determined by multiplying the past token rate with the load multiplier. The past 30 seconds of data is used for finding the past token rate.

$$adjusted\_token\_rate = past\_token\_rate * load\_multiplier$$

If the incoming request rate surpasses the adjusted rate, the scheduler starts queuing requests. The queued requests get admitted as tokens become available in an order determined by the scheduler based on the weighted fair queuing algorithm. Any request that fails to be scheduled within its designated timeout is rejected.

Adaptive Load Scheduler

See Also

Adaptive Load Scheduler Reference

Adaptive Load Scheduler is a high-level circuit component that uses the Load Scheduler internally. In addition, it employs a Gradient Controller and an Integrator for computing the load multiplier. From the Agents' perspective, the Load Scheduler and Adaptive Load Scheduler are identical.

Offering a more high-level interface, this component has signal, setpoint, and overload_confirmation ports. The core function of the Adaptive Load Scheduler is its ability to modify the accepted token rate based on the deviation of the input signal from the setpoint. This functionality allows for the translation of health signals into adjustments in token rate, thereby providing an active defense mechanism for the service.

Additionally, by defining workloads with varying priorities and weights (tokens), the scheduler can prioritize certain requests over others, facilitating graceful service degradation during high-traffic periods.