Skip to main content
Version: 2.7.0

Workload Prioritization

note

The following policy is based on the Load Scheduling with Average Latency Feedback blueprint.

Overview

When dealing with services in resource-limited scenarios, it becomes paramount to prioritize key user experiences and business-critical features over less crucial tasks or background workloads. For instance, in an e-commerce platform, the checkout process must take precedence over functionalities like personalized recommendations, especially during resource shortage or high traffic. Aperture's Weighted Fair Queuing Scheduler (WFQ) enables such prioritization of flows over others based on their labels, ensuring user experience or revenue is maximized during overloads or other failure scenarios.

Configuration

This policy defines service protection on cart-service.prod.svc.cluster.local using a load scheduler and overload detection is based on average latency similar to the protection policy. In addition, workload prioritization is specified in the load scheduler based on user types accessing the service. User types are identified based on the value of a header label http.request.header.user_type. Requests matching label value guest are assigned a priority of 50, whereas those matching subscriber are given a priority of 200.

# yaml-language-server: $schema=../../../../../../blueprints/load-scheduling/average-latency/gen/definitions.json
# Generated values file for load-scheduling/average-latency blueprint
# Documentation/Reference for objects and parameters can be found at:
# https://docs.fluxninja.com/reference/blueprints/load-scheduling/average-latency
policy:
# Name of the policy.
# Type: string
# Required: True
policy_name: workload-prioritization
# List of additional circuit components.
# Type: []aperture.spec.v1.Component
components: []
# The interval between successive evaluations of the Circuit.
# Type: string
evaluation_interval: "10s"
service_protection_core:
adaptive_load_scheduler:
load_scheduler:
# The selectors determine the flows that are protected by this policy.
# Type: []aperture.spec.v1.Selector
# Required: True
selectors:
- control_point: ingress
service: cart-service.prod.svc.cluster.local
# Scheduler parameters.
# Type: aperture.spec.v1.SchedulerParameters
scheduler:
workloads:
- label_matcher:
match_labels:
http.request.header.user_type: guest
parameters:
priority: "50"
- label_matcher:
match_labels:
http.request.header.user_type: subscriber
parameters:
priority: "200"
latency_baseliner:
# Tolerance factor beyond which the service is considered to be in overloaded state. E.g. if EMA of latency is 50ms and if Tolerance is 1.1, then service is considered to be in overloaded state if current latency is more than 55ms.
# Type: float64
latency_tolerance_multiplier: 1.1
# Flux Meter defines the scope of latency measurements.
# Type: aperture.spec.v1.FluxMeter
# Required: True
flux_meter:
selectors:
- service: cart-service.prod.svc.cluster.local
control_point: ingress

Generated Policy

apiVersion: fluxninja.com/v1alpha1
kind: Policy
metadata:
labels:
fluxninja.com/validate: "true"
name: workload-prioritization
spec:
circuit:
components:
- flow_control:
adaptive_load_scheduler:
dry_run: false
dry_run_config_key: dry_run
in_ports:
overload_confirmation:
constant_signal:
value: 1
setpoint:
signal_name: SETPOINT
signal:
signal_name: SIGNAL
out_ports:
desired_load_multiplier:
signal_name: DESIRED_LOAD_MULTIPLIER
observed_load_multiplier:
signal_name: OBSERVED_LOAD_MULTIPLIER
parameters:
alerter:
alert_name: Load Throttling Event
gradient:
max_gradient: 1
min_gradient: 0.1
slope: -1
load_multiplier_linear_increment: 0.025
load_scheduler:
scheduler:
workloads:
- label_matcher:
match_labels:
http.request.header.user_type: guest
parameters:
priority: "50"
- label_matcher:
match_labels:
http.request.header.user_type: subscriber
parameters:
priority: "200"
selectors:
- control_point: ingress
service: cart-service.prod.svc.cluster.local
max_load_multiplier: 2
- query:
promql:
evaluation_interval: 10s
out_ports:
output:
signal_name: SIGNAL
query_string: sum(increase(flux_meter_sum{flow_status="OK", flux_meter_name="workload-prioritization",
policy_name="workload-prioritization"}[30s]))/sum(increase(flux_meter_count{flow_status="OK",
flux_meter_name="workload-prioritization", policy_name="workload-prioritization"}[30s]))
- query:
promql:
evaluation_interval: 30s
out_ports:
output:
signal_name: SIGNAL_LONG_TERM
query_string: sum(increase(flux_meter_sum{flow_status="OK", flux_meter_name="workload-prioritization",
policy_name="workload-prioritization"}[1800s]))/sum(increase(flux_meter_count{flow_status="OK",
flux_meter_name="workload-prioritization", policy_name="workload-prioritization"}[1800s]))
- arithmetic_combinator:
in_ports:
lhs:
signal_name: SIGNAL_LONG_TERM
rhs:
constant_signal:
value: 1.1
operator: mul
out_ports:
output:
signal_name: SETPOINT
evaluation_interval: 10s
resources:
flow_control:
classifiers: []
flux_meters:
workload-prioritization:
selectors:
- control_point: ingress
service: cart-service.prod.svc.cluster.local
infra_meters: {}

info

Circuit Diagram for this policy.

Policy in Action

The traffic generator in this scenario is configured to generate similar traffic pattern (number of concurrent users) for 2 types of users - subscribers and guests.

The below dashboard shows that, during overload periods, requests from subscriber users have a higher acceptance rate than those from guest users.

Workload Prioritization

Demo Video

The below demo video shows the basic concurrency limiter and workload prioritization policy in action within Aperture Playground.

Demo Video