Prioritization

Overview

When dealing with services in resource-limited scenarios, it becomes paramount to prioritize key user experiences and business-critical features over less crucial tasks or background workloads. For instance, in an e-commerce platform, the checkout process must take precedence over functionalities like personalized recommendations, especially during resource shortage or high traffic. Aperture's Weighted Fair Queuing Scheduler (WFQ) enables such prioritization of flows over others based on their labels, ensuring user experience or revenue is maximized during overloads or other failure scenarios.

Configuration

This policy defines service protection on cart-service.prod.svc.cluster.local using a load scheduler and overload detection is based on average latency similar to the protection policy. In addition, workload prioritization is specified in the load scheduler based on user types accessing the service. User types are identified based on the value of a header label http.request.header.user_type. Requests matching label value guest are assigned a priority of 50, whereas those matching subscriber are given a priority of 200.

aperturectl values.yaml

# yaml-language-server: $schema=../../../../../../blueprints/policies/service-protection/average-latency/gen/definitions.json
# Generated values file for policies/service-protection/average-latency blueprint
# Documentation/Reference for objects and parameters can be found at:
# https://docs.fluxninja.com/reference/blueprints/policies/service-protection/average-latency
policy:
  # Name of the policy.
  # Type: string
  # Required: True
  policy_name: workload-prioritization
  # List of additional circuit components.
  # Type: []aperture.spec.v1.Component
  components: []
  # The interval between successive evaluations of the Circuit.
  # Type: string
  evaluation_interval: "10s"
  service_protection_core:
    adaptive_load_scheduler:
      load_scheduler:
        # The selectors determine the flows that are protected by this policy.
        # Type: []aperture.spec.v1.Selector
        # Required: True
        selectors:
          - control_point: ingress
            service: cart-service.prod.svc.cluster.local
        # Scheduler parameters.
        # Type: aperture.spec.v1.SchedulerParameters
        scheduler:
          workloads:
            - label_matcher:
                match_labels:
                  http.request.header.user_type: guest
              parameters:
                priority: 50
            - label_matcher:
                match_labels:
                  http.request.header.user_type: subscriber
              parameters:
                priority: 200
  latency_baseliner:
    # Tolerance factor beyond which the service is considered to be in overloaded state. E.g. if EMA of latency is 50ms and if Tolerance is 1.1, then service is considered to be in overloaded state if current latency is more than 55ms.
    # Type: float64
    latency_tolerance_multiplier: 1.1
    # Flux Meter defines the scope of latency measurements.
    # Type: aperture.spec.v1.FluxMeter
    # Required: True
    flux_meter:
      selectors:
        - service: cart-service.prod.svc.cluster.local
          control_point: ingress

Generated Policy

apiVersion: fluxninja.com/v1alpha1
kind: Policy
metadata:
  annotations:
    fluxninja.com/blueprint-name: policies/service-protection/average-latency
    fluxninja.com/blueprints-uri: local
    fluxninja.com/values:
      '{"policy": {"components": [ ], "evaluation_interval": "1s",
      "latency_baseliner": {"flux_meter": {"selectors": [{"control_point": "ingress",
      "service": "cart-service.prod.svc.cluster.local"}]}, "latency_tolerance_multiplier":
      1.1000000000000001}, "policy_name": "workload-prioritization", "service_protection_core":
      {"adaptive_load_scheduler": {"load_scheduler": {"scheduler": {"workloads": [{"label_matcher":
      {"match_labels": {"http.request.header.user_type": "guest"}}, "parameters":
      {"priority": 50}}, {"label_matcher": {"match_labels": {"http.request.header.user_type":
      "subscriber"}}, "parameters": {"priority": 200}}]}, "selectors": [{"control_point":
      "ingress", "service": "cart-service.prod.svc.cluster.local"}]}}}}}'
  labels:
    fluxninja.com/validate: "true"
  name: workload-prioritization
spec:
  circuit:
    components:
      - flow_control:
          adaptive_load_scheduler:
            dry_run: false
            dry_run_config_key: dry_run
            in_ports:
              overload_confirmation:
                constant_signal:
                  value: 1
              setpoint:
                signal_name: SETPOINT
              signal:
                signal_name: SIGNAL
            out_ports:
              desired_load_multiplier:
                signal_name: DESIRED_LOAD_MULTIPLIER
              observed_load_multiplier:
                signal_name: OBSERVED_LOAD_MULTIPLIER
            parameters:
              alerter:
                alert_name: Load Throttling Event
              gradient:
                max_gradient: 1
                min_gradient: 0.1
                slope: -1
              load_multiplier_linear_increment: 0.0025
              load_scheduler:
                scheduler:
                  workloads:
                    - label_matcher:
                        match_labels:
                          http.request.header.user_type: guest
                      parameters:
                        priority: 50
                    - label_matcher:
                        match_labels:
                          http.request.header.user_type: subscriber
                      parameters:
                        priority: 200
                selectors:
                  - control_point: ingress
                    service: service1-demo-app.demoapp.svc.cluster.local
              max_load_multiplier: 2
      - query:
          promql:
            evaluation_interval: 10s
            out_ports:
              output:
                signal_name: SIGNAL
            query_string:
              sum(increase(flux_meter_sum{flow_status="OK", flux_meter_name="workload-prioritization"}[30s]))/sum(increase(flux_meter_count{flow_status="OK",
              flux_meter_name="workload-prioritization"}[30s]))
      - arithmetic_combinator:
          in_ports:
            lhs:
              signal_name: SIGNAL
          out_ports:
            desired_load_multiplier:
              signal_name: DESIRED_LOAD_MULTIPLIER
            observed_load_multiplier:
              signal_name: OBSERVED_LOAD_MULTIPLIER
          parameters:
            alerter:
              alert_name: Load Throttling Event
            gradient:
              max_gradient: 1
              min_gradient: 0.1
              slope: -1
            load_multiplier_linear_increment: 0.0025
            load_scheduler:
              scheduler:
                workloads:
                  - label_matcher:
                      match_labels:
                        http.request.header.user_type: guest
                    parameters:
                      priority: 50
                  - label_matcher:
                      match_labels:
                        http.request.header.user_type: subscriber
                    parameters:
                      priority: 200
              selectors:
                - control_point: ingress
                  service: cart-service.prod.svc.cluster.local
            max_load_multiplier: 2
      - query:
          promql:
            evaluation_interval: 10s
            out_ports:
              output:
                signal_name: MAX_EMA
      - ema:
          in_ports:
            input:
              signal_name: SIGNAL
            max_envelope:
              signal_name: MAX_EMA
          out_ports:
            output:
              signal_name: SIGNAL_EMA
          parameters:
            correction_factor_on_max_envelope_violation: 0.95
            ema_window: 1500s
            warmup_window: 60s
      - arithmetic_combinator:
          in_ports:
            lhs:
              signal_name: SIGNAL_EMA
            rhs:
              constant_signal:
                value: 1.1
          operator: mul
          out_ports:
            output:
              signal_name: SETPOINT
    evaluation_interval: 10s
  resources:
    flow_control:
      classifiers: []
      flux_meters:
        workload-prioritization:
          selectors:
            - control_point: ingress
              service: cart-service.prod.svc.cluster.local

info

Circuit Diagram for this policy.

Policy in Action

The traffic generator in this scenario is configured to generate similar traffic pattern (number of concurrent users) for 2 types of users - subscribers and guests.

The below dashboard shows that, during overload periods, requests from subscriber users have a higher acceptance rate than those from guest users.

Demo Video

The below demo video shows the basic concurrency limiter and workload prioritization policy in action within Aperture Playground.