Client-side Rate Limiting

Overview

When interacting with external APIs or services, adhering to the rate limits imposed by the service provider is crucial. Aperture can model these external API rate limits, enabling client-side implementation in a distributed application. This approach not only prevents potential penalties for exceeding the rate limits, but also allows Aperture to prioritize access to the external API, ensuring essential workloads receive a fair share of the API quota.

Configuration

This policy is based on the Quota Scheduler blueprint. It uses a quota scheduler to regulate the control point some-external-api, which represents all outgoing requests made to an external API from services within a distributed application. The rate limit is applied per api_key label, with an interval set to 1 second, meaning that the token bucket is replenished with 25 tokens every second, up to a maximum of 500 tokens. Lazy sync is enabled on the rate limiter, which allows the rate limit counters on each Agent to sync four times every interval (1 second).

The WFQ Scheduler prioritizes interactive and background requests with respective priorities of 200 and 50, ensuring that interactive calls receive roughly four times the quota share of background requests.

The below values.yaml file can be generated by following the steps in the Installation section.

aperturectl values.yaml

# yaml-language-server: $schema=../../../../../../blueprints/quota-scheduling/base/gen/definitions.json
# Generated values file for quota-scheduling/base blueprint
# Documentation/Reference for objects and parameters can be found at:
# https://docs.fluxninja.com/reference/blueprints/quota-scheduling
policy:
  policy_name: client-side-rate-limiting
  quota_scheduler:
    bucket_capacity: 500
    fill_amount: 25
    selectors:
      - control_point: some-external-api
    rate_limiter:
      label_key: api_key
      interval: 1s
      lazy_sync:
        enabled: false
        num_sync: 4
    scheduler:
      workloads:
        - label_matcher:
            match_labels:
              call_type: background
          parameters:
            priority: "50"
        - label_matcher:
            match_labels:
              call_type: interactive
          parameters:
            priority: "200"

Generated Policy

apiVersion: fluxninja.com/v1alpha1
kind: Policy
metadata:
  labels:
    fluxninja.com/validate: "true"
  name: client-side-rate-limiting
spec:
  circuit:
    components:
    - flow_control:
        quota_scheduler:
          in_ports:
            bucket_capacity:
              constant_signal:
                value: 500
            fill_amount:
              constant_signal:
                value: 25
          rate_limiter:
            interval: 1s
            label_key: api_key
            lazy_sync:
              enabled: false
              num_sync: 4
          scheduler:
            workloads:
            - label_matcher:
                match_labels:
                  call_type: background
              parameters:
                priority: "50"
            - label_matcher:
                match_labels:
                  call_type: interactive
              parameters:
                priority: "200"
          selectors:
          - control_point: some-external-api
    evaluation_interval: 1s
  resources:
    flow_control:
      classifiers: []

Installation

Generate a values file specific to the policy. This can be achieved using the command provided below.

aperturectl blueprints values --name=quota-scheduling/base --version=v2.8.0 --output-file=values.yaml

Adjust the values to match the application requirements. Use the following command to generate the policy.

aperturectl blueprints generate --name=quota-scheduling/base --values-file=values.yaml --output-dir=policy-gen --version=v2.8.0

Apply the policy using the aperturectl CLI or kubectl.

aperturectl

Pass the --kube flag with aperturectl to directly apply the generated policy on a Kubernetes cluster in the namespace where the Aperture Controller is installed.

aperturectl apply policy --file=policy-gen/policies/quota-scheduling.yaml --kube 

kubectl

Apply the policy YAML generated (Kubernetes Custom Resource) using the above example with kubectl.

kubectl apply -f policy-gen/configuration/quota-scheduling-cr.yaml -n aperture-controller

Policy in Action

The quota scheduler successfully ensures that the request rate remains within the specified limits, as seen below, with a steady state of 25 requests per second. Additionally, the workload decisions panel shows that interactive requests receive approximately four times the acceptance rate compared to background requests.

info

Circuit Diagram for this policy.