Skip to main content
Version: development

Load Scheduling for PostgreSQL

Introduction

This policy detects traffic overloads and cascading failure build-up on PostgreSQL by checking the real-time percentage of PostgreSQL connections against the maximum number of connections.

It also uses the CPU utilization ratio of the PostgreSQL pod to confirm traffic overloads and cascading failure build-up. The CPU utilization ratio is the percentage of CPU used by the PostgreSQL pod divided by the total CPU available to the pod.

All the PostgreSQL related metrics are collected by the PostgreSQL OpenTelemetry Collector so if the system under observation requires using different metrics for the overload confirmation, the list of available metrics can be used to configure the policy.

A gradient controller calculates a proportional response to limit accepted concurrency. The concurrency is reduced by a multiplicative factor when the service is overloaded, and increased by an additive factor while the service is no longer overloaded.

info

Please see reference for the AdaptiveLoadScheduler component that is used within this blueprint.

Configuration

Blueprint name: load-scheduling/postgresql

Parameters

policy

Parameterpolicy.components
DescriptionList of additional circuit components.
TypeArray of Object (aperture.spec.v1.Component)
Default Value
Expand
[]
Parameterpolicy.policy_name
DescriptionName of the policy.
Typestring
Default Value__REQUIRED_FIELD__
Parameterpolicy.resources
DescriptionAdditional resources.
TypeObject (aperture.spec.v1.Resources)
Default Value
Expand
flow_control:
classifiers: []
Parameterpolicy.evaluation_interval
DescriptionThe interval between successive evaluations of the Circuit.
Typestring
Default Value10s
Parameterpolicy.promql_query
DescriptionPromQL query to detect PostgreSQL overload.
Typestring
Default Value(sum(postgresql_backends) / sum(postgresql_connection_max)) * 100
Parameterpolicy.setpoint
DescriptionSetpoint.
TypeNumber (double)
Default Value__REQUIRED_FIELD__
Parameterpolicy.postgresql
DescriptionConfiguration for PostgreSQL OpenTelemetry receiver. Refer https://docs.fluxninja.com/integrations/metrics/postgresql for more information.
TypeObject (postgresql)
Default Value
Expand
agent_group: default
endpoint: __REQUIRED_FIELD__
password: __REQUIRED_FIELD__
username: __REQUIRED_FIELD__
policy.service_protection_core
Parameterpolicy.service_protection_core.adaptive_load_scheduler
DescriptionParameters for Adaptive Load Scheduler.
TypeObject (aperture.spec.v1.AdaptiveLoadSchedulerParameters)
Default Value
Expand
alerter:
alert_name: Load Throttling Event
gradient:
max_gradient: 1
min_gradient: 0.1
slope: -1
load_multiplier_linear_increment: 0.025
load_scheduler:
selectors:
- control_point: __REQUIRED_FIELD__
service: __REQUIRED_FIELD__
max_load_multiplier: 2
Parameterpolicy.service_protection_core.dry_run
DescriptionDefault configuration for setting dry run mode on Load Scheduler. In dry run mode, the Load Scheduler acts as a passthrough and does not throttle flows. This config can be updated at runtime without restarting the policy.
TypeBoolean
Default Valuefalse
Parameterpolicy.service_protection_core.kubelet_overload_confirmations
DescriptionOverload confirmation signals from kubelet.
TypeObject (kubelet_overload_confirmations)
Default Value
Expand
{}
Parameterpolicy.service_protection_core.overload_confirmations
DescriptionList of overload confirmation criteria. Load scheduler can throttle flows when all of the specified overload confirmation criteria are met.
TypeArray of Object (overload_confirmation)
Default Value
Expand
[]

dashboard

Parameterdashboard.extra_filters
DescriptionAdditional filters to pass to each query to Grafana datasource.
TypeObject (map[string]string)
Default Value
Expand
{}
Parameterdashboard.refresh_interval
DescriptionRefresh interval for dashboard panels.
Typestring
Default Value15s
Parameterdashboard.time_from
DescriptionFrom time of dashboard.
Typestring
Default Valuenow-15m
Parameterdashboard.time_to
DescriptionTo time of dashboard.
Typestring
Default Valuenow
Parameterdashboard.title
DescriptionName of the main dashboard.
Typestring
Default ValueAperture Service Protection for PostgreSQL
dashboard.datasource
Parameterdashboard.datasource.filter_regex
DescriptionDatasource filter regex.
Typestring
Default Value
Parameterdashboard.datasource.name
DescriptionDatasource name.
Typestring
Default Value$datasource

Schemas

driver_criteria

Parameterenabled
DescriptionEnables the driver.
TypeBoolean
Default Value__REQUIRED_FIELD__
Parameterthreshold
DescriptionThreshold for the driver.
TypeNumber (double)
Default Value__REQUIRED_FIELD__

overload_confirmation_driver

Parameterpod_cpu
DescriptionThe driver for using CPU usage as overload confirmation.
TypeObject (driver_criteria)
Default Value
Expand
{}
Parameterpod_memory
DescriptionThe driver for using CPU usage as overload confirmation.
TypeObject (driver_criteria)
Default Value
Expand
{}

kubelet_overload_confirmations

Parametercriteria
DescriptionCriteria for overload confirmation.
TypeObject (overload_confirmation_driver)
Default Value__REQUIRED_FIELD__
Parameterinfra_context
DescriptionKubernetes selector for scraping metrics.
TypeObject (aperture.spec.v1.KubernetesObjectSelector)
Default Value__REQUIRED_FIELD__

overload_confirmation

Parameteroperator
DescriptionThe operator for the overload confirmation criteria. oneof: `gt | lt | gte | lte | eq | neq`
Typestring
Default Value
Parameterquery_string
DescriptionThe Prometheus query to be run. Must return a scalar or a vector with a single element.
Typestring
Default Value
Parameterthreshold
DescriptionThe threshold for the overload confirmation criteria.
TypeNumber (double)
Default Value

postgresql

Parameteragent_group
DescriptionName of the Aperture Agent group.
Typestring
Default Valuedefault
Parametercollection_interval
DescriptionThis receiver collects metrics on an interval.
Typestring
Default Value
Parameterdatabase
DescriptionThe list of databases for which the receiver will attempt to collect statistics.
TypeArray of string
Default Value
Parameterendpoint
DescriptionEndpoint of the PostgreSQL.
Typestring
Default Value__REQUIRED_FIELD__
Parameterinitial_delay
DescriptionDefines how long this receiver waits before starting.
Typestring
Default Value
Parameterpassword
DescriptionPassword of the PostgreSQL.
Typestring
Default Value__REQUIRED_FIELD__
Parametertransport
DescriptionThe transport protocol being used to connect to postgresql. Available options are tcp and unix.
Typestring
Default Value
Parameterusername
DescriptionUsername of the PostgreSQL.
Typestring
Default Value__REQUIRED_FIELD__
tls
Parameterca_file
DescriptionA set of certificate authorities used to validate the database server SSL certificate.
Typestring
Default Value
Parametercert_file
DescriptionA cerficate used for client authentication, if necessary.
Typestring
Default Value
Parameterinsecure
DescriptionWhether to enable client transport security for the postgresql connection.
TypeBoolean
Default Value
Parameterinsecure_skip_verify
DescriptionWhether to validate server name and certificate if client transport security is enabled.
TypeBoolean
Default Value
Parameterkey_file
DescriptionAn SSL key used for client authentication, if necessary.
Typestring
Default Value

Dynamic Configuration

note

The following configuration parameters can be dynamically configured at runtime, without reloading the policy.

Parameters

Parameterdry_run
DescriptionDynamic configuration for setting dry run mode at runtime without restarting this policy. In dry run mode the scheduler acts as pass through to all flow and does not queue flows. It is useful for observing the behavior of load scheduler without disrupting any real traffic.
TypeBoolean
Default Value__REQUIRED_FIELD__