Azure Kubernetes Service (AKS) - How Pod CPU and Memory Usage (%) metrics are derived

Azure Kubernetes Service (AKS) - How Pod CPU and Memory Usage (%) metrics are derived

Overview

Notes
Note: Pod CPU & Memory Usage (%) metrics are supported from Applications Manager v17.81.00
This KB explains how the Pod CPU Usage (%) and Pod Memory Usage (%) metrics are calculated in the Pods tab → Pod CPU and Memory Usage table.
The calculation can be derived in three ways, each offering a different monitoring perspective:
      1. Pod Limit (Recommended)
      2. Pod Request
      3. Node Allocatable
         

Pod Usage (%) Calculation Methods

    1. Formulas:
      1. Pod CPU Usage (%) = (Pod CPU Used (millicores) / Pod CPU Limit (millicores)) × 100
      2. Pod Memory Usage (%) = (Pod Memory Used (MiB) / Pod Memory Limit (MiB)) × 100
    2. Purpose: The Usage (%) calculation based on the Pod limit helps monitoring performance efficiency and detect potential throttling risks.
    3. This is the recommended and default method for deriving Pod Usage (%).

  2. Pod Request based Calculation

    1. Formulas:
      1. Pod CPU Usage (%) = (Pod CPU Used (millicores) / Pod CPU Request (millicores)) × 100
      2. Pod Memory Usage (%) = (Pod Memory Used (MiB) / Pod Memory Request (MiB)) × 100
    2. Purpose: The Usage (%) calculation based on the Pod request helps monitoring resource efficiency and rightsizing opportunities.

  3. Node based Calculation

    1. Formulas:
      1. Pod CPU Usage (%) = (Pod CPU Used (millicores) / Node's Allocatable CPU (millicores)) × 100
      2. Pod Memory Usage (%) = (Pod Memory Used (MiB) / Node's Allocatable Memory (MiB)) × 100
    2. Purpose: The Usage (%) calculation based on the Node allocatable helps monitoring pod utilization relative to node capacity and aids in capacity planning.

How are the Pod metrics retrieved?

      All metrics used in these calculations are fetched from 'kubectl' commands
    1. Pod CPU and Memory Consumption
      1. Command: kubectl top pod --all-namespaces
      2. Retrieved Metrics:
        1. Pod CPU Used (millicores): Current CPU consumption by each pod.
        2. Pod Memory Used (MiB): Current memory consumption by each pod.
    2. Pod Resource Limits and Node Capacity
      1. Command: kubectl describe nodes
      2. Retrieved Metrics:
        1. Pod CPU Limit (millicores): Maximum CPU each pod can use.
        2. Pod Memory Limit (MiB): Maximum memory each pod can use.
        3. Pod CPU Request (millicores): Guaranteed CPU allocation for each pod.
        4. Pod Memory Request (MiB): Guaranteed memory allocation for each pod.
        5. Node's Allocatable CPU (millicores): Available CPU capacity on each node.
        6. Node's Allocatable Memory (MiB): Available memory capacity on each node.
                                                           

How to configure the calculation method?

      The calculation method can be configured by updating the Global Config key in the AM_GLOBALCONFIG table.
    1. Global Config Key: am.cloud.azure.aks.pod.usage.calculation
    2. Possible Values:
      1. podlimits – Pod limit based calculation
      2. podrequests – Pod request based calculation
      3. nodeallocatable – Node allocatable based calculation
    3. Default Behavior: By default, the global config key does not exist. In this case, the Pod CPU and Memory Usage (%) calculation is based on Pod Limits.
    4. Steps to configure: Open the query tool and perform the below steps.
      1. Verify Key Existence: Check whether the key 'am.cloud.azure.aks.pod.usage.calculation' exists in the AM_GLOBALCONFIG table.
        2. QuoteSELECT * FROM AM_GLOBALCONFIG WHERE NAME = 'am.cloud.azure.aks.pod.usage.calculation';
      2. If the key does not exist, to switch the calculation method perform an INSERT operation.
        3. QuoteINSERT INTO AM_GLOBALCONFIG (NAME, VALUE) VALUES ('am.cloud.azure.aks.pod.usage.calculation', 'podrequests');
      3. If the key exists, perform an UPDATE operation to switch the calculation method.
        4. QuoteUPDATE AM_GLOBALCONFIG  SET VALUE = 'nodeallocatable' WHERE NAME = 'am.cloud.azure.aks.pod.usage.calculation';