Cloud Monitoring API Usage

Cloud Monitoring API Usage

How Cloud Monitoring Works?

      Cloud Monitoring involves collecting, analyzing, and visualizing performance metrics and telemetry data from cloud infrastructure and applications via APIs.

Cloud Monitoring data collection flow:

  1. API Authentication: Authenticate with cloud providers using secure credentials such as API keys, IAM roles, service principals, or OAuth tokens.
  2. Data Collection: Make scheduled API calls to retrieve resource states, performance metrics, and configuration information at defined intervals.
  3. Data Processing & Presentation: After data collection, data is normalized, processed, and stored for analysis, then visualized.

Types of APIs

  1. LIST APIs: Used to get the list of resources.
  2. GET / DESCRIBE API: Used to get the metadata or configuration data of the specific resource.
  3. Monitor APIs: Used to get the performance data for the resource. (ex: From AWS CloudWatch, Azure Monitor, etc.).

Cloud Monitoring Comparison

 

Native Monitoring

    (AWS CloudWatch, Azure Monitor)

Applications Manager

Cloud Support

Single Cloud

Multi Cloud

Threshold / Alarm Pricing

Charged per alarm / threshold

NA

API Pricing
  1. AWS - Metrics Ingestion
  2. Azure - Platform metrics
  1. AWS - GetMetricData API
  2. Azure - Platform metrics
  1. Detailed RCA &
  2. Customized Dashboard
  1. Basic level
  1. Advanced level

Cloud API Pricing

AWS

For AWS monitoring, please find the API cost below
  1. CloudWatch
    1. GetMetricData: $0.01 per 1,000 metrics requested
  2. GET / DESCRIBE API: Free
InfoNote:
Please find the list of API permissions needed for Amazon monitoring here.
Please find the respective service endpoints used for monitoring in this document.

Microsoft Azure

For Azure monitoring, please find the API cost below
  1. Platform Metrics - First 1M API calls/month free, then $0.01 per 1,000 calls.
    1. In Applications Manager, we use Metrics - List API to fetch Azure monitors metrics.
Note: Please find the list of APIs used for monitoring here.

Cost Calculation

AWS Pricing (Example)

Please find the calculation for the API cost incurred in AWS monitors for the default poll interval - 5 mins.
Notes
Note: AWS bills the GetMetricData API per metric requested, not per API call.
Monitor type Metrics
per poll		 Metrics per hour
(12 polls)		 Metrics per month
(× 8,640)		 Monthly cost
per monitor		 Metrics per month,
10 monitors		 Monthly cost,
10 monitors
Amazon SQS 6 72 51,840 $0.52 518,400 $5.1
Amazon API Gateway (REST) 7 84 60,480 $0.60 604,800 $6.0
AWS Transfer Family 11 132 95,040 $0.95 950,400 $9.5
EC2 Instance
 Total - 33
- Instance (21) 
- Burst CPU (4)
- 1 EBS (8)
 396 2,85,120 $2.85 28,51,200 $28.5
EC2 Instance monitoring with CloudWatch agent (additional cost)
 OS Metrics - 5
Network - 20
 60
240
 43,200
1,72,800
 $0.43
$1.73
 4,32,000
17,28,000
 $4.32
$17.28
EKS Cluster
 Total - 14
- cluster (7) 
- 1 node (3) 
- 1 pod (2) 
- 1 namespace (1)
- 1 service (1) 
 168 120,960 $1.21 12,09,600 $12.1

Microsoft Azure (Example)

Please find the calculation for the API cost incurred in Azure monitors for the default poll interval - 5 mins.
Notes
Note: Azure bills metric queries per API call, with the first 1,000,000 calls per month free.

Monitor type API calls
per poll		 Calls per hour
(12 polls)		 Calls per month,
1 monitor		 Calls per month,
10 monitors		 Monthly cost,
1 monitor		 Monthly cost,
10 monitors
Azure ExpressRoute 4 48 34,560 345,600 $0.35 $3.46
Azure Service Bus (ARM) 2 24 17,280 172,800 $0.17 $1.73

Cost Optimization Recommendations

  1. Reduce API Call Frequency: 
    1. Increase the poll interval of the non-critical monitors (UAT) to reduce the API call frequency (For ex: increase the poll interval from 5 mins to 15 mins).
  2. Prioritize Critical Resources:
    1. Leverage advanced discovery filters in Applications Manager to target essential resources.
      1. Utilize regex-based filtering, choose required services & regions, etc.
    2. Use stop discovery to halt further discovery once sufficient resource required for monitoring have been discovered.

Cloud Monitoring Architecture