How to enable prometheus in Podman (Podman-exporter, Node-Exporter)

1. Overview of Monitoring Components

• Prometheus: Collects and stores metrics from configured targets
• Podman Exporter: Provides container-level metrics for Podman workloads
• Node Exporter: Provides system-level metrics (CPU, memory, disk, network)

2. Install and Configure Prometheus ( From Scratch)

Step 1: Remove existing Prometheus (if any)

podman rm -f prometheus

Step 2: Create Prometheus configuration directory

mkdir -p /opt/prometheus

Step 3: Start Prometheus container on port 8087

podman run -d --name prometheus \
  -p 8087:8087 \
  -v /opt/prometheus/prometheus.yml:/etc/prometheus/prometheus.yml:Z \
  docker.io/prom/prometheus:latest \
  --config.file=/etc/prometheus/prometheus.yml \
  --storage.tsdb.path=/prometheus \
  --web.listen-address=":8087"

3. Install Podman Exporter

Step 1: Run Podman Exporter container

podman run -d --name podman-exporter \
  --privileged \
  --network host \
  -v /run/podman/podman.sock:/run/podman/podman.sock:Z \
  -e CONTAINER_HOST=unix:///run/podman/podman.sock \
  quay.io/navidys/prometheus-podman-exporter:latest \
  --collector.enable-all

Step 2: Validate exporter

curl http://<server-ip>:8091/metrics

4. Install Node Exporter

Step 1: Run Node Exporter container

podman run -d --name node-exporter \
  --net=host \
  --pid=host \
  --privileged \
  -v "/:/host:ro,rslave" \
  quay.io/prometheus/node-exporter:latest \
  --path.rootfs=/host

Step 2: Validate Node Exporter

curl http://<server-ip>:9100/metrics

5. Configure Prometheus Scraping

Step 1: Open configuration file

/opt/prometheus/prometheus.yml

Step 2: Add scrape configuration

global:
  scrape_interval: 15s

scrape_configs:
  - job_name: "prometheus"
    static_configs:
      - targets: ["localhost:8087"]

  - job_name: "podman-exporter"
    static_configs:
      - targets: ["<server-ip>:8091"]

  - job_name: "node-exporter"
    static_configs:
      - targets: ["<server-ip>:9100"]

6. Restart Prometheus

podman restart prometheus

7. Validation Steps

1. Open Prometheus UI: http://<server-ip>:8087/targets
2. Verify all targets are in UP state
3. Check metrics endpoints manually using curl

8. Port Summary

9. Troubleshooting Tips

• If Prometheus is not accessible, verify port 8087 is not blocked by firewall
• If targets show DOWN, check network connectivity between Prometheus and exporters
• Use podman logs <container> for debugging