Files
bakery-ia/infrastructure/monitoring/k8s-infra

Kubernetes Infrastructure Monitoring

This directory contains configurations for deploying Kubernetes infrastructure monitoring components that integrate with SigNoz.

Components

Component Purpose Metrics Endpoint
kube-state-metrics Kubernetes object metrics (pods, deployments, nodes, etc.) :8080/metrics
node-exporter Host-level metrics (CPU, memory, disk, network) :9100/metrics

Quick Start (MicroK8s Production)

# 1. Deploy infrastructure monitoring components
./deploy-k8s-infra-monitoring.sh --microk8s install

# 2. Upgrade SigNoz to scrape the new metrics
microk8s helm3 upgrade signoz signoz/signoz \
  -n bakery-ia \
  -f ../signoz/signoz-values-prod.yaml

Usage

Install

# Standard Kubernetes
./deploy-k8s-infra-monitoring.sh install

# MicroK8s
./deploy-k8s-infra-monitoring.sh --microk8s install

Upgrade

./deploy-k8s-infra-monitoring.sh --microk8s upgrade

Uninstall

./deploy-k8s-infra-monitoring.sh --microk8s uninstall

Check Status

./deploy-k8s-infra-monitoring.sh --microk8s status

Dry Run

./deploy-k8s-infra-monitoring.sh --microk8s --dry-run install

Files

  • kube-state-metrics-values.yaml - Helm values for kube-state-metrics
  • node-exporter-values.yaml - Helm values for node-exporter
  • deploy-k8s-infra-monitoring.sh - Deployment automation script

SigNoz Integration

The SigNoz OTel Collector is configured (in signoz-values-prod.yaml) to scrape metrics from:

  • kube-state-metrics.bakery-ia.svc.cluster.local:8080
  • node-exporter-prometheus-node-exporter.bakery-ia.svc.cluster.local:9100

After deploying these components, metrics will appear in SigNoz under:

  • Infrastructure > Kubernetes (for K8s object metrics)
  • Infrastructure > Hosts (for node metrics)

Metrics Available

From kube-state-metrics

  • Pod status, phase, restarts
  • Deployment replicas (desired vs available)
  • Node conditions and capacity
  • PVC status and capacity
  • Resource requests and limits
  • Job/CronJob status

From node-exporter

  • CPU usage per core
  • Memory usage (total, free, cached)
  • Disk I/O and space
  • Network traffic (bytes in/out)
  • System load average
  • Filesystem usage

Troubleshooting

Check if metrics are being scraped

# Port-forward to kube-state-metrics
microk8s kubectl port-forward svc/kube-state-metrics 8080:8080 -n bakery-ia &
curl localhost:8080/metrics | head -50

# Port-forward to node-exporter
microk8s kubectl port-forward svc/node-exporter-prometheus-node-exporter 9100:9100 -n bakery-ia &
curl localhost:9100/metrics | head -50

Check OTel Collector logs

microk8s kubectl logs -l app.kubernetes.io/name=signoz-otel-collector -n bakery-ia --tail=100

Verify pods are running

microk8s kubectl get pods -n bakery-ia | grep -E "(kube-state|node-exporter)"