Prometheus Grafana

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update
kubectl create namespace monitoring
helm install prometheus-stack prometheus-community/kube-prometheus-stack -n monitoring

This deploys:

• Prometheus Server
• Alertmanager
• Grafana
• Node Exporters
• Kube State Metrics

kubectl get pods -n monitoring

• Prometheus pod: 9090
• Grafana pod: 3000

Host Based Ingress

root@template-host:~# cat prom-ingress.yaml
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: prom-ingress
spec:
  entryPoints:
    - web
  routes:
  - match: "Host(`prom1`)"
    kind: Rule
    services:
    - name: prometheus-stack-kube-prom-prometheus
      port: 9090

---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: graf-ingress
spec:
  entryPoints:
    - web
  routes:
  - match: "Host(`graf1`)"
    kind: Rule
    services:
    - name: prometheus-stack-grafana
      port: 80

Apply it:

kubectl apply -f prom-ingress.yaml

Add hosts:

/etc/hosts
192.168.3.228 kube-test1 kube-test1.lab.net prom1 graf1

Access with these hosts:

• http://prom1/
• http://graf1/

Creds:

• Username: admin
• password: prom-operator

Password might be different, get it with this:

kubectl get secret -n monitoring prometheus-stack-grafana -o jsonpath="{.data.admin-password}" | base64 --decode

This was already setup by default:

• Login to Grafana (http://localhost:3000).
• Go to Configuration → Data Sources.
• Click Add Data Source → Select Prometheus.
• Set the URL as:
• http://prometheus-stack-kube-prom-prometheus.monitoring.svc.cluster.local:9090
• Save & Test.

Might want to add these dashboards:

• Go to Grafana → Dashboards → Import.
• Enter the dashboard ID.
  • Kubernetes Cluster Monitoring → 315
  • Node Exporter Dashboard → 1860
• Select Prometheus as the data source.
• Click Import.

Persistent Storage

Add persistent storage:

values.yaml

prometheus:
  server:
    persistentVolume:
      enabled: true
      size: 10Gi

Update it with helm:

helm upgrade prometheus-stack prometheus-community/kube-prometheus-stack -n monitoring -f values.yaml