高可用Kuberbetes部署Prometheus + Grafana

概述

阅读官方文档部署部署Prometheus + Grafana

GitHub - prometheus-operator/kube-prometheus at release-0.10

环境

步骤

下周官方github仓库

git clone https://github.com/prometheus-operator/kube-prometheus.git
git checkout release-0.10

进入工作目录

cd kube-prometheus/manifests
mkdir -p adapter  alertmanager  blackbox  grafana  kube-state-metrics  node-exporter  operator  prometheus

修改镜像地址

主要是镜像难寻，全靠运气。

prometheusAdapter-deployment.yaml

修改为：thinkingdata/prometheus-adapter:v0.10.0 镜像可用

kubeStateMetrics-deployment.yaml

修改为：bitnami/kube-state-metrics:2.7.0 镜像可用

部署服务

根据官方文档部署服务

kubectl apply --server-side -f manifests/setup
until kubectl get servicemonitors --all-namespaces ; do date; sleep 1; echo ""; done
kubectl apply -f manifests/

验证

NodePort服务暴露

由ClusterIP修改为NodePort

网页访问验证：

只能在指定的节点访问服务。

如图是:

• node04访问grafana + Prometheus。
• master03访问Prometheus

默认登录账号密码都为 admin 登录就会要求你重设密码，重设密码仍为admin

Ingress服务暴露

---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:name: prom-ingressnamespace: monitoring
spec:ingressClassName: nginxrules:- host: alert.k8s.comhttp:paths:- path: /pathType: Prefixbackend:service:name: alertmanager-mainport:number: 9093- host: grafana.k8s.comhttp:paths:- path: /pathType: Prefixbackend:service:name: grafanaport:number: 3000- host: prom.k8s.comhttp:paths:- path: /pathType: Prefixbackend:service:name: prometheus-k8sport:number: 9090

配置域名解析

Error

ingress 部署失败

Error from server (InternalError): error when creating "ingress.yml": Internal error occurred: failed calling webhook "validate.nginx.ingress.kubernetes.io": Posetworking/v1/ingresses?timeout=10s": dial tcp 10.96.222.96:443: connect: connection refused

删除错误pod

kubectl get pod -n ingress-nginx | grep Evicted | awk '{print $1}' | xargs kubectl delete pod -n ingress-nginx

重新部署pod

kubectl delete  pod/ingress-nginx-controller-6ff65d977f-q2kw9  -n ingress-nginx

备注

使用kube-prometheus部署k8s监控(最新版)_净夜凡尘的博客-CSDN博客

Deploy Grafana on Kubernetes | Grafana documentation

kubernetes 部署Prometheus监控集群传统部署方案_ghostwritten的博客-CSDN博客