建设一个公司网站/网站流量统计平台

摘要

The Elastic Stack，包括Elasticsearch、Kibana、Beats和Logstash（也成为ELK Stack）

Elasticsearch：简称ES，是一个开源的高扩展的分布式全文搜索引擎，是整个Elastic Stack技术栈的核心。它可以近乎实时地存储、检索数据；本身扩展性很好，可以扩展到上百台服务器，处理PB级的数据。

本文主要讲解如何部署Elasticsearch，使用最新版8.15.3。

环境准备

环境信息

主机名	操作系统版本	IP地址
elk1	Centos7	192.168.30.133
elk2	Centos7	192.168.30.134
elk3	Centos7	192.168.30.135

系统初始化

#关闭防火墙
systemctl stop firewalld
systemctl disable firewalld
#关闭selinux
sed -i 's/enforcing/disabled/' /etc/selinux/config # 永久
setenforce 0 # 临时
#查看selinux状态
getenforce
#根据规划设置主机名
hostnamectl set-hostname
#添加hosts
cat >> /etc/hosts << EOF
192.168.30.133 elk1
192.168.30.134 elk2
192.168.30.135 elk3
EOF
# 创建用户，因为elastic相关服务不允许root启动
groupadd elk
useradd elk -g elk
# 创建数据及⽇志⽂件并授权
mkdir -pv /opt/elk/
chown -R elk:elk /opt/elk/

启动先决条件

• 调整进程最大打开文件数数量

#临时设置
ulimit -n 65535
#永久设置，退出重连生效
vi /etc/security/limits.conf
elk - memlock unlimited
elk - nproc 4096    ##noproc 是代表最大进程数
elk - nofile 65535  ##nofile 是代表最大文件打开数
#验证
ulimit -n

• 调整进程最大虚拟内存区域数量

#临时设置
sysctl -w vm.max_map_count=262144
#永久设置
echo "vm.max_map_count=262144" >> /etc/sysctl.conf
sysctl -p

下载&安装

• 下载页面
  
• Elasticsearch下载地址
• 下载

mkdir -p /opt/elk/ && cd /opt/elk/
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-8.15.3-linux-x86_64.tar.gz
tar -xzf elasticsearch-8.15.3-linux-x86_64.tar.gz
cd elasticsearch-8.15.3/

修改elasticsearch.yml

vi config/elasticsearch.yml

#必改配置
cluster.name: elk-cluster # 集群名称
node.name: elk1 # 集群节点名称
network.host: 0.0.0.0 # 监听地址
discovery.seed_hosts: ["192.168.30.133", "192.168.30.134","192.168.30.135"] # 集群节点列表
cluster.initial_master_nodes: ["elk1"] # 首次启动指定的Master节点
#可选配置
path.data: data # 数据目录
path.logs: logs # 日志目录
bootstrap.memory_lock: false # 锁内存，尽量不使⽤交换内存
http.port: 9200 # 监听端口
http.cors.allow-origin: "*" # Only use unrestricted value for local development
# Use a specific origin value in production, like `http.cors.allow-origin: "https://<my-website-domain.example>"`
http.cors.enabled: true
http.cors.allow-credentials: true
http.cors.allow-methods: OPTIONS, POST
http.cors.allow-headers: X-Requested-With, X-Auth-Token, Content-Type, Content-Length, Authorization, Access-Control-Allow-Headers, Accept
action.auto_create_index: '*'

控制台启动

# 不允许root用户启动
chown -R elk:elk /opt/elk/
su elk
#控制台启动
./bin/elasticsearch
#后台启动
./bin/elasticsearch -d

• 启动成功截图
  
• elastic密码: Ne5=s3QNimmzOey0D=kF
• CA证书指纹: c493885f81be0090bc625aba200706439038ab34b1f26b183565676b681c9dfd

Linux服务启动

修改/etc/systemd/system/elasticsearch.service

[Unit]
Description=Elasticsearch
After=network.target[Service]
User=elk
Group=elk
LimitNOFILE=65535
LimitNPROC=4096
LimitMEMLOCK=infinity
ExecStart=/opt/elk/elasticsearch-8.15.3/bin/elasticsearch[Install]
WantedBy=multi-user.target

• 注册服务开机启动: systemctl enable elasticsearch.service
• 手动启动服务: systemctl start elasticsearch.service
• 查看ES服务状态: systemctl status elasticsearch.service

访问验证

查看集群信息

运行命令

curl -ik -u elastic:m+NBIqOO+jX6hu+_V8Dd https://127.0.0.1:9200/

返回

HTTP/1.1 200 OK
X-elastic-product: Elasticsearch
content-type: application/json
content-length: 529{"name" : "node-1","cluster_name" : "elk-cluster","cluster_uuid" : "fPVHn9FzSz6FzaVznIqj-Q","version" : {"number" : "8.15.3","build_flavor" : "default","build_type" : "zip","build_hash" : "f97532e680b555c3a05e73a74c28afb666923018","build_date" : "2024-10-09T22:08:00.328917561Z","build_snapshot" : false,"lucene_version" : "9.11.1","minimum_wire_compatibility_version" : "7.17.0","minimum_index_compatibility_version" : "7.0.0"},"tagline" : "You Know, for Search"
}

查看es健康状态

运行命令

curl -ik -u elastic:m+NBIqOO+jX6hu+_V8Dd https://127.0.0.1:9200/_cat/health

返回

HTTP/1.1 200 OK
X-elastic-product: Elasticsearch
content-type: text/plain; charset=UTF-8
Transfer-Encoding: chunked1731134586 06:43:06 elk-cluster green 1 1 45 45 0 0 0 0 - 100.0%

• green: 集群所有数据都处于正常状态
• yellow: 集群所有数据都可以访问，但一些数据的副本还没有分配
• red: 集群部分数据不可访问

查看集群节点

运行命令

curl -ik -u elastic:m+NBIqOO+jX6hu+_V8Dd -XGET "https://127.0.0.1:9200/_cat/nodes?pretty"

返回

HTTP/1.1 200 OK
X-elastic-product: Elasticsearch
content-type: text/plain; charset=UTF-8
Transfer-Encoding: chunked192.168.4.103 10 97 1    cdfhilmrstw * node-1

查询集群状态

运行命令

curl -ik -u elastic:m+NBIqOO+jX6hu+_V8Dd -XGET "https://127.0.0.1:9200/_cluster/health?pretty"

返回

HTTP/1.1 200 OK
X-elastic-product: Elasticsearch
content-type: application/json
content-length: 466{"cluster_name" : "elk-cluster","status" : "green","timed_out" : false,"number_of_nodes" : 1,"number_of_data_nodes" : 1,"active_primary_shards" : 45,"active_shards" : 45,"relocating_shards" : 0,"initializing_shards" : 0,"unassigned_shards" : 0,"delayed_unassigned_shards" : 0,"number_of_pending_tasks" : 0,"number_of_in_flight_fetch" : 0,"task_max_waiting_in_queue_millis" : 0,"active_shards_percent_as_number" : 100.0
}

生成service token

运行命令

curl -ik -u elastic:m+NBIqOO+jX6hu+_V8Dd -XPOST https://localhost:9200/_security/service/elastic/kibana/credential/token/mytoken

返回

{"created":true,"token":{"name":"mytoken","value":"AAEAAWVsYXN0aWMva2liYW5hL215dG9rZW46cXo1bUxFZ0pSMldGWm9nTUlfeTA2UQ"}}

验证service token

运行命令

curl -ik -H "Authorization: Bearer AAEAAWVsYXN0aWMva2liYW5hL215dG9rZW46REtzdWE5cVBSTFdHbW1OS1hrNzA5QQ" https://localhost:9200/_security/_authenticate

返回

HTTP/1.1 200 OK
X-elastic-product: Elasticsearch
content-type: application/json
content-length: 395{"username":"elastic/kibana","roles":[],"full_name":"Service account - elastic/kibana","email":null,"token":{"name":"mytoken","type":"_service_account_index"},"metadata":{"_elastic_service_account":true},"enabled":true,"authentication_realm":{"name":"_service_account","type":"_service_account"},"lookup_realm":{"name":"_service_account","type":"_service_account"},"authentication_type":"token"}

IK分词器下载

• 下载页面
• 下载地址

wget https://release.infinilabs.com/analysis-ik/stable/elasticsearch-analysis-ik-8.15.3.zip
# 把zip包的内容解压到elasticsearch-analysis-ik-8.15.3目录
unzip elasticsearch-analysis-ik-8.15.3.zip -d elasticsearch-analysis-ik-8.15.3
# 将ik文件夹移动到ES安装目录下的plugins文件夹下
mv elasticsearch-analysis-ik-8.15.3 elasticsearch-8.15.3/plugins/

• 重启Elasticsearch