【实战】K8S集群部署nacos并接入Springcloud项目容器化运维

前言

相信很多同学都开发过以微服务为架构的系统，开发微服务必不可少要使用注册中心，比如nacos\consul等等。当然在自动化运维流行的今天，我们也会将注册中心部署在k8s集群中。今天我们就分享一期使用helm部署nacos到k8s集群并接入soringcloud项目，各位大大敬请鉴赏。值得注意的是使用helm在k8s部署中间件应用，真的比原始部署简单太多。

Nacos集群搭建

首先我们先从官网把nacos helm相关的代码down下来

git clone https://github.com/nacos-group/nacos-k8s.git

然后进入到helm目录，修改values.yaml文件，下面是我的:

# Default values for nacos.
# This is a YAML-formatted file.
# Declare variables to be passed into your templates.global:#mode: standalonemode: cluster############################nacos###########################
namespace: nacos-cluster  ## 命名空间
nacos:image:repository: nacos/nacos-servertag: latestpullPolicy: IfNotPresentplugin:enable: trueimage:repository: nacos/nacos-peer-finder-plugintag: 1.1pullPolicy: IfNotPresentreplicaCount: 3 ## 节点数podManagementPolicy: ParalleldomainName: cluster.localpreferhostmode: hostnameserverPort: 8848health:enabled: truestorage:#type: embedded  type: mysql  ## 用mysql保存配置信息db:host: mysql-cluster-primary.mysql-cluster.svc.cluster.localname: nacosport: 3306username: rootpassword: 123456rootparam: characterEncoding=utf8&connectTimeout=1000&socketTimeout=3000&autoReconnect=true&useSSL=false&allowPublicKeyRetrieval=trueservice:#type: ClusterIPtype: NodePortport: 8848nodePort: 30000############################nacos###########################

接着我们来安装nacos，如下所示：
#数据库增加nacos库
#创建命名空间

[root@master helm]# kubectl create ns nacos-cluster

namespace/nacos-cluster created
#安装nacos

[root@master helm]# helm install nacos-cluster -n nacos-cluster .

NAME: nacos-cluster
LAST DEPLOYED: Tue Sep 5 10:16:46 2023
NAMESPACE: nacos-cluster
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:

1. Get the application URL by running these commands:
   export NODE_PORT=$(kubectlget--namespacenacos-cluster-ojsonpath=".spec.ports[0].nodePort"servicesnacos-cs)exportNOD{E}_{I}P=$(kubectl get nodes --namespace nacos-cluster -o jsonpath=“{.items[0].status.addresses[0].address}”)
   echo http://$NOD{E}_{I}P:$NODE_PORT/nacos
2. MODE:
   standalone: you need to modify replicaCount in the values.yaml, .Values.replicaCount=1
   cluster: kubectl scale sts nacos-cluster-nacos --replicas=3

#重新部署 的命令是 helm upgrade nacos-cluster -n nacos-cluster .

安装好了之后，我们查看控制台显示是不是安装好了，如下所示：

[root@master helm]# kubectl get pods,svc -n nacos-cluster

NAME READY STATUS RESTARTS AGE
pod/nacos-cluster-0 0/1 Running 0 67s
pod/nacos-cluster-1 0/1 Running 0 67s
pod/nacos-cluster-2 0/1 Running 0 67s

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/nacos-cs NodePort 10.98.1.221 8848:30040/TCP,9848:31784/TCP,9849:30165/TCP,7848:30000/TCP 67s
service/nacos-hs ClusterIP None 8848/TCP,9848/TCP,9849/TCP,7848/TCP

很显然安装成功了！然后我们获取一下ip和port

[root@node2 ~]#   export NODE_PORT=$(kubectl get --namespace nacos-cluster -o jsonpath="{.spec.ports[0].nodePort}" services  nacos-cs)export NODE_IP=$(kubectl get nodes --namespace nacos-cluster -o jsonpath="{.items[0].status.addresses[0].address}")echo http://$NODE_IP:$NODE_PORT/nacos[root@node2 ~]#   export NODE_IP=$(kubectl get nodes --namespace nacos-cluster -o jsonpath="{.items[0].status.addresses[0].address}")

http://10.10.22.91:31673/nacos

这里说明一下，因为nacos ui是通过http协议访问的，而且8848端口是提供web访问的，所以这里的8848对外的端口就是31673，然后我们在浏览器中输入：xxx.xxx.xxx.xxx:31673就可以访问了，如下所示：

Spring cloud配置nacos

nacos安装成功之后，我们可以在springcloud中配置并使用，下面是我的配置步骤：
1、引入依赖包

<modelVersion>4.0.0</modelVersion>
<parent><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-parent</artifactId><version>2.3.12.RELEASE</version><relativePath/> <!-- lookup parent from repository -->
</parent>
<groupId>com.example</groupId>
<artifactId>k8s-demo</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>k8s-demo</name>
<description>Demo project for Spring Boot</description>
<properties><java.version>8</java.version><spring-cloud.version>Hoxton.SR10</spring-cloud.version>
</properties><!-- nacos start -->
<dependency><groupId>com.alibaba.cloud</groupId><artifactId>spring-cloud-starter-alibaba-nacos-config</artifactId><version>2.2.3.RELEASE</version>
</dependency>
<dependency><groupId>com.alibaba.cloud</groupId><artifactId>spring-cloud-starter-alibaba-nacos-discovery</artifactId><version>2.2.3.RELEASE</version>
</dependency>
<!-- nacos end --><dependencyManagement><dependencies><dependency><groupId>org.springframework.cloud</groupId><artifactId>spring-cloud-dependencies</artifactId><version>${spring-cloud.version}</version><type>pom</type><scope>import</scope></dependency></dependencies>
</dependencyManagement>

2、配置启动类

/*** K8sDemoApplication* @return a* @author senfel* @date 2023/9/5 15:02*/
@SpringBootApplication
@EnableDiscoveryClient
public class K8sDemoApplication {public static void main(String[] args) {SpringApplication.run(K8sDemoApplication.class, args);}}

3、配置bootstrap.yaml
注意，nacos在springcloud中对应的端口是暴露外网端口30040

server:port: 9999
spring:application:name: test-demoprofiles:active: devredis:host: 10.10.22.91port: 30617password: 123456pwtimeout: 5000database: 0jedis:pool:max-active: 1000max-idle: 50min-idle: 4cloud:nacos:config:server-addr: 10.10.22.82:30040,10.10.22.91:30040,10.10.22.199:30040username: nacospassword: nacosfile-extension: yamldiscovery:server-addr: 10.10.22.82:30040,10.10.22.91:30040,10.10.22.199:30040username: nacospassword: nacos

设置完成启动正常。

注意：如果新增配置报参数异常
1.删掉config_info 和 his_config_info 表中的encrypted_data_key字段
2.config_info 和 his_config_info 表中非空encrypted_data_key字段设置为可以为空

将Springcloud项目部署在k8s

在上述配置的基础上
在k8s集群中用域名进行配置

server:port: 9999
spring:application:name: test-demoprofiles:active: devredis:host: 10.10.22.91port: 30617password: 123456pwtimeout: 5000database: 0jedis:pool:max-active: 1000max-idle: 50min-idle: 4cloud:nacos:config:server-addr: nacos-cs.nacos-cluster.svc.cluster.local:8848username: nacospassword: nacosfile-extension: yamldiscovery:server-addr: nacos-cs.nacos-cluster.svc.cluster.local:8848username: nacospassword: nacos

注意：headless 服务没有负载均衡，一般集群我们都访问普通svc

增加配置：
pom.xml

<dependency><groupId>org.springframework.cloud</groupId><artifactId>spring-cloud-starter-netflix-hystrix</artifactId>
</dependency>
<dependency><groupId>org.springframework.cloud</groupId><artifactId>spring-cloud-starter-openfeign</artifactId>
</dependency>

启动类开启feign client

 @EnableFeignClients

增加测试feign代码
DemoService

/*** test feign* @author senfel* @version 1.0* @date 2023/9/5 16:18*/
@Service
@FeignClient(value = "k8s-demo",fallback = FailDemoService.class)
public interface DemoService {/*** test feign*/@GetMapping("/feign")String feign(@RequestParam String str);
}

FailDemoService

/*** FailDemoService* @author senfel* @version 1.0* @date 2023/9/5 16:20*/
@Slf4j
@Service
public class FailDemoService  implements DemoService {@Overridepublic String feign(String str) {log.error("调用feign传入参数{}，已降级。",str);return null;}
}

AppController

/*** AppController {* @author senfel* @version 1.0* @date 2023/9/5 16:25*/
@Slf4j
@RestController
public class AppController {@Value("${app.name}")private String appName;@Resourceprivate DemoService demoService;/*** getRedisValueByKey* @param key* @author senfel* @date 2023/8/31 16:22* @return java.lang.String*/@GetMapping("/getRedisValueByKey")public String getRedisValueByKey(String key){try{if(null == key){return null;}String str = RedisUtil.getString(key);log.error("获取redis中key:{}的数据为：{}",key,str);log.error("测试nacos配置，app.name:{}",appName);str = "redis-"+key+"："+str+",nacos-appName:"+appName;//调用feignString feign = demoService.feign(str);return feign;}catch (Exception e){e.printStackTrace();return e.getMessage();}}/*** feign* @param str* @author senfel* @date 2023/9/5 16:26* @return java.lang.String*/@GetMapping("feign")public String feign(@RequestParam String str){try{InetAddress address = InetAddress.getLocalHost();System.out.println(address.getHostName());//主机名System.out.println(address.getCanonicalHostName());//主机别名System.out.println(address.getHostAddress());//获取IP地址}catch (Exception e){e.printStackTrace();}return str+"-feign-success";}
}

制作app镜像
Dockerfile

# version 1.0
# 基础镜像
FROM openjdk:8-jre
# 维护人
MAINTAINER senfel<[e-mail]>
# 创建目录
RUN mkdir -p /home/app
# 拷贝项目jar
COPY ./target/k8s-demo-0.0.1-SNAPSHOT.jar /home/app/app.jar
# 执行命令启动jar
ENTRYPOINT ["java","-jar","/home/app/app.jar"]
# 暴露端口
EXPOSE 9999

将本地jar打包成镜像并提交到云仓库

docker build -t k8s-demo-nacos:v1.0.0 .

编写kube执行文件
kube-k8s-demo-nacos.yaml

apiVersion: apps/v1
kind: Deployment # 声明一个Deployment资源对象
metadata:name: deployment-myapp
spec:replicas: 3 # 通过replicas声明pod个数是3selector:   # 通过标签选择被控制的podmatchLabels:app: myapptemplate:   # 在template中定义podmetadata:labels: # 给pod打上标签app=myappapp: myappspec:containers:- name: myapp  # 声明容器名称，注意不是pod名称，pod名称应该定义在metadata中image: registry.cn-hangzhou.aliyuncs.com/senfel/k8s-demo-nacos:v1.0.0ports:- name: httpcontainerPort: 9999
--- # 在一个yaml文件中通过---分割多个资源对象
apiVersion: v1
kind: Service # 声明一个Service资源对象
metadata:name: service-myapp
spec:type: NodePortselector: # service-myapp将选择标签包含app=myapp的podapp: myappports:- name: httpport: 9999  # Service监听端口targetPort: 9999 # 转发到后端Pod的端口号protocol: TCP # tcp协议nodePort: 30999 # 外网访问端口，范围：30000-32767

执行kube

kubectl apply -f kube-k8s-demo-nacos.yaml

查看执行结果

[root@master k8s]# kubectl get pods,svc | grep app

pod/deployment-myapp-9c6fb8dd9-jvmwl 1/1 Running 0 14m
pod/deployment-myapp-9c6fb8dd9-z64zj 1/1 Running 0 14m
pod/deployment-myapp-9c6fb8dd9-zklpc 1/1 Running 0 14m
service/service-myapp NodePort 10.102.62.181 9999:30999/TCP 14m
#删除pod svc

kubectl delete -f kube-k8s-demo-nacos.yaml

查看nacos情况
配置列表：

服务集群：

测试app集群是否正常服务
posman测试

写在最后

在k8s中部署nacos还是使用helm比较简单，直接下载安装包修改配置文件安装即可。我们在Springcloud、Springboot项目中集成nacos需要注意需要使用集群内部地址，当我们将项目部署在k8s集群中可以直接使用nacos配置和查询服务集群。