springcoud-kubernetes实践之用kubernetes做注册中心（包括负载均衡熔断）

本文是spring-cloud-kubernetes实战，主要内容是在kubernetes上部署两个应用：provider-service和consumer-service，通过spring-cloud-kubernetes提供的注册发现能力，实现consumer-service调用provider-service提供的http服务；

springcoud-kubernetes实践之用kubernetes做注册中心（包括负载均衡熔断）

前提条件
开发provider-service
开发consumer-service
验证ribbon轮询能力
验证熔断能力
错误处理

前提条件

请确保你已经在服务器上搭建kubernetes集群

开发provider-service

1.创建maven项目，添加如下pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd"><modelVersion>4.0.0</modelVersion><parent><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-parent</artifactId><version>2.3.3.RELEASE</version><relativePath/> <!-- lookup parent from repository --></parent><groupId>com.purcotton</groupId><artifactId>k8sprovider</artifactId><version>0.0.1-SNAPSHOT</version><name>k8sprovider</name><description>Demo project for k8sprovider</description><properties><java.version>1.8</java.version><spring.cloud.k8s.version>1.1.6.RELEASE</spring.cloud.k8s.version></properties><dependencyManagement><dependencies><dependency><groupId>org.springframework.cloud</groupId><artifactId>spring-cloud-kubernetes-dependencies</artifactId><version>${spring.cloud.k8s.version}</version><type>pom</type><scope>import</scope></dependency></dependencies></dependencyManagement><dependencies><dependency><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter</artifactId></dependency><dependency><groupId>org.projectlombok</groupId><artifactId>lombok</artifactId><optional>true</optional></dependency><dependency><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-test</artifactId><scope>test</scope><exclusions><exclusion><groupId>org.junit.vintage</groupId><artifactId>junit-vintage-engine</artifactId></exclusion></exclusions></dependency><dependency><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-web</artifactId></dependency></dependencies><build><plugins><plugin><groupId>org.springframework.boot</groupId><artifactId>spring-boot-maven-plugin</artifactId></plugin></plugins></build></project>

2.application.properties如下：

spring.application.name=k8s-provider-service
server.port=8083

3.启动类ProviderApplication.java如下

@SpringBootApplication
public class ProviderApplication {public static void main(String[] args) {SpringApplication.run(ProviderApplication.class, args);}}

4.对外提供服务的是ProviderController ，方法getName返回了当前容器的hostname，方法health用于响应kubernetes的两个探针，方法ribbonPing用于响应使用了ribbon服务的调用方，它们会调用这个接口来确定当前服务是否正常

@RestController
public class ProviderController {Logger LOG = LoggerFactory.getLogger(ProviderController.class);private final String hostName = System.getenv("HOSTNAME");/*** 探针检查响应类* @return*/@RequestMapping("/health")public String health() {return "OK";}@RequestMapping("/")public String ribbonPing(){LOG.info("ribbonPing of {}", hostName);return hostName;}/*** 返回hostname* @return 当前应用所在容器的hostname.*/@RequestMapping("/name")public String getName() {return this.hostName+ ", "+ new SimpleDateFormat("yyyy-MM-dd HH:mm:ss").format(new Date());}
}

5.Dockerfile文件如下：

FROM $REPOSITORY/jdk:1.0.0
COPY provider-0.0.1-SNAPSHOT.jar $TARGENAME.jar
RUN rm -rf /etc/localtime && ln -s /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
EXPOSE 8080
#ENTRYPOINT ["java","-jar","$TARGENAME.jar"]
ENTRYPOINT [ "sh", "-c", "exec java -jar $TARGENAME.jar"]

打包duocker镜像：docker build -t REPOSITORY/REPOSITORY/REPOSITORY/TARGENAME:1.0.0 .
推送镜像到私库：docker push REPOSITORY/REPOSITORY/REPOSITORY/TARGENAME:1.0.0

6.编写provider-service.yml文件如下：

apiVersion: apps/v1
kind: Deployment
metadata:name: provider-servicelabels:app: provider-service
spec:replicas: 1selector:matchLabels:app: provider-servicetemplate:metadata:labels:app: provider-servicespec:containers:- name: provider-serviceimage: "xxxx/provider-service:1.0.0"imagePullPolicy: Alwaysports:- name: httpcontainerPort: 8083protocol: TCP---apiVersion: v1
kind: Service
metadata:name: provider-service
spec:type: NodePortports:- port: 8083nodePort: 30082selector:app: provider-service

执行kubectl apply -f provider-service.yml
查看pod

kubectl get deployments
NAME                               READY   UP-TO-DATE   AVAILABLE   AGE
provider-service                   1/1     1            1           42m

查看service

kubectl get svc
NAME                       TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGE
provider-service           NodePort    10.105.30.208    <none>        8083:30082/TCP   42m

通过浏览器此时就可以访问服务了 http://{NodeIP}:30082/name

此时，我们的服务提供者就已经开发完成了，现在我们开始开发消费者服务。

开发consumer-service

consumer-service服务是个springboot应用，用到了spring-cloud-kubernetes提供的注册发现能力，以轮询的方式访问指定服务的全部pod
1.创建pom.xml文件

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd"><modelVersion>4.0.0</modelVersion><parent><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-parent</artifactId><version>2.3.3.RELEASE</version><relativePath/> <!-- lookup parent from repository --></parent><groupId>com.purcotton</groupId><artifactId>consumer</artifactId><version>0.0.1-SNAPSHOT</version><name>sconsumer</name><description>Demo project for consumer</description><properties><java.version>1.8</java.version><spring-cloud.version>Hoxton.SR8</spring-cloud.version></properties><dependencies><dependency><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-web</artifactId></dependency><dependency><groupId>org.springframework.cloud</groupId><artifactId>spring-cloud-kubernetes-core</artifactId><version>1.1.6.RELEASE</version></dependency><dependency><groupId>org.springframework.cloud</groupId><artifactId>spring-cloud-kubernetes-discovery</artifactId><version>1.1.6.RELEASE</version></dependency><!-- https://mvnrepository.com/artifact/org.springframework.cloud/spring-cloud-starter-kubernetes-ribbon --><dependency><groupId>org.springframework.cloud</groupId><artifactId>spring-cloud-starter-kubernetes-ribbon</artifactId><version>1.1.6.RELEASE</version></dependency><dependency><groupId>org.springframework.cloud</groupId><artifactId>spring-cloud-commons</artifactId></dependency><dependency><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter</artifactId></dependency><dependency><groupId>org.springframework.cloud</groupId><artifactId>spring-cloud-starter-netflix-ribbon</artifactId></dependency><!-- https://mvnrepository.com/artifact/org.springframework.cloud/spring-cloud-starter-netflix-hystrix --><dependency><groupId>org.springframework.cloud</groupId><artifactId>spring-cloud-starter-netflix-hystrix</artifactId><version>2.2.5.RELEASE</version></dependency></dependencies><dependencyManagement><dependencies><dependency><groupId>org.springframework.cloud</groupId><artifactId>spring-cloud-dependencies</artifactId><version>${spring-cloud.version}</version><type>pom</type><scope>import</scope></dependency></dependencies></dependencyManagement><build><plugins><plugin><groupId>org.springframework.boot</groupId><artifactId>spring-boot-maven-plugin</artifactId></plugin></plugins></build></project>

2.application.properties如下：

spring.cloud.kubernetes.reload.enabled=true
#spring.cloud.kubernetes.reload.strategy=restart_context
#spring.cloud.kubernetes.reload.mode=pollingmanagement.endpoint.health.enabled=true
management.endpoint.info.enabled=true
management.endpoint.restart.enabled=true

3.application.yml如下：

spring:application:name: k8s-consumer-serviceserver:port: 8080backend:ribbon:eureka:enabled: falseclient:enabled: trueServerListRefreshInterval: 5000hystrix.command.BackendCall.execution.isolation.thread.timeoutInMilliseconds: 5000
hystrix.threadpool.BackendCallThread.coreSize: 5

4.ConsumerApplication .java启动类如下：

@SpringBootApplication
@EnableDiscoveryClient
@EnableCircuitBreaker
@RibbonClient(name="provider-service", configuration = RibbonConfiguration.class)
public class ConsumerApplication {public static void main(String[] args) {SpringApplication.run(ConsumerApplication.class, args);}@LoadBalanced@BeanRestTemplate restTemplate(){return new RestTemplate();}
}

5.创建一个ribbon的配置类RibbonConfiguration：

public class RibbonConfiguration {@AutowiredIClientConfig ribbonClientConfig;/*** 检查服务是否可用的实例，* 此地址返回的响应的返回码如果是200表示服务可用* @param config* @return*/@Beanpublic IPing ribbonPing(IClientConfig config){return new PingUrl();}/*** 轮询规则* @param config* @return*/@Beanpublic IRule ribbonRule(IClientConfig config){return new AvailabilityFilteringRule();}
}

6.创建ConsumerService 类如下：

@Service
public class ConsumerService {@Autowiredprivate RestTemplate restTemplate;@HystrixCommand(fallbackMethod = "getFallbackName" ,commandProperties = {@HystrixProperty(name = "execution.isolation.thread.timeoutInMilliseconds", value = "1000") })public String getDataFromSpringCloudK8SProvider(){return this.restTemplate.getForObject("http://k8s-provider-service/name", String.class);}/*** 熔断时调用的方法* @return*/private String getFallbackName() {return "Fallback"+ ", "+ new SimpleDateFormat("yyyy-MM-dd HH:mm:ss").format(new Date());}
}

7.创建外部能访问的controller如下：

@RestController
public class WebServiceController {@Autowiredprivate ConsumerService consumerService ;/*** 探针检查响应类* @return*/@RequestMapping("/health")public String health() {return "OK";}/*** 远程调用account-service提供的服务* @return 多次远程调返回的所有结果.*/@RequestMapping("/account")public String account() {StringBuilder sbud = new StringBuilder();for(int i = 0; i < 10; i++){sbud.append(consumerService.getDataFromSpringCloudK8SProvider()).append("<br>");}return sbud.toString();}
}

8.创建Dockerfile文件如下：

FROM $REPOSITORY/jdk:1.0.0
COPY consumer-0.0.1-SNAPSHOT.jar $TARGENAME.jar
RUN rm -rf /etc/localtime && ln -s /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
EXPOSE 8080
#ENTRYPOINT ["java","-jar","$TARGENAME.jar"]
ENTRYPOINT [ "sh", "-c", "exec java -jar $TARGENAME.jar"]

打包duocker镜像：docker build -t REPOSITORY/REPOSITORY/REPOSITORY/TARGENAME:1.0.0 .
推送镜像到私库：docker push REPOSITORY/REPOSITORY/REPOSITORY/TARGENAME:1.0.0

9.编写consumer-service.yml文件如下：

apiVersion: v1
kind: Service
metadata:name: consumer-servicelabels:app: consumer-service
spec:type: NodePortports:- port: 8080nodePort: 30081protocol: TCPname: httpselector:app: consumer-service---
apiVersion: apps/v1
kind: Deployment
metadata:name: consumer-servicelabels:app: consumer-service
spec:replicas: 1selector:matchLabels:app: consumer-servicetemplate:metadata:labels:app: consumer-servicespec:containers:- name: consumer-serviceimage: "xxxx/consumer-service:1.0.0"imagePullPolicy: Alwaysports:- name: httpcontainerPort: 8080protocol: TCP

执行kubectl apply -f consumer-service.yml
查看pod

kubectl get deployments
NAME                               READY   UP-TO-DATE   AVAILABLE   AGE
consumer-service                   1/1     1            1           42m

查看service

kubectl get svc
NAME                       TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGE
consumer-service           NodePort    10.110.106.105    <none>        8080:30081/TCP   42m

通过浏览器此时就可以访问服务了 http://{NodeIP}:30081/account

可以看到可以访问到prodvider-service了

验证ribbon轮询能力

1.首先将prodvider-service扩容成两个，执行如下命令

kubectl scale --replicas=2 deployment provider-service

2.检查provider-service是不是两个

kubectl get pods
NAME                                                READY   STATUS    RESTARTS   AGE
consumer-service-9d46cbdd8-fct2v                    1/1     Running   0          67m
provider-service-59ff66fd8b-bbm6p                   1/1     Running   0          35m
provider-service-59ff66fd8b-g47hx                   1/1     Running   0          5s

3.再次用浏览器访问 http://{NodeIP}:30081/account

可以看到已经轮询多个服务了，说明此时验证成功

验证熔断能力

1.我们运行如下命令，停掉服务

kubectl delete -f provider-service.yaml

2.再次进入浏览器访问 http://{NodeIP}:30081/account

我们已经看到访问了我们的熔断方法返回的内容，证明配置成功。
3.最后我们再次启动provider-service
执行 kubectl apply -f provider-service.yml
返回浏览器可以看到浏览器如下内容：

此时，spring-cloud-kubernetes的服务发现（轮询及熔断)就全部完成了，利用spring-cloud-kubernetes将原生的kubernetes服务带给了SpringCloud应用，帮助传统微服务更好的融合在kubernetes环境中。

错误处理

如果你遇到了如下错误

Message: Forbidden!Configured service account doesn't have access. Service account may have been revoked. services is forbidden: User "system:serviceaccount:default:default" cannot list resource "services" in API group "" in the namespace "default".

这表示了当前执行用户没有访问API得权限，具体可看

https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/

https://kubernetes.io/docs/reference/access-authn-authz/rbac/

解决：
新建一个endpoints-cluster-role.yml 如下：

kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:namespace: defaultname: endpoints-reader
rules:
- apiGroups: [""]resources: ["endpoints"]verbs: ["get", "watch", "list"]

执行kubectl apply -f endpoints-cluster-role.yml
添加serviceacount的binding如下

kubectl create clusterrolebinding endpoints-reader-mydefault \--clusterrole=endpoints-reader  \--serviceaccount=default:default

如上面还不能解决，也可以使用以下管理员权限（不推荐）

kubectl create clusterrolebinding permissive-binding \--clusterrole=cluster-admin \--user=admin \--user=kubelet \--group=system:serviceaccounts