kubernetes实战-配置中心（四）分环境使用apollo配置中心

小程序：扫一扫查出行
【扫一扫了解最新限行尾号】
复制小程序

要进行分环境，需要将现有实验环境进行拆分

portal服务，可以各个环境共用，但是apollo-adminservice和apollo-configservice必须要分开。

1、zk环境拆分为test和prod环境

添加dns解析：

# vi /var/named/od.com.zone

zk-test.od.com A 10.4.7.11

zk-prod.od.com A 10.4.7.12

2、namespace 分环境，创建test 和prod

# kubectl create ns test
# kubectl create ns prod

创建secret：

# kubectl create secret docker-registry harbor --docker-server=harbor.od.com --docker-username=harbor --docker-password=Harbor12345 -n test# kubectl create secret docker-registry harbor --docker-server=harbor.od.com --docker-username=harbor --docker-password=Harbor12345 -n prod

3、数据库进行拆分，因实验资源有限，故使用分库的形式模拟分环境

修改数据库初始化脚本，分别创建ApolloConfigTestDB和ApolloConfigProdDB

修改数据库中eureka的地址，这里用到了两个新的域名，自行在bind9中添加解析

> update ApolloConfigProdDB.ServerConfig set ServerConfig.Value="http://config-prod.od.com/eureka" where ServerConfig.Key="eureka.service.url";> grant INSERT,DELETE,UPDATE,SELECT on ApolloConfigProdDB.* to "apolloconfig"@"10.4.7.%" identified by "123456";> update ApolloConfigTestDB.ServerConfig set ServerConfig.Value="http://config-test.od.com/eureka" where ServerConfig.Key="eureka.service.url";> grant INSERT,DELETE,UPDATE,SELECT on ApolloConfigTestDB.* to "apolloconfig"@"10.4.7.%" identified by "123456";

修改portal数据，支持fat和pro环境：

> update ServerConfig set Value='fat,pro' where Id=1;

修改portal的cm资源配置清单：

# vi /data/k8s-yaml/apollo-portal/cm.yaml

# kubectl apply -f http://k8s-yaml.od.com/apollo-portal/cm.yaml

分别创建修改两个环境的资源配置文件：

test：

# cd /data/k8s-yaml

# mkdir -p test/{apollo-adminservice,apollo-configservice,dubbo-demo-server,dubbo-demo-consumer}# mkdir -p prod/{apollo-adminservice,apollo-configservice,dubbo-demo-server,dubbo-demo-consumer}

将之前的资源配置清单cp到对应环境的目录中，进行修改：

# cd test/apollo-configservice/

# cp ../../apollo-configservice/* ./

1、cm.yaml 修改ns，数据库库名，eureka地址

apiVersion: v1
kind: ConfigMap
metadata:
name: apollo-configservice-cm
namespace: test
data:
application-github.properties: |
# DataSource
spring.datasource.url = jdbc:mysql://mysql.od.com:3306/ApolloConfigTestDB?characterEncoding=utf8
spring.datasource.username = apolloconfig
spring.datasource.password = 123456
eureka.service.url = http://config-test.od.com/eureka
app.properties: |
appId=100003171

2、dp.yaml

kind: Deployment
apiVersion: extensions/v1beta1
metadata:
name: apollo-configservice
namespace: test
labels:
name: apollo-configservice
spec:
replicas: 1
selector:
matchLabels:
name: apollo-configservice
template:
metadata:
labels:
app: apollo-configservice
name: apollo-configservice
spec:
volumes:
- name: configmap-volume
configMap:
name: apollo-configservice-cm
containers:
- name: apollo-configservice
image: harbor.od.com/infra/apollo-configservice:v1.5.1
ports:
- containerPort: 8080
protocol: TCP
volumeMounts:
- name: configmap-volume
mountPath: /apollo-configservice/config
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
imagePullPolicy: IfNotPresent
imagePullSecrets:
- name: harbor
restartPolicy: Always
terminationGracePeriodSeconds: 30
securityContext:
runAsUser: 0
schedulerName: default-scheduler
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 1
maxSurge: 1
revisionHistoryLimit: 7
progressDeadlineSeconds: 600

3、svc.yaml

kind: Service
apiVersion: v1
metadata:
name: apollo-configservice
namespace: test
spec:
ports:
- protocol: TCP
port: 8080
targetPort: 8080
selector:
app: apollo-configservice

4、ingress.yaml

kind: Ingress
apiVersion: extensions/v1beta1
metadata:
name: apollo-configservice
namespace: test
spec:
rules:
- host: config-test.od.com
http:
paths:
- path: /
backend:
serviceName: apollo-configservice
servicePort: 8080

服务已经注册进来了

接下来部署apollo-adminservice

修改apollo-adminservice的资源配置清单：

# cd /data/k8s-yaml/test/apollo-adminservice

# cp ../../apollo-adminservice/* ./

1、cm.yaml

apiVersion: v1
kind: ConfigMap
metadata:
name: apollo-adminservice-cm
namespace: test
data:
application-github.properties: |
# DataSource
spring.datasource.url = jdbc:mysql://mysql.od.com:3306/ApolloConfigTestDB?characterEncoding=utf8
spring.datasource.username = apolloconfig
spring.datasource.password = 123456
eureka.service.url = http://config-test.od.com/eureka
app.properties: |
appId=100003172

2、dp.yaml

kind: Deployment
apiVersion: extensions/v1beta1
metadata:
name: apollo-adminservice
namespace: test
labels:
name: apollo-adminservice
spec:
replicas: 1
selector:
matchLabels:
name: apollo-adminservice
template:
metadata:
labels:
name: apollo-adminservice
spec:
volumes:
- name: configmap-volume
configMap:
name: apollo-adminservice-cm
containers:
- name: apollo-adminservice
image: harbor.od.com/infra/apollo-adminservice:v1.5.1
ports:
- containerPort: 8080
protocol: TCP
volumeMounts:
- name: configmap-volume
mountPath: /apollo-adminservice/config
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
imagePullPolicy: IfNotPresent
imagePullSecrets:
- name: harbor
restartPolicy: Always
terminationGracePeriodSeconds: 30
securityContext:
runAsUser: 0
schedulerName: default-scheduler
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 1
maxSurge: 1
revisionHistoryLimit: 7
progressDeadlineSeconds: 600

应用资源配置清单：

# kubectl apply -f http://k8s-yaml.od.com/test/apollo-adminservice/cm.yaml
# kubectl apply -f http://k8s-yaml.od.com/test/apollo-adminservice/dp.yaml

接下来部署prod环境的apollo-configservice，还是一样的套路：

# cd ../../prod/apollo-configservice/
# cp ../../apollo-configservice/* ./

修改资源配置清单：

1、cm.yaml

apiVersion: v1
kind: ConfigMap
metadata:
name: apollo-configservice-cm
namespace: prod
data:
application-github.properties: |
# DataSource
spring.datasource.url = jdbc:mysql://mysql.od.com:3306/ApolloConfigProdDB?characterEncoding=utf8
spring.datasource.username = apolloconfig
spring.datasource.password = 123456
eureka.service.url = http://config-prod.od.com/eureka
app.properties: |
appId=100003171

2、dp.yaml

kind: Deployment
apiVersion: extensions/v1beta1
metadata:
name: apollo-configservice
namespace: prod
labels:
name: apollo-configservice
spec:
replicas: 1
selector:
matchLabels:
name: apollo-configservice
template:
metadata:
labels:
app: apollo-configservice
name: apollo-configservice
spec:
volumes:
- name: configmap-volume
configMap:
name: apollo-configservice-cm
containers:
- name: apollo-configservice
image: harbor.od.com/infra/apollo-configservice:v1.5.1
ports:
- containerPort: 8080
protocol: TCP
volumeMounts:
- name: configmap-volume
mountPath: /apollo-configservice/config
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
imagePullPolicy: IfNotPresent
imagePullSecrets:
- name: harbor
restartPolicy: Always
terminationGracePeriodSeconds: 30
securityContext:
runAsUser: 0
schedulerName: default-scheduler
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 1
maxSurge: 1
revisionHistoryLimit: 7
progressDeadlineSeconds: 600

3、svc.yaml

kind: Service
apiVersion: v1
metadata:
name: apollo-configservice
namespace: prod
spec:
ports:
- protocol: TCP
port: 8080
targetPort: 8080
selector:
app: apollo-configservice

4、ingress.yaml

kind: Ingress
apiVersion: extensions/v1beta1
metadata:
name: apollo-configservice
namespace: prod
spec:
rules:
- host: config-prod.od.com
http:
paths:
- path: /
backend:
serviceName: apollo-configservice
servicePort: 8080

应用资源配置清单：

# kubectl apply -f http://k8s-yaml.od.com/prod/apollo-configservice/cm.yaml
# kubectl apply -f http://k8s-yaml.od.com/prod/apollo-configservice/dp.yaml
# kubectl apply -f http://k8s-yaml.od.com/prod/apollo-configservice/svc.yaml
# kubectl apply -f http://k8s-yaml.od.com/prod/apollo-configservice/ingress.yaml

修改apollo-adminservice的资源配置清单：

# cd ../apollo-adminservice/
# cp ../../apollo-adminservice/* ./

1、cm.yaml

apiVersion: v1
kind: ConfigMap
metadata:
name: apollo-adminservice-cm
namespace: prod
data:
application-github.properties: |
# DataSource
spring.datasource.url = jdbc:mysql://mysql.od.com:3306/ApolloConfigProdDB?characterEncoding=utf8
spring.datasource.username = apolloconfig
spring.datasource.password = 123456
eureka.service.url = http://config-prod.od.com/eureka
app.properties: |
appId=100003172

2、dp.yaml

kind: Deployment
apiVersion: extensions/v1beta1
metadata:
name: apollo-adminservice
namespace: prod
labels:
name: apollo-adminservice
spec:
replicas: 1
selector:
matchLabels:
name: apollo-adminservice
template:
metadata:
labels:
name: apollo-adminservice
spec:
volumes:
- name: configmap-volume
configMap:
name: apollo-adminservice-cm
containers:
- name: apollo-adminservice
image: harbor.od.com/infra/apollo-adminservice:v1.5.1
ports:
- containerPort: 8080
protocol: TCP
volumeMounts:
- name: configmap-volume
mountPath: /apollo-adminservice/config
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
imagePullPolicy: IfNotPresent
imagePullSecrets:
- name: harbor
restartPolicy: Always
terminationGracePeriodSeconds: 30
securityContext:
runAsUser: 0
schedulerName: default-scheduler
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 1
maxSurge: 1
revisionHistoryLimit: 7
progressDeadlineSeconds: 600

应用资源配置清单：

# kubectl apply -f http://k8s-yaml.od.com/prod/apollo-adminservice/cm.yaml
# kubectl apply -f http://k8s-yaml.od.com/prod/apollo-adminservice/dp.yaml

两个服务都已经注册进来了，删除portal数据库中存储的关于之前项目的配置，接下来启动portal项目：

# mysql -uroot -p

> use ApolloPortalDB ;

> truncate table App;
> truncate table AppNamespace;

打开portal.od.com验证，并且创建两个项目：

首先创建dubbo-demo-service

添加配置：两个环境都添加上：注意连接地址一个是test.od.com,一个是prod.od.com

接下来创建dubbo-demo-web项目：同样是两个环境都发布，注意一个是test.od.com,一个是prod.od.com

接下来交付dubbo服务分环境交付：

同样操作，修改之前项目的资源配置清单：

# cd /data/k8s-yaml/test/dubbo-demo-server

# cp ../../dubbo-server/* ./

1、dp.yaml

kind: Deployment
apiVersion: extensions/v1beta1
metadata:
name: dubbo-demo-service
namespace: test
labels:
name: dubbo-demo-service
spec:
replicas: 1
selector:
matchLabels:
name: dubbo-demo-service
template:
metadata:
labels:
app: dubbo-demo-service
name: dubbo-demo-service
spec:
containers:
- name: dubbo-demo-service
image: harbor.od.com/app/dubbo-demo-service:apollo_191211_1916
ports:
- containerPort: 20880
protocol: TCP
env:
- name: C_OPTS
value: -Denv=fat -Dapollo.meta=http://config-test.od.com
- name: JAR_BALL
value: dubbo-server.jar
imagePullPolicy: IfNotPresent
imagePullSecrets:
- name: harbor
restartPolicy: Always
terminationGracePeriodSeconds: 30
securityContext:
runAsUser: 0
schedulerName: default-scheduler
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 1
maxSurge: 1
revisionHistoryLimit: 7
progressDeadlineSeconds: 600

应用资源配置清单：

# kubectl apply -f http://k8s-yaml.od.com/test/dubbo-demo-server/dp.yaml

接下来交付dubbo-demo-consumer:

# cd /data/k8s-yaml/test/dubbo-demo-consumer

# cp ../../dubbo-consumer/* ./

1、dp.yaml

kind: Deployment
apiVersion: extensions/v1beta1
metadata:
name: dubbo-demo-consumer
namespace: test
labels:
name: dubbo-demo-consumer
spec:
replicas: 1
selector:
matchLabels:
name: dubbo-demo-consumer
template:
metadata:
labels:
app: dubbo-demo-consumer
name: dubbo-demo-consumer
spec:
containers:
- name: dubbo-demo-consumer
image: harbor.od.com/app/dubbo-demo-web:apollo_191212_1715
ports:
- containerPort: 8080
protocol: TCP
- containerPort: 20880
protocol: TCP
env:
- name: C_OPTS
value: -Denv=fat -Dapollo.meta=http://config-test.od.com
- name: JAR_BALL
value: dubbo-client.jar
imagePullPolicy: IfNotPresent
imagePullSecrets:
- name: harbor
restartPolicy: Always
terminationGracePeriodSeconds: 30
securityContext:
runAsUser: 0
schedulerName: default-scheduler
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 1
maxSurge: 1
revisionHistoryLimit: 7
progressDeadlineSeconds: 600

2、svc.yaml

kind: Service
apiVersion: v1
metadata:
name: dubbo-demo-consumer
namespace: test
spec:
ports:
- protocol: TCP
port: 8080
targetPort: 8080
selector:
app: dubbo-demo-consumer

3、ingress.yaml

kind: Ingress
apiVersion: extensions/v1beta1
metadata:
name: dubbo-demo-consumer
namespace: test
spec:
rules:
- host: demo-test.od.com
http:
paths:
- path: /
backend:
serviceName: dubbo-demo-consumer
servicePort: 8080

这里会用到两个新的域名，添加解析：

应用test环境的dubbo-consumer资源配置清单：

# kubectl apply -f http://k8s-yaml.od.com/test/dubbo-demo-consumer/dp.yaml
# kubectl apply -f http://k8s-yaml.od.com/test/dubbo-demo-consumer/svc.yaml
# kubectl apply -f http://k8s-yaml.od.com/test/dubbo-demo-consumer/ingress.yaml

接下来交付prod环境的dubbo-demo-server和dubbo-demo-consumer服务：

同样套路，复制资源配置清单，然后修改成prod环境：

cd /data/k8s-yaml/prod/dubbo-demo-server

# cp ../../dubbo-server/* ./

1、dp.yaml

kind: Deployment
apiVersion: extensions/v1beta1
metadata:
name: dubbo-demo-service
namespace: prod
labels:
name: dubbo-demo-service
spec:
replicas: 1
selector:
matchLabels:
name: dubbo-demo-service
template:
metadata:
labels:
app: dubbo-demo-service
name: dubbo-demo-service
spec:
containers:
- name: dubbo-demo-service
image: harbor.od.com/app/dubbo-demo-service:apollo_191211_1916
ports:
- containerPort: 20880
protocol: TCP
env:
- name: C_OPTS
value: -Denv=pro -Dapollo.meta=http://config-prod.od.com
- name: JAR_BALL
value: dubbo-server.jar
imagePullPolicy: IfNotPresent
imagePullSecrets:
- name: harbor
restartPolicy: Always
terminationGracePeriodSeconds: 30
securityContext:
runAsUser: 0
schedulerName: default-scheduler
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 1
maxSurge: 1
revisionHistoryLimit: 7
progressDeadlineSeconds: 600

应用资源配置清单：

# kubectl apply -f http://k8s-yaml.od.com/prod/dubbo-demo-server/dp.yaml

接下来做dubbo-demo-consumer的资源配置清单：

# cd /data/k8s-yaml/prod/dubbo-demo-consumer

# cp ../../dubbo-consumer/* ./

1、dp.yaml http://apollo-configservice:8080 这里可以直接使用svc资源名称调用，不用走ingress，因为svc资源只在当前namespace中生效。

kind: Deployment
apiVersion: extensions/v1beta1
metadata:
name: dubbo-demo-consumer
namespace: prod
labels:
name: dubbo-demo-consumer
spec:
replicas: 1
selector:
matchLabels:
name: dubbo-demo-consumer
template:
metadata:
labels:
app: dubbo-demo-consumer
name: dubbo-demo-consumer
spec:
containers:
- name: dubbo-demo-consumer
image: harbor.od.com/app/dubbo-demo-web:apollo_191212_1715
ports:
- containerPort: 8080
protocol: TCP
- containerPort: 20880
protocol: TCP
env:
- name: C_OPTS
value: -Denv=pro -Dapollo.meta=http://apollo-configservice:8080
- name: JAR_BALL
value: dubbo-client.jar
imagePullPolicy: IfNotPresent
imagePullSecrets:
- name: harbor
restartPolicy: Always
terminationGracePeriodSeconds: 30
securityContext:
runAsUser: 0
schedulerName: default-scheduler
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 1
maxSurge: 1
revisionHistoryLimit: 7
progressDeadlineSeconds: 600

2、svc.yaml

kind: Service
apiVersion: v1
metadata:
name: dubbo-demo-consumer
namespace: prod
spec:
ports:
- protocol: TCP
port: 8080
targetPort: 8080
selector:
app: dubbo-demo-consumer

3、ingress.yaml

kind: Service
apiVersion: v1
metadata:
name: dubbo-demo-consumer
namespace: prod
spec:
ports:
- protocol: TCP
port: 8080
targetPort: 8080
selector:
app: dubbo-demo-consumer
[root@hdss7-200 dubbo-demo-consumer]# cat ingress.yaml
kind: Ingress
apiVersion: extensions/v1beta1
metadata:
name: dubbo-demo-consumer
namespace: prod
spec:
rules:
- host: demo-prod.od.com
http:
paths:
- path: /
backend:
serviceName: dubbo-demo-consumer
servicePort: 8080

应用资源配置清单：

# kubectl apply -f http://k8s-yaml.od.com/prod/dubbo-demo-consumer/dp.yaml
# kubectl apply -f http://k8s-yaml.od.com/prod/dubbo-demo-consumer/svc.yaml
# kubectl apply -f http://k8s-yaml.od.com/prod/dubbo-demo-consumer/ingress.yaml