容器版Jenkins连接Kubernetes---------非容器版jenkins是无法安装kubenetes插件的，所以无法连接k8s

容器版Jenkins连接Kubernetes

特别注意：必须用谷歌浏览器，而且非容器版jenkins是无法安装kubernetes插件的，所以无法连接k8s

一、环境说明

OS系统版本:Ubuntu 18.04 TLS

软件版本:Jenkins 2.121.2

Kubernetes plugin 1.10.2

Kubernetes集群 1.10.4

二、添加kubernetes plugin

系统管理->管理插件->可选插件,搜索kubernetes plugin并选择安装

三、配置kubernetes plugin连接kubernetes集群

1、点击系统管理->系统设置-添加一个云,在下拉菜单中选择kubernets并添加，如下图所示：



2.、写云kubernetes配置内容

注：Name值任意添加，Kubernetes URL值添加K8S apiserver连接地址和端口，jenkins URL值添加jenkins UI访问地址和端口,如下图所示：



3、在添加云kubernetes中添加pod template并配置


参照：https://www.cnblogs.com/effortsing/p/10486960.html     这里面有怎么添加pod template

4、配置云kubernetes连接K8S集群的验证文件

在jenkins所在的node节点上操作

创建目录

1）mkdir -p /opt/crt/

2）获取K8S的/root/.kube/config文件

3）获取/root/.kube/config中certificate-authority-data的内容并转化成base64 encoded文件

[root@test2 ~]# echo LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUR2akNDQXFhZ0F3SUJBZ0lVYTdJRGZlOENZdmc0ZGpBR0cxQS81WHBLZVRNd0RRWUpLb1pJaHZjTkFRRUwKQlFBd1pURUxNQWtHQTFVRUJoTUNRMDR4RURBT0JnTlZCQWdUQjBKbGFVcHBibWN4RURBT0JnTlZCQWNUQjBKbAphVXBwYm1jeEREQUtCZ05WQkFvVEEyczRjekVQTUEwR0ExVUVDeE1HVTNsemRHVnRNUk13RVFZRFZRUURFd3ByCmRXSmxjbTVsZEdWek1CNFhEVEU0TURjeE9UQTBNVEl3TUZvWERUSXpNRGN4T0RBME1USXdNRm93WlRFTE1Ba0cKQTFVRUJoTUNRMDR4RURBT0JnTlZCQWdUQjBKbGFVcHBibWN4RURBT0JnTlZCQWNUQjBKbGFVcHBibWN4RERBSwpCZ05WQkFvVEEyczRjekVQTUEwR0ExVUVDeE1HVTNsemRHVnRNUk13RVFZRFZRUURFd3ByZFdKbGNtNWxkR1Z6Ck1JSUJJakFOQmdrcWhraUc5dzBCQVFFRkFBT0NBUThBTUlJQkNnS0NBUUVBdTRxK1l2TDZMMjRZcEx3Y0cvMmoKdllFTTY5QmIxMmFzZFBtNXBLekMzc3o5Q2JnZ3llb2ZzZFNxSmFIVkNHRkFmNjBEN3J5elhPMVgxY1RGaGdYdQp5SGtrcnBLdGVyNm1aNXpRVGxrQXlyM1AyOXd0NXhJRHJINkhKOE9FcGZNYWlCRlkrcHhTKzJicEZNVUJ3UTcwCksxVE5tWXFsN0lIOEt5V290UXhKQURtQS9VWUpDWHgwNnB5bVlvRzRmcTVjeUVJK3hBSkRsZ3l5blZpc2Y1cEkKUGo2dEl3SHc1UzVGbGVNUE04WVFMSE5JNUlwUEYvR0txNmdaYldLTHFOamNvL2dVTExXSThsWm5mSlVSbE16VwpLa05XYkxETktwSzZOOG5jc09Yc2lFTjNvMW0zeGg2RjlJUWsyWU95d3hTM1dvWTlUQWp3d0VxdTV0bUtVZ1lhCjN3SURBUUFCbzJZd1pEQU9CZ05WSFE4QkFmOEVCQU1DQVFZd0VnWURWUjBUQVFIL0JBZ3dCZ0VCL3dJQkFqQWQKQmdOVkhRNEVGZ1FVQ1QyM3ora3J5clcvNHJGblR4cFlBalJqa253d0h3WURWUjBqQkJnd0ZvQVVDVDIzeitrcgp5clcvNHJGblR4cFlBalJqa253d0RRWUpLb1pJaHZjTkFRRUxCUUFEZ2dFQkFHNmRWZitPTnlUcVJzRlBVaWRxClVvODJSUSs3RytVOWdxN3VpRjFyamx6cms4d0JxQ0J3SEVwOGFKbzhKa1hBN3JNZUptVk9hcmJPK1lsempmbTcKUlM5QnJNV3I0ZEt5TUw0UTIvWHgreCs5Um1GemxabmZoTFR4SE9XTTJiam9HTjliNk90RnF1VmFqaWtjU0Zkdwp3cTF2cS9Ud05ock41aXlIRko1V3h6YXRpa3BlVmZQdnhsS0xKbmpQOGhxeVh2RWxYcEIydnV1ZStJeEdmUWJiCjROQUFJcVIwOUsvS1FVQ3dQK0lQNjlxK0FMazJIWWRrNVB0SmpOMnpOSEw5SUppWHVQUGViZGtSYVNnMGIvZzMKc3BNeFZTY05jNDR3MGFyQ0ZFU2Foa0tMbHJFRGNjbjcxSVpPb2dEMStaMHgycFB4RDc5OWNlQjlxMUtsNnhOWQp6T2s9Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K | base64 -d > /opt/crt/ca.crt


4）获取/root/.kube/config中client-certificate-data和client-key-data的内容并转化成base64 encoded文件

[root@test2 ~]# echo LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUQzVENDQXNXZ0F3SUJBZ0lVSkw2RzR5YjlsaGwvWlIyT01laENmbVBOK2s4d0RRWUpLb1pJaHZjTkFRRUwKQlFBd1pURUxNQWtHQTFVRUJoTUNRMDR4RURBT0JnTlZCQWdUQjBKbGFVcHBibWN4RURBT0JnTlZCQWNUQjBKbAphVXBwYm1jeEREQUtCZ05WQkFvVEEyczRjekVQTUEwR0ExVUVDeE1HVTNsemRHVnRNUk13RVFZRFZRUURFd3ByCmRXSmxjbTVsZEdWek1CNFhEVEU0TURjeE9UQTJORFF3TUZvWERUSTRNRGN4TmpBMk5EUXdNRm93YXpFTE1Ba0cKQTFVRUJoTUNRMDR4RURBT0JnTlZCQWdUQjBKbGFVcHBibWN4RURBT0JnTlZCQWNUQjBKbGFVcHBibWN4RnpBVgpCZ05WQkFvVERuTjVjM1JsYlRwdFlYTjBaWEp6TVE4d0RRWURWUVFMRXdaVGVYTjBaVzB4RGpBTUJnTlZCQU1UCkJXRmtiV2x1TUlJQklqQU5CZ2txaGtpRzl3MEJBUUVGQUFPQ0FROEFNSUlCQ2dLQ0FRRUF2Ui9GZ2krKzJMdmUKU0ttNU11TG9DeVZ6aUVsOEVCVFhIS0tacWczMG5iOWgvUHAxZEV2a3JFbHFtOFJHN3JOOGFFRkkyZi9mcHF4SwpMdFlTWmVYTjEybWFRQTBiV1JlSG1nYWxnTzBZbVpYV1psc0FOR0xLcEtJOEF4czVXeDB3WlNIdWlxeFlCOXlnCkdPR0dhMW41SmZPUDBRWFZNeTVDZlcweWtLRnZmbDJ6TENwSnRNTzJtdWZxR20zM2wrd0l0SlNKaEJUdUc0RDUKa0pnOFFSNUZzUDk1eXF2RWNnT1NrNUZpNFVKZkcwakY1Rnd3UGt1ekhnTmNFQ29YMVNrTlkrZFJ3aSt5OUNZYQpSdWcvbXBrd1F5RWFBVW5BbzNhaTlWTFd5dnRkc3h1YlFMMUJMUGczZE4xTFRPeUxFaGMzZVJNdWFUR0dhNDc4CmwxUWJYeTV2K3dJREFRQUJvMzh3ZlRBT0JnTlZIUThCQWY4RUJBTUNCYUF3SFFZRFZSMGxCQll3RkFZSUt3WUIKQlFVSEF3RUdDQ3NHQVFVRkJ3TUNNQXdHQTFVZEV3RUIvd1FDTUFBd0hRWURWUjBPQkJZRUZCRFdIRTIxUzhLSApHeFBCT3lNem1LQjZqc1V3TUI4R0ExVWRJd1FZTUJhQUZBazl0OC9wSzhxMXYrS3haMDhhV0FJMFk1SjhNQTBHCkNTcUdTSWIzRFFFQkN3VUFBNElCQVFBRkpaSFpYbmR3ZGJQQVcxSUtMTFUzNlRTUm9pUXVIVk5nWWwzZFRJcVYKMzllTFcxSFJ5djIzdTlOSkhXRkhyZHBWbjNWM0ZTR1lsUW5jRSsxU1o5MUFWZXhGcDMxeWNnbVo0WEdjZ0pVZwphcExoZFJOdHNSVmpHKzF0OFplL2VCdHVXYXFlNXluZUFtT3ZzdmpPcDU5cVpDakxSbDk0YWZ4WVYrVlNiVFByCnAvZHRCcHh3MUlOK3puajh1SXRnRDhQTEpzUXViYWsyeTU2QThWUTJCTmVkSG5ZK3lVaC9NVHgzQ1BkSlB3OTUKbnk3T0ovaWNCV1drbS95Qy9weURqSFhEczdtNVkvWG80VnJ2YUQwd05ucllOWWluTTh0VHRReU1zUEpBaVlLYQpmZDAycEs1eGFzVjUweWdpL3AxaUl4MzEybHMxaWc1eG8zRjAyb2xocWtNKwotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg== | base64 -d > /opt/crt/client.crt

[root@test2 ~]# echo LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFb3dJQkFBS0NBUUVBdlIvRmdpKysyTHZlU0ttNU11TG9DeVZ6aUVsOEVCVFhIS0tacWczMG5iOWgvUHAxCmRFdmtyRWxxbThSRzdyTjhhRUZJMmYvZnBxeEtMdFlTWmVYTjEybWFRQTBiV1JlSG1nYWxnTzBZbVpYV1psc0EKTkdMS3BLSThBeHM1V3gwd1pTSHVpcXhZQjl5Z0dPR0dhMW41SmZPUDBRWFZNeTVDZlcweWtLRnZmbDJ6TENwSgp0TU8ybXVmcUdtMzNsK3dJdEpTSmhCVHVHNEQ1a0pnOFFSNUZzUDk1eXF2RWNnT1NrNUZpNFVKZkcwakY1Rnd3ClBrdXpIZ05jRUNvWDFTa05ZK2RSd2kreTlDWWFSdWcvbXBrd1F5RWFBVW5BbzNhaTlWTFd5dnRkc3h1YlFMMUIKTFBnM2ROMUxUT3lMRWhjM2VSTXVhVEdHYTQ3OGwxUWJYeTV2K3dJREFRQUJBb0lCQUQzbVNqVEQvOGpjSkhMUAo2aWUza0k4bFlOejRnRHliTlpUUHUwK25aYXJEMndSN3pUbVZKWEVtVGxoUk00NG8vTXo2b1NlSTBlQ3hmMDQ1CkRxaC9RSklDcEZQV2RsOEFqb2RoS1lZN0U5UWc4SjFycDNOOTZpbGNXQndFS3craFRCZXR0Vzk1M1E0bHJkaTIKNTlIM0RzN1hHdmtrMlpUNHpSWlVTVHFCUEFhMWY4c2ZuSzFicHNwOEd2OWxZQTZTa1FKNG5XSDIvZXlMVXdZNQpoTXVDZW94TGc3STVIN24vOGpJWGV0Wkk0RW96MkpZa0dJN3FYZVNtdkE5U0JadFVBNGlzSVFSK0lzdVRManpXCmNrOGVrWkpkZU5PeFhrRXpaTC9iSUxwQ1ZZWVdRa2c1R2pCaFdyVGxqWGpWZzltcDJNeENLRlEwT0VGM3FZU0kKa1lZWkFSRUNnWUVBN3VmWnlPZ0o4RFh2NTdUNlFFNmdyM3U2YXdKTXZxcWhJU1l6RnNlRXJ3U0Z0b3hGY2Y1eQpWWHFIOElEUkFDMmRhMjFvV01GcDBrQUN4cjVBK014R2pxUXRTQXZSRjJES09TQk1HUHJiOG9YaktNNytZOHp5CmdueXhaMm5yYjFOdGRqN21Nc1B4UnU0V3FLeWdLRnE3a1YzOFM3Sm5aN0xRQUxiSGJmOUFMSGNDZ1lFQXlxZ00KVlpXYmIyTjZhaGRZdi8vUkVmdXo5Q0c2RGxETmhNNW0xRFlJdjNzMVZEcmhqNHM1MDR2c3JFbHZwVFlLT1o3VQppbUNjSDBZUXpsZnN2UUsxQTNSNFpvNmpFdWNMcUE0S1JWbGlyQk9jODhqSi9hUEdHa1lzQ2NBUzZ0d2pZdy9oCjliQUxVWTJISTFsOVlhYmtSYXdEYk10bi9MWWZFaExMTU5EZjdaMENnWUVBdWw5WXdLaDBDRmFyZndEcU1QeWwKMGdBZDM1ajlzY2greHROOEM0cytjU0tBQlhiTVBpK1hsaU51cFNwNDRVQzBpN2ZnTFUxRmRtWEZSTEhyRWF5YQpabkNoZXBEdFh1VjlISytiYmVsVmFJOFdOU0cxeHJsOWZsbzBNMDZvQWtMOUk3L1I2VXgranl6eHRFaG04TlJICmV4SHMza2lnN243S1VhUkZWQVJLVmVVQ2dZQW9HTjN2NVIwUENnakRpd0VGWkRGU3RKR2pnVFRWOWtqanVROEIKZC90OUgzeXF3TWUyWmg2MzY1eVZiaVpIOHd4TTRFOC9YZVFtRCsvdFU5cEVmNCtmTW1GTU1YYTBtOEJqclB0OQpRelZSeE1PdVBKRXl2VC9LSFE1RGs1eHFtY25xcE03WmxNNTRnVjgyc0ZNdGloN3FaaUY3V2plbCtjYm1CWS9zCmhiZDR4UUtCZ0hsYkNDeFhEOFVVSktIQjNraWVnSHYrOTZlN2oxUHNUTGdjZG9adXp6ejZZK0ZERGdITzk0ZWoKeXdndm12cW1aZmZ0cmdxK1BmOW54WmRGaHc1WmxjbDRtZTltalB6UFc5WFMyVHVsandGZytabDNrNjRESEo3VwpmM2VwOGIvSlpHTmFOWFQ3akF2ZnNhclhYNVQzNUJpS3J4Z0tFOTdvMStFbVhBVENaMXprCi0tLS0tRU5EIFJTQSBQUklWQVRFIEtFWS0tLS0tCg== | base64 -d > /opt/crt/client.key

5）生产Client P12认证文件cert.pfx，并下载至本地windows

[root@test2 ~]# openssl pkcs12 -export -out /opt/crt/cert.pfx -inkey /opt/crt/client.key -in /opt/crt/client.crt -certfile /opt/crt/ca.crt

Enter Export Password: 123

Verifying - Enter Export Password: 123

[root@test2 ~]# sz /opt/crt/cert.pfx

注：自定义一个password并牢记

6）在云kubernetes中添加凭证，如下图所示

添加凭证-》首先密码填写123-》类型-》Certificate-》Upload PKCS#12 certificate-》上传证书-》选择文件-》cert-》打开-》上传-》添加-》选择凭证

注：Upload certificate上次刚生成并下载至本地的cert.pfx文件，Password值添加生成cert.pfx文件时输入的密钥











7）Kubernetes 命名空间中的值添加/root/.kube/config文件中cluster部分中name的内容（否则连接失败）


  

8）测试连接kubernetes集群，如下图所示：








注：Kubernetes Namespace值添加/root/.kube/config文件中cluster部分中name的内容

Connection test successful

5、配置jenkins jnlp代理端口

系统管理->全局安全配置中的"代理"项,指定端口为50000

报错解决：

1、显示证书不对

Error testing connection https://192.168.0.91:6443: Get Key failed: Given final block not properly padded. Such issues can arise if a bad key is used during decryption


解决：是因为添加平局那一界面的下面没有输入之前设置的密码（123）。输入密码、再次导入即可解决

2、显示无法连接

解决： cert.pfx 可能没有生成好 ；或者ca.cert没有生成好，有空格， 重新生成一遍就好