全面掌握容器化应用在Kubernetes集群中的部署策略与最佳实践从环境搭建到服务发布解决企业级容器化转型难题

引言

容器化技术已成为现代云原生应用开发与部署的标准方式，而Kubernetes作为容器编排的事实标准，正在引领企业IT架构的变革。随着微服务架构的普及和DevOps文化的深入，越来越多的企业开始进行容器化转型，但在实际操作过程中往往面临诸多挑战。本文将全面介绍从Kubernetes环境搭建到服务发布的完整流程，分享企业级容器化转型的最佳实践，帮助读者解决转型过程中的难题。

1. Kubernetes环境搭建

1.1 环境选择与规划

在开始Kubernetes部署之前，首先需要根据企业需求选择合适的部署环境：

本地开发环境：适用于开发测试阶段，推荐使用Minikube、Kind或Docker Desktop内置的Kubernetes功能。
私有云环境：企业自建数据中心，可使用kubeadm、Kubespray等工具部署。
公有云环境：AWS EKS、Azure AKS、Google GKE等托管Kubernetes服务。

1.2 使用kubeadm搭建生产级集群

以下是使用kubeadm搭建生产级Kubernetes集群的详细步骤：

1.2.1 系统准备

# 在所有节点上执行 # 关闭防火墙 sudo systemctl stop firewalld sudo systemctl disable firewalld # 禁用SELinux sudo setenforce 0 sudo sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config # 禁用swap sudo swapoff -a sudo sed -i '/ swap / s/^(.*)$/#1/g' /etc/fstab # 设置内核参数 cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf br_netfilter EOF cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf net.bridge.bridge-nf-call-ip6tables = 1 net.bridge.bridge-nf-call-iptables = 1 net.ipv4.ip_forward = 1 EOF sudo sysctl --system # 安装Docker sudo yum install -y yum-utils sudo yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo sudo yum install -y containerd.io docker-ce docker-ce-cli sudo mkdir -p /etc/docker cat <<EOF | sudo tee /etc/docker/daemon.json { "exec-opts": ["native.cgroupdriver=systemd"], "log-driver": "json-file", "log-opts": { "max-size": "100m" }, "storage-driver": "overlay2" } EOF sudo systemctl enable docker sudo systemctl daemon-reload sudo systemctl restart docker # 安装kubeadm, kubelet, kubectl cat <<EOF | sudo tee /etc/yum.repos.d/kubernetes.repo [kubernetes] name=Kubernetes baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-$basearch enabled=1 gpgcheck=1 repo_gpgcheck=1 gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg exclude=kubelet kubeadm kubectl EOF sudo yum install -y kubelet kubeadm kubectl --disableexcludes=kubernetes sudo systemctl enable --now kubelet

1.2.2 初始化控制平面节点

# 在master节点上执行 sudo kubeadm init --pod-network-cidr=10.244.0.0/16 --control-plane-endpoint=LOAD_BALANCER_DNS # 配置kubectl mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config # 安装网络插件（以Calico为例） kubectl apply -f https://docs.projectcalico.org/manifests/calico.yaml

1.2.3 加入工作节点

# 在worker节点上执行 # 使用kubeadm init命令输出的join命令 sudo kubeadm join LOAD_BALANCER_DNS:6443 --token <token> --discovery-token-ca-cert-hash <hash>

1.3 高可用集群配置

对于生产环境，建议配置高可用Kubernetes集群：

# 使用外部负载均衡器（如HAProxy、Nginx或云厂商提供的LB） # 配置示例（HAProxy）: frontend kubernetes-frontend bind *:6443 mode tcp default_backend kubernetes-backend backend kubernetes-backend mode tcp balance roundrobin option tcp-check server master1 192.168.1.11:6443 check server master2 192.168.1.12:6443 check server master3 192.168.1.13:6443 check

1.4 云环境部署

以AWS EKS为例，使用eksctl快速部署集群：

# 安装eksctl curl --silent --location "https://github.com/weaveworks/eksctl/releases/latest/download/eksctl_$(uname -s)_amd64.tar.gz" | tar xz -C /tmp sudo mv /tmp/eksctl /usr/local/bin # 创建集群配置文件 cat > cluster.yaml <<EOF apiVersion: eksctl.io/v1alpha5 kind: ClusterConfig metadata: name: prod-cluster region: us-west-2 version: "1.21" nodeGroups: - name: ng-1 instanceType: m5.large desiredCapacity: 3 minSize: 1 maxSize: 5 volumeSize: 80 ssh: allow: true publicKeyPath: ~/.ssh/id_rsa.pub EOF # 创建集群 eksctl create cluster -f cluster.yaml

2. 容器化应用基础

2.1 Docker容器化最佳实践

在将应用部署到Kubernetes之前，首先需要将应用容器化。以下是Docker容器化的最佳实践：

2.1.1 编写高效的Dockerfile

# 使用官方基础镜像 FROM python:3.9-slim # 设置工作目录 WORKDIR /app # 安装依赖 COPY requirements.txt . RUN pip install --no-cache-dir -r requirements.txt # 复制应用代码 COPY . . # 创建非root用户 RUN useradd --create-home --shell /bin/bash app && chown -R app:app /app USER app # 暴露端口 EXPOSE 8000 # 健康检查 HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 CMD curl -f http://localhost:8000/health || exit 1 # 启动命令 CMD ["gunicorn", "--bind", "0.0.0.0:8000", "app:application"]

2.1.2 多阶段构建优化镜像大小

# 构建阶段 FROM node:16 as builder WORKDIR /app COPY package*.json ./ RUN npm install COPY . . RUN npm run build # 生产阶段 FROM nginx:alpine COPY --from=builder /app/build /usr/share/nginx/html COPY nginx.conf /etc/nginx/conf.d/default.conf EXPOSE 80 CMD ["nginx", "-g", "daemon off;"]

2.2 容器镜像管理

2.2.1 使用私有镜像仓库

# 搭建Harbor私有镜像仓库 # 安装Docker Compose sudo curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose sudo chmod +x /usr/local/bin/docker-compose # 下载Harbor离线安装包 wget https://github.com/goharbor/harbor/releases/download/v2.3.5/harbor-offline-installer-v2.3.5.tgz tar xvf harbor-offline-installer-v2.3.5.tgz cd harbor # 配置Harbor cp harbor.yml.tmpl harbor.yml # 编辑harbor.yml，设置hostname、端口、密码等参数 # 安装Harbor sudo ./install.sh # 配置Docker信任私有仓库 sudo mkdir -p /etc/docker/certs.d/your-harbor-domain sudo cp /path/to/harbor/cert/ca.crt /etc/docker/certs.d/your-harbor-domain/ # 登录并推送镜像 docker login your-harbor-domain docker tag my-app:latest your-harbor-domain/project/my-app:latest docker push your-harbor-domain/project/my-app:latest

2.2.2 镜像安全扫描

使用Trivy进行镜像安全扫描：

# 安装Trivy sudo apt-get install wget apt-transport-https gnupg lsb-release wget -qO - https://aquasecurity.github.io/trivy-repo/deb/public.key | sudo apt-key add - echo deb https://aquasecurity.github.io/trivy-repo/deb $(lsb_release -sc) main | sudo tee -a /etc/apt/sources.list.d/trivy.list sudo apt-get update sudo apt-get install trivy # 扫描镜像 trivy image your-harbor-domain/project/my-app:latest # 在CI/CD中集成Trivy # GitLab CI示例: image_scan: stage: test script: - trivy image --exit-code 1 --severity CRITICAL,HIGH $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA

3. Kubernetes核心概念

3.1 Pod基础

Pod是Kubernetes中最小的部署单元，可以包含一个或多个容器。

3.1.1 Pod定义示例

apiVersion: v1 kind: Pod metadata: name: my-app-pod labels: app: my-app tier: frontend spec: containers: - name: main-container image: your-harbor-domain/project/my-app:latest ports: - containerPort: 8000 env: - name: ENVIRONMENT value: "production" resources: requests: memory: "64Mi" cpu: "250m" limits: memory: "128Mi" cpu: "500m" livenessProbe: httpGet: path: /health port: 8000 initialDelaySeconds: 30 periodSeconds: 10 readinessProbe: httpGet: path: /ready port: 8000 initialDelaySeconds: 5 periodSeconds: 5 - name: sidecar-container image: your-harbor-domain/project/log-collector:latest volumeMounts: - name: log-volume mountPath: /var/log/app volumes: - name: log-volume emptyDir: {}

3.2 Deployment管理应用

Deployment提供了声明式的更新方式，用于管理Pod和ReplicaSet。

3.2.1 Deployment定义示例

apiVersion: apps/v1 kind: Deployment metadata: name: my-app-deployment labels: app: my-app spec: replicas: 3 selector: matchLabels: app: my-app strategy: type: RollingUpdate rollingUpdate: maxUnavailable: 1 maxSurge: 1 template: metadata: labels: app: my-app spec: containers: - name: my-app image: your-harbor-domain/project/my-app:v1.0.0 ports: - containerPort: 8000 env: - name: DATABASE_URL valueFrom: secretKeyRef: name: app-secrets key: database-url resources: requests: memory: "256Mi" cpu: "250m" limits: memory: "512Mi" cpu: "500m"

3.2.2 Deployment更新策略

# 金丝雀发布示例 apiVersion: apps/v1 kind: Deployment metadata: name: my-app-canary spec: replicas: 1 selector: matchLabels: app: my-app version: canary template: metadata: labels: app: my-app version: canary spec: containers: - name: my-app image: your-harbor-domain/project/my-app:v2.0.0 ports: - containerPort: 8000

3.3 Service暴露服务

Service为一组功能相同的Pod提供统一访问入口。

3.3.1 Service类型及示例

# ClusterIP Service（默认类型，仅在集群内部可见） apiVersion: v1 kind: Service metadata: name: my-app-service spec: selector: app: my-app ports: - protocol: TCP port: 80 targetPort: 8000 --- # NodePort Service（通过每个节点的固定端口暴露服务） apiVersion: v1 kind: Service metadata: name: my-app-nodeport-service spec: type: NodePort selector: app: my-app ports: - protocol: TCP port: 80 targetPort: 8000 nodePort: 30007 --- # LoadBalancer Service（使用云提供商的负载均衡器） apiVersion: v1 kind: Service metadata: name: my-app-loadbalancer-service spec: type: LoadBalancer selector: app: my-app ports: - protocol: TCP port: 80 targetPort: 8000

3.3.2 Ingress管理外部访问

apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: my-app-ingress annotations: nginx.ingress.kubernetes.io/rewrite-target: / cert-manager.io/cluster-issuer: letsencrypt-prod spec: tls: - hosts: - myapp.example.com secretName: myapp-tls rules: - host: myapp.example.com http: paths: - path: / pathType: Prefix backend: service: name: my-app-service port: number: 80

4. 应用部署策略

4.1 滚动更新策略

滚动更新是Kubernetes默认的更新策略，可以逐步替换旧版本的Pod。

apiVersion: apps/v1 kind: Deployment metadata: name: rolling-update-demo spec: replicas: 5 strategy: type: RollingUpdate rollingUpdate: maxUnavailable: 2 # 最多可以有多少个Pod同时处于不可用状态 maxSurge: 2 # 最多可以额外创建多少个Pod template: metadata: labels: app: rolling-update-demo spec: containers: - name: app image: your-harbor-domain/project/my-app:v1.0.0 ports: - containerPort: 8000

执行滚动更新：

# 更新镜像版本 kubectl set image deployment/rolling-update-demo app=your-harbor-domain/project/my-app:v2.0.0 # 或者直接编辑Deployment kubectl edit deployment rolling-update-demo # 查看更新状态 kubectl rollout status deployment/rolling-update-demo # 查看更新历史 kubectl rollout history deployment/rolling-update-demo # 回滚到上一版本 kubectl rollout undo deployment/rolling-update-demo # 回滚到指定版本 kubectl rollout undo deployment/rolling-update-demo --to-revision=1

4.2 蓝绿部署策略

蓝绿部署通过同时运行两个版本的应用（蓝和绿），然后切换流量来实现零停机部署。

# 蓝环境（v1） apiVersion: apps/v1 kind: Deployment metadata: name: my-app-blue spec: replicas: 3 selector: matchLabels: app: my-app version: v1 template: metadata: labels: app: my-app version: v1 spec: containers: - name: my-app image: your-harbor-domain/project/my-app:v1.0.0 ports: - containerPort: 8000 --- # 绿环境（v2） apiVersion: apps/v1 kind: Deployment metadata: name: my-app-green spec: replicas: 0 # 初始不创建Pod selector: matchLabels: app: my-app version: v2 template: metadata: labels: app: my-app version: v2 spec: containers: - name: my-app image: your-harbor-domain/project/my-app:v2.0.0 ports: - containerPort: 8000 --- # Service指向蓝环境 apiVersion: v1 kind: Service metadata: name: my-app-service spec: selector: app: my-app version: v1 # 指向蓝环境 ports: - protocol: TCP port: 80 targetPort: 8000

蓝绿部署切换脚本：

#!/bin/bash # 部署绿环境 kubectl scale deployment my-app-green --replicas=3 # 等待绿环境就绪 kubectl wait --for=condition=available deployment/my-app-green --timeout=60s # 切换Service到绿环境 kubectl patch service my-app-service -p '{"spec":{"selector":{"version":"v2"}}}' # 等待切换完成 sleep 10 # 缩容蓝环境 kubectl scale deployment my-app-blue --replicas=0 echo "蓝绿部署完成，流量已切换到v2版本"

4.3 金丝雀发布策略

金丝雀发布通过向新版本逐步引导流量来降低发布风险。

# 主版本（v1） apiVersion: apps/v1 kind: Deployment metadata: name: my-app-primary spec: replicas: 4 selector: matchLabels: app: my-app track: stable template: metadata: labels: app: my-app track: stable spec: containers: - name: my-app image: your-harbor-domain/project/my-app:v1.0.0 ports: - containerPort: 8000 --- # 金丝雀版本（v2） apiVersion: apps/v1 kind: Deployment metadata: name: my-app-canary spec: replicas: 1 # 初始只部署1个实例 selector: matchLabels: app: my-app track: canary template: metadata: labels: app: my-app track: canary spec: containers: - name: my-app image: your-harbor-domain/project/my-app:v2.0.0 ports: - containerPort: 8000 --- # 使用Service将流量分配到两个版本 apiVersion: v1 kind: Service metadata: name: my-app-service spec: selector: app: my-app ports: - protocol: TCP port: 80 targetPort: 8000

使用Istio实现更精细的流量控制：

apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: my-app spec: hosts: - my-app-service http: - route: - destination: host: my-app-service subset: v1 weight: 90 # 90%流量到v1 - destination: host: my-app-service subset: v2 weight: 10 # 10%流量到v2（金丝雀） --- apiVersion: networking.istio.io/v1alpha3 kind: DestinationRule metadata: name: my-app spec: host: my-app-service subsets: - name: v1 labels: version: v1 - name: v2 labels: version: v2

金丝雀发布渐进式流量调整脚本：

#!/bin/bash # 初始流量分配 v1_weight=90 v2_weight=10 # 渐进式增加金丝雀流量 for i in {1..5} do # 更新VirtualService中的流量权重 cat <<EOF | kubectl apply -f - apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: my-app spec: hosts: - my-app-service http: - route: - destination: host: my-app-service subset: v1 weight: $v1_weight - destination: host: my-app-service subset: v2 weight: $v2_weight EOF echo "流量分配: v1=$v1_weight%, v2=$v2_weight%" # 等待一段时间观察指标 sleep 300 # 调整权重 v1_weight=$((v1_weight - 20)) v2_weight=$((v2_weight + 20)) done # 最终全部切换到v2 cat <<EOF | kubectl apply -f - apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: my-app spec: hosts: - my-app-service http: - route: - destination: host: my-app-service subset: v2 weight: 100 EOF echo "金丝雀发布完成，100%流量已切换到v2版本"

5. 配置管理

5.1 ConfigMap管理应用配置

ConfigMap用于存储非机密的配置数据，如环境变量、配置文件等。

5.1.1 创建ConfigMap

# 使用字面量创建 apiVersion: v1 kind: ConfigMap metadata: name: app-config data: LOG_LEVEL: "info" FEATURE_FLAGS: "feature1,feature2" --- # 使用文件创建 apiVersion: v1 kind: ConfigMap metadata: name: app-config-files data: application.properties: | server.port=8080 spring.datasource.url=jdbc:mysql://db:3306/mydb spring.datasource.username=admin spring.datasource.password=pass logback.xml: | <configuration> <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender"> <encoder> <pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern> </encoder> </appender> <root level="info"> <appender-ref ref="STDOUT" /> </root> </configuration>

5.1.2 在Pod中使用ConfigMap

apiVersion: v1 kind: Pod metadata: name: configmap-demo-pod spec: containers: - name: demo image: your-harbor-domain/project/my-app:latest # 作为环境变量 envFrom: - configMapRef: name: app-config # 作为单个环境变量 env: - name: SPECIAL_LOG_LEVEL valueFrom: configMapKeyRef: name: app-config key: LOG_LEVEL # 作为文件挂载 volumeMounts: - name: config-volume mountPath: /etc/config volumes: - name: config-volume configMap: name: app-config-files

5.2 Secret管理敏感数据

Secret用于存储敏感数据，如密码、API密钥、TLS证书等。

5.2.1 创建Secret

# 通用类型Secret apiVersion: v1 kind: Secret metadata: name: app-secret type: Opaque data: # echo -n 'admin' | base64 username: YWRtaW4= # echo -n 'password' | base64 password: cGFzc3dvcmQ= --- # Docker registry类型的Secret apiVersion: v1 kind: Secret metadata: name: registry-secret type: kubernetes.io/dockerconfigjson data: # echo -n '{"auths":{"your-harbor-domain":{"username":"user","password":"pass","email":"user@example.com","auth":"base64-encoded-auth"}}}' | base64 .dockerconfigjson: eyJhdXRocyI6eyJ5b3VyLWhhcmJvci1kb21haW4iOnsidXNlcm5hbWUiOiJ1c2VyIiwicGFzc3dvcmQiOiJwYXNzIiwiZW1haWwiOiJ1c2VyQGV4YW1wbGUuY29tIiwiYXV0aCI6ImJhc2U2NC1lbmNvZGVkLWF1dGgifX19 --- # TLS类型的Secret apiVersion: v1 kind: Secret metadata: name: tls-secret type: kubernetes.io/tls data: # base64编码的证书和私钥 tls.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0t... tls.key: LS0tLS1CRUdJTiBQUklWQVRFIEtFWS0tLS0t...

5.2.2 在Pod中使用Secret

apiVersion: v1 kind: Pod metadata: name: secret-demo-pod spec: containers: - name: demo image: your-harbor-domain/project/my-app:latest # 作为环境变量 env: - name: DB_USERNAME valueFrom: secretKeyRef: name: app-secret key: username - name: DB_PASSWORD valueFrom: secretKeyRef: name: app-secret key: password # 作为文件挂载 volumeMounts: - name: secret-volume mountPath: "/etc/secret" readOnly: true # 使用镜像拉取Secret imagePullSecrets: - name: registry-secret volumes: - name: secret-volume secret: secretName: app-secret

5.3 配置管理最佳实践

配置外部化：避免将配置硬编码到应用中，使用ConfigMap和Secret管理。
环境隔离：为不同环境（开发、测试、生产）创建不同的配置。

# 开发环境ConfigMap apiVersion: v1 kind: ConfigMap metadata: name: app-config-dev namespace: development data: LOG_LEVEL: "debug" DATABASE_URL: "jdbc:mysql://dev-db:3306/mydb" --- # 生产环境ConfigMap apiVersion: v1 kind: ConfigMap metadata: name: app-config-prod namespace: production data: LOG_LEVEL: "warn" DATABASE_URL: "jdbc:mysql://prod-db:3306/mydb"

配置版本控制：将配置文件纳入版本控制系统，使用GitOps方法管理。
敏感数据加密：使用Sealed Secrets或Vault等工具增强Secret的安全性。

# 安装Sealed Secrets kubectl apply -f https://github.com/bitnami-labs/sealed-secrets/releases/download/v0.17.0/controller.yaml # 创建密钥对 kubeseal --fetch-cert > mycert.pem # 加密Secret kubeseal --format=yaml --cert=mycert.pem < secret.yaml > sealed-secret.yaml # 应用加密后的Secret kubectl apply -f sealed-secret.yaml

6. 存储管理

6.1 持久化存储基础

Kubernetes提供了多种持久化存储解决方案，以满足有状态应用的需求。

6.1.1 Volume类型

Kubernetes支持多种Volume类型：

emptyDir：临时存储，Pod删除时数据丢失。
hostPath：使用节点上的文件系统，不适用于多节点集群。
nfs：网络文件系统，适用于多节点共享存储。
persistentVolumeClaim：动态或静态持久化存储。
configMap/secret：将配置或密钥作为文件挂载。
云存储：如AWS EBS、GCE Persistent Disk、Azure Disk等。

6.1.2 使用持久化存储示例

apiVersion: v1 kind: PersistentVolumeClaim metadata: name: app-pvc spec: accessModes: - ReadWriteOnce resources: requests: storage: 10Gi storageClassName: standard --- apiVersion: apps/v1 kind: Deployment metadata: name: app-with-pvc spec: replicas: 1 selector: matchLabels: app: app-with-pvc template: metadata: labels: app: app-with-pvc spec: containers: - name: app image: your-harbor-domain/project/my-app:latest volumeMounts: - name: app-storage mountPath: /data volumes: - name: app-storage persistentVolumeClaim: claimName: app-pvc

6.2 StorageClass与动态卷供应

StorageClass定义了存储的”类型”，允许动态创建PersistentVolume。

6.2.1 创建StorageClass

# AWS EBS StorageClass示例 apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: fast-ssd provisioner: kubernetes.io/aws-ebs parameters: type: gp2 replication-type: io1 iopsPerGB: "10" fsType: ext4 --- # NFS StorageClass示例 apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: nfs-storage provisioner: example.com/nfs parameters: archiveOnDelete: "false"

6.2.2 使用动态卷供应

apiVersion: v1 kind: PersistentVolumeClaim metadata: name: dynamic-pvc spec: accessModes: - ReadWriteOnce storageClassName: fast-ssd # 指定StorageClass resources: requests: storage: 20Gi

6.3 有状态应用部署

StatefulSet用于管理有状态应用，如数据库、消息队列等。

6.3.1 StatefulSet示例

apiVersion: v1 kind: Service metadata: name: mysql labels: app: mysql spec: ports: - port: 3306 name: mysql clusterIP: None # Headless Service selector: app: mysql --- apiVersion: apps/v1 kind: StatefulSet metadata: name: mysql spec: serviceName: mysql replicas: 3 selector: matchLabels: app: mysql template: metadata: labels: app: mysql spec: containers: - name: mysql image: mysql:5.7 ports: - containerPort: 3306 name: mysql env: - name: MYSQL_ROOT_PASSWORD valueFrom: secretKeyRef: name: mysql-secret key: password volumeMounts: - name: mysql-persistent-storage mountPath: /var/lib/mysql - name: mysql-config mountPath: /etc/mysql/conf.d volumeClaimTemplates: # 为每个Pod创建独立的PVC - metadata: name: mysql-persistent-storage spec: accessModes: [ "ReadWriteOnce" ] resources: requests: storage: 10Gi storageClassName: fast-ssd - metadata: name: mysql-config spec: accessModes: [ "ReadWriteOnce" ] resources: requests: storage: 1Gi storageClassName: fast-ssd

6.3.2 使用Operator管理复杂应用

Operator是使用自定义资源扩展Kubernetes API的方法，常用于管理复杂的有状态应用。

以MySQL Operator为例：

# 安装MySQL Operator kubectl apply -f https://raw.githubusercontent.com/mysql/mysql-operator/trunk/deploy/deploy-crds.yaml kubectl apply -f https://raw.githubusercontent.com/mysql/mysql-operator/trunk/deploy/deploy-operator.yaml # 创建MySQL集群 cat <<EOF | kubectl apply -f - apiVersion: mysql.oracle.com/v2 kind: InnoDBCluster metadata: name: mycluster spec: secretName: mypwds tlsUseSecret: my-tls-secret instances: 3 router: instances: 1 EOF

7. 网络策略

7.1 Kubernetes网络模型

Kubernetes网络模型要求所有Pod可以在不使用NAT的情况下相互通信，并且节点可以与所有Pod通信。

7.1.1 CNI插件选择

常见的CNI插件包括：

Calico：支持网络策略，提供网络安全和连接。
Flannel：简单易用的覆盖网络。
Weave Net：创建虚拟网络，加密通信。
Cilium：基于eBPF的高性能网络。

7.1.2 安装Calico网络插件

# 安装Calico kubectl apply -f https://docs.projectcalico.org/manifests/calico.yaml # 验证安装 kubectl get pods -n kube-system -l k8s-app=calico-node

7.2 网络策略

网络策略用于控制Pod之间的网络流量，实现微服务隔离。

7.2.1 基本网络策略示例

# 允许来自特定命名空间的流量 apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: allow-namespace namespace: production spec: podSelector: {} policyTypes: - Ingress ingress: - from: - namespaceSelector: matchLabels: name: monitoring --- # 允许特定Pod之间的通信 apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: backend-policy spec: podSelector: matchLabels: app: backend policyTypes: - Ingress - Egress ingress: - from: - podSelector: matchLabels: app: frontend ports: - protocol: TCP port: 8080 egress: - to: - podSelector: matchLabels: app: database ports: - protocol: TCP port: 3306

7.2.2 默认拒绝所有流量

# 默认拒绝所有入站流量 apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: default-deny-ingress spec: podSelector: {} policyTypes: - Ingress --- # 默认拒绝所有出站流量 apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: default-deny-egress spec: podSelector: {} policyTypes: - Egress

7.3 服务网格

服务网格如Istio提供了更高级的网络管理功能，包括流量管理、安全、可观察性等。

7.3.1 安装Istio

# 下载Istio curl -L https://istio.io/downloadIstio | sh - cd istio-* # 安装Istio istioctl install --set profile=demo # 启用sidecar自动注入 kubectl label namespace default istio-injection=enabled

7.3.2 使用Istio管理流量

# 目标规则定义版本 apiVersion: networking.istio.io/v1alpha3 kind: DestinationRule metadata: name: my-app spec: host: my-app-service subsets: - name: v1 labels: version: v1 - name: v2 labels: version: v2 --- # 虚拟服务定义路由规则 apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: my-app spec: hosts: - my-app-service http: - match: - headers: end-user: exact: jason route: - destination: host: my-app-service subset: v2 - route: - destination: host: my-app-service subset: v1 --- # 故障注入 apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: reviews spec: hosts: - reviews http: - fault: delay: percentage: value: 0.1 fixedDelay: 5s route: - destination: host: reviews subset: v1

8. 安全最佳实践

8.1 RBAC权限控制

基于角色的访问控制（RBAC）是Kubernetes中管理权限的主要方式。

8.1.1 创建Role和RoleBinding

# 定义Role apiVersion: rbac.authorization.k8s.io/v1 kind: Role metadata: namespace: development name: pod-reader rules: - apiGroups: [""] # "" indicates the core API group resources: ["pods"] verbs: ["get", "watch", "list"] --- # 绑定Role到用户 apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding metadata: name: read-pods namespace: development subjects: - kind: User name: jane apiGroup: rbac.authorization.k8s.io roleRef: kind: Role name: pod-reader apiGroup: rbac.authorization.k8s.io

8.1.2 创建ClusterRole和ClusterRoleBinding

# 定义ClusterRole apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: secret-reader rules: - apiGroups: [""] resources: ["secrets"] verbs: ["get", "watch", "list"] --- # 绑定ClusterRole到ServiceAccount apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: read-secrets-global subjects: - kind: ServiceAccount name: my-sa namespace: development roleRef: kind: ClusterRole name: secret-reader apiGroup: rbac.authorization.k8s.io

8.2 Pod安全策略

Pod安全策略（Pod Security Policy）控制Pod的安全敏感配置。

8.2.1 创建Pod安全策略

apiVersion: policy/v1beta1 kind: PodSecurityPolicy metadata: name: restricted spec: privileged: false # 禁止特权容器 allowPrivilegeEscalation: false requiredDropCapabilities: - ALL volumes: - 'configMap' - 'emptyDir' - 'projected' - 'secret' - 'downwardAPI' - 'persistentVolumeClaim' runAsUser: rule: 'MustRunAsNonRoot' seLinux: rule: 'RunAsAny' fsGroup: rule: 'RunAsAny' --- # 创建Role和RoleBinding以使用PSP apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: psp:restricted rules: - apiGroups: ['policy'] resources: ['podsecuritypolicies'] verbs: ['use'] resourceNames: ['restricted'] --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: psp:restricted roleRef: kind: ClusterRole name: psp:restricted apiGroup: rbac.authorization.k8s.io subjects: - kind: Group name: system:serviceaccounts

8.3 容器运行时安全

8.3.1 使用安全上下文

apiVersion: v1 kind: Pod metadata: name: security-context-demo spec: securityContext: fsGroup: 2000 containers: - name: sec-ctx-demo image: your-harbor-domain/project/my-app:latest securityContext: runAsUser: 1000 runAsGroup: 3000 allowPrivilegeEscalation: false readOnlyRootFilesystem: true capabilities: drop: ["ALL"] add: ["NET_ADMIN"]

8.3.2 使用AppArmor或SELinux

# 使用AppArmor apiVersion: v1 kind: Pod metadata: name: apparmor-demo annotations: container.apparmor.security.beta.kubernetes.io/my-app: runtime/default spec: containers: - name: my-app image: your-harbor-domain/project/my-app:latest

8.4 镜像安全

8.4.1 使用镜像签名和验证

# 安装sigstore工具 go install github.com/sigstore/cosign/cmd/cosign@latest # 生成密钥对 cosign generate-key-pair # 签名镜像 cosign sign --key cosign.key your-harbor-domain/project/my-app:latest # 验证镜像 cosign verify --key cosign.pub your-harbor-domain/project/my-app:latest

8.4.2 在Kubernetes中启用镜像验证

# 创建验证策略 apiVersion: apps/v1 kind: Deployment metadata: name: verified-image-deployment spec: replicas: 1 template: spec: containers: - name: my-app image: your-harbor-domain/project/my-app:latest imagePullSecrets: - name: regcred metadata: annotations: admission.policy.sigstore.dev/include: "true" admission.policy.sigstore.dev/verify: "true" admission.policy.sigstore.dev/keyless: "false" admission.policy.sigstore.dev/keys: | -----BEGIN PUBLIC KEY----- MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAE8nXRh950IZbRj8Ra/R9X2rB4R5d -----END PUBLIC KEY-----

9. 监控与日志

9.1 监控系统

9.1.1 安装Prometheus和Grafana

# 创建命名空间 kubectl create namespace monitoring # 安装Prometheus Operator kubectl apply -f https://raw.githubusercontent.com/prometheus-operator/prometheus-operator/master/bundle.yaml # 创建Prometheus实例 cat <<EOF | kubectl apply -f - apiVersion: monitoring.coreos.com/v1 kind: Prometheus metadata: name: prometheus namespace: monitoring spec: serviceAccountName: prometheus serviceMonitorSelector: matchLabels: team: frontend resources: requests: memory: 400Mi enableAdminAPI: false EOF # 创建ServiceMonitor cat <<EOF | kubectl apply -f - apiVersion: monitoring.coreos.com/v1 kind: ServiceMonitor metadata: name: my-app-monitor namespace: monitoring labels: team: frontend spec: selector: matchLabels: app: my-app endpoints: - port: web interval: 30s path: /metrics EOF # 安装Grafana helm repo add grafana https://grafana.github.io/helm-charts helm install grafana grafana/grafana --namespace monitoring --set persistence.storageClassName="fast-ssd" --set persistence.enabled=true --set adminPassword="admin"

9.1.2 配置告警规则

apiVersion: monitoring.coreos.com/v1 kind: PrometheusRule metadata: name: my-app-alerts namespace: monitoring labels: team: frontend spec: groups: - name: my-app rules: - alert: HighRequestLatency expr: job:request_latency_seconds:mean5m{job="my-app"} > 0.5 for: 10m labels: severity: warning annotations: summary: "High request latency on {{ $labels.instance }}" description: "{{ $labels.job }} has a high request latency: {{ $value }}s" - alert: ServiceDown expr: up{job="my-app"} == 0 for: 1m labels: severity: critical annotations: summary: "Service {{ $labels.job }} is down" description: "{{ $labels.job }} on {{ $labels.instance }} has been down for more than 1 minute"

9.2 日志管理

9.2.1 使用EFK栈（Elasticsearch、Fluentd、Kibana）

# 安装Elasticsearch helm repo add elastic https://helm.elastic.co helm install elasticsearch elastic/elasticsearch --namespace logging --set replicas=3 --set volumeClaimTemplate.resources.requests.storage=50Gi # 安装Kibana helm install kibana elastic/kibana --namespace logging # 安装Fluentd helm repo add fluent https://fluent.github.io/helm-charts helm install fluent fluent/fluentd --namespace logging -f fluentd-values.yaml

fluentd-values.yaml示例：

# fluentd-values.yaml fullnameOverride: fluentd image: tag: v1.14.0 configMaps: fluent.conf: |- <source> @type tail path /var/log/containers/*my-app*.log pos_file /var/log/fluentd-containers.log.pos tag kubernetes.* format json time_format %Y-%m-%dT%H:%M:%S.%NZ </source> <match kubernetes.**> @type elasticsearch host elasticsearch-master port 9200 index_name fluentd type_name _doc </match>

9.2.2 使用Loki和Promtail

# 安装Loki helm repo add grafana https://grafana.github.io/helm-charts helm install loki grafana/loki --namespace logging # 安装Promtail helm install promtail grafana/promtail --namespace logging --set config.lokiAddress=http://loki:3100

9.3 分布式追踪

9.3.1 安装Jaeger

# 使用Operator安装Jaeger kubectl create namespace observability kubectl apply -f https://github.com/jaegertracing/jaeger-operator/releases/download/v1.28.0/jaeger-operator.yaml -n observability # 创建Jaeger实例 cat <<EOF | kubectl apply -f - apiVersion: jaegertracing.io/v1 kind: Jaeger metadata: name: simplest EOF

9.3.2 在应用中集成Jaeger客户端

// Java应用集成Jaeger客户端示例 // 添加依赖 // implementation 'io.jaegertracing:jaeger-client:1.6.0' // 初始化Tracer public class JaegerConfig { @Bean public io.opentracing.Tracer jaegerTracer() { return new Configuration("my-app") .withSampler(new Configuration.SamplerConfiguration() .withType("const") .withParam(1)) .withReporter(new Configuration.ReporterConfiguration() .withLogSpans(true) .withSender(new Configuration.SenderConfiguration() .withAgentHost("jaeger-agent") .withAgentPort(6831))) .getTracer(); } } // 使用Tracer @RestController public class MyController { private final io.opentracing.Tracer tracer; public MyController(io.opentracing.Tracer tracer) { this.tracer = tracer; } @GetMapping("/api/data") public String getData() { Span span = tracer.buildSpan("getData-operation").startActive().span(); try { // 业务逻辑 return "data"; } finally { span.finish(); } } }

10. CI/CD集成

10.1 GitOps工作流

GitOps是一种持续交付方法，它使用Git作为声明式基础设施和应用程序的真实来源。

10.1.1 使用Argo CD实现GitOps

# 安装Argo CD kubectl create namespace argocd kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml # 获取初始密码 kubectl get pods -n argocd -l app.kubernetes.io/name=argocd-server -o name | cut -d'/' -f 2 argocd admin initial-password -n argocd <pod-name> # 创建应用程序 cat <<EOF | kubectl apply -f - apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: my-app namespace: argocd spec: project: default source: repoURL: https://github.com/your-org/my-app-manifests.git targetRevision: HEAD path: prod destination: server: https://kubernetes.default.svc namespace: production syncPolicy: automated: prune: true selfHeal: true EOF

10.1.2 使用Flux CD实现GitOps

# 安装Flux CLI curl -s https://toolkit.fluxcd.io/install.sh | sudo bash # 检查先决条件 flux check --pre # 导出GitHub个人访问令牌 export GITHUB_TOKEN=<your-token> export GITHUB_USER=<your-username> export GITHUB_REPO=<your-repo> # 在集群上引导Flux flux bootstrap github --owner=$GITHUB_USER --repository=$GITHUB_REPO --path=clusters/my-cluster --personal # 创建应用程序源 flux create source git my-app --url=https://github.com/$GITHUB_USER/$GITHUB_REPO --branch=main --interval=30s --export > ./clusters/my-cluster/my-app-source.yaml # 创建Kustomization flux create kustomization my-app --source=my-app --path="./deploy/production" --prune=true --interval=5m --export > ./clusters/my-cluster/my-app-kustomization.yaml

10.2 Jenkins与Kubernetes集成

10.2.1 在Kubernetes中部署Jenkins

apiVersion: v1 kind: PersistentVolumeClaim metadata: name: jenkins-pvc spec: accessModes: - ReadWriteOnce resources: requests: storage: 20Gi --- apiVersion: apps/v1 kind: Deployment metadata: name: jenkins spec: replicas: 1 selector: matchLabels: app: jenkins template: metadata: labels: app: jenkins spec: containers: - name: jenkins image: jenkins/jenkins:lts ports: - containerPort: 8080 - containerPort: 50000 volumeMounts: - name: jenkins-home mountPath: /var/jenkins_home volumes: - name: jenkins-home persistentVolumeClaim: claimName: jenkins-pvc --- apiVersion: v1 kind: Service metadata: name: jenkins-service spec: type: LoadBalancer ports: - port: 80 targetPort: 8080 name: http - port: 50000 targetPort: 50000 name: agent selector: app: jenkins

10.2.2 Jenkins Pipeline示例

pipeline { agent { kubernetes { label 'jenkins-agent' defaultContainer 'jnlp' yaml """ apiVersion: v1 kind: Pod metadata: labels: app: jenkins-agent spec: containers: - name: jnlp image: jenkins/inbound-agent:4.6-1 - name: docker image: docker:latest command: - cat tty: true volumeMounts: - name: docker-sock mountPath: /var/run/docker.sock - name: kubectl image: bitnami/kubectl:latest command: - cat tty: true volumes: - name: docker-sock hostPath: path: /var/run/docker.sock """ } } environment { DOCKER_REGISTRY = 'your-harbor-domain' DOCKER_CREDENTIALS_ID = 'docker-registry-credentials' KUBECONFIG_CREDENTIALS_ID = 'kubeconfig' } stages { stage('Checkout') { steps { checkout scm } } stage('Build') { steps { container('docker') { script { def appImage = docker.build("${DOCKER_REGISTRY}/project/my-app:${env.BUILD_ID}") docker.withRegistry("https://${DOCKER_REGISTRY}", DOCKER_CREDENTIALS_ID) { appImage.push() } } } } } stage('Test') { steps { container('docker') { sh 'docker run --rm ${DOCKER_REGISTRY}/project/my-app:${env.BUILD_ID} npm test' } } } stage('Deploy to Staging') { steps { container('kubectl') { withCredentials([file(credentialsId: KUBECONFIG_CREDENTIALS_ID, variable: 'KUBECONFIG')]) { sh 'kubectl config use-context staging' sh "sed -i 's/IMAGE_TAG/${env.BUILD_ID}/g' k8s/staging/deployment.yaml" sh 'kubectl apply -f k8s/staging/' sh 'kubectl rollout status deployment/my-app-staging' } } } } stage('Approval') { steps { input "Deploy to Production?" } } stage('Deploy to Production') { steps { container('kubectl') { withCredentials([file(credentialsId: KUBECONFIG_CREDENTIALS_ID, variable: 'KUBECONFIG')]) { sh 'kubectl config use-context production' sh "sed -i 's/IMAGE_TAG/${env.BUILD_ID}/g' k8s/production/deployment.yaml" sh 'kubectl apply -f k8s/production/' sh 'kubectl rollout status deployment/my-app-production' } } } } } post { always { echo 'Cleaning up...' cleanWs() } success { echo 'Pipeline succeeded!' } failure { echo 'Pipeline failed!' } } }

10.3 GitHub Actions与Kubernetes集成

10.3.1 GitHub Actions工作流示例

name: Build and Deploy to Kubernetes on: push: branches: - main pull_request: branches: - main jobs: build: runs-on: ubuntu-latest steps: - name: Checkout code uses: actions/checkout@v2 - name: Set up Docker Buildx uses: docker/setup-buildx-action@v1 - name: Login to Docker Registry uses: docker/login-action@v1 with: registry: your-harbor-domain username: ${{ secrets.DOCKER_USERNAME }} password: ${{ secrets.DOCKER_PASSWORD }} - name: Build and push Docker image uses: docker/build-push-action@v2 with: context: . push: true tags: your-harbor-domain/project/my-app:${{ github.sha }} cache-from: type=registry,ref=your-harbor-domain/project/my-app:buildcache cache-to: type=registry,ref=your-harbor-domain/project/my-app:buildcache,mode=max - name: Update Kubernetes manifest run: | sed -i 's|your-harbor-domain/project/my-app:.*|your-harbor-domain/project/my-app:${{ github.sha }}|' k8s/production/deployment.yaml - name: Commit and push changes run: | git config --global user.name 'GitHub Actions' git config --global user.email 'actions@github.com' git add k8s/production/deployment.yaml git commit -m "Update image tag to ${{ github.sha }}" git push deploy-staging: needs: build runs-on: ubuntu-latest if: github.ref == 'refs/heads/main' environment: staging steps: - name: Checkout code uses: actions/checkout@v2 - name: Set up Kustomize uses: imjasonh/setup-kustomize@v1 - name: Deploy to staging run: | kustomize build k8s/staging | kubectl apply -f - kubectl rollout status deployment/my-app-staging deploy-production: needs: [build, deploy-staging] runs-on: ubuntu-latest if: github.ref == 'refs/heads/main' environment: production steps: - name: Checkout code uses: actions/checkout@v2 - name: Set up Kustomize uses: imjasonh/setup-kustomize@v1 - name: Deploy to production run: | kustomize build k8s/production | kubectl apply -f - kubectl rollout status deployment/my-app-production

11. 企业级转型挑战与解决方案

11.1 文化与组织变革

11.1.1 DevOps文化转型

企业容器化转型不仅是技术变革，更是文化和组织变革。以下是推动DevOps文化转型的策略：

跨功能团队：组建包含开发、运维、安全和业务人员的跨功能团队，打破部门壁垒。
共同目标：设定团队共同关注的目标，如部署频率、变更失败率、恢复时间等。
持续学习：鼓励知识分享和技术创新，定期举办技术分享会和培训。
自动化优先：推动从手动流程到自动化流程的转变，减少人为错误。
失败容忍：建立安全的失败环境，鼓励实验和创新。

11.1.2 人才培养与技能提升

# Kubernetes学习路径示例 apiVersion: training/v1 kind: LearningPath metadata: name: kubernetes-journey spec: levels: - name: beginner description: "Kubernetes基础概念和操作" duration: "4 weeks" modules: - title: "容器化基础" content: "Docker基础、容器化应用开发" resources: - "Docker官方文档" - "容器化最佳实践指南" - title: "Kubernetes核心概念" content: "Pod、Deployment、Service、Ingress等核心资源" resources: - "Kubernetes官方文档" - "Kubernetes Patterns书籍" - title: "kubectl基础操作" content: "常用kubectl命令、调试技巧" resources: - "kubectl Cheat Sheet" - "Kubernetes调试指南" - name: intermediate description: "Kubernetes进阶应用" duration: "8 weeks" modules: - title: "配置管理" content: "ConfigMap、Secret使用、配置最佳实践" resources: - "Kubernetes配置管理指南" - "12-Factor App方法论" - title: "存储管理" content: "持久化存储、StatefulSet、存储类" resources: - "Kubernetes存储管理指南" - "有状态应用部署最佳实践" - title: "网络策略" content: "CNI插件、网络策略、服务网格" resources: - "Kubernetes网络指南" - "Istio服务网格文档" - name: advanced description: "企业级Kubernetes运维" duration: "12 weeks" modules: - title: "安全实践" content: "RBAC、Pod安全策略、镜像安全" resources: - "Kubernetes安全指南" - "CIS Kubernetes基准" - title: "监控与日志" content: "Prometheus、Grafana、EFK栈、分布式追踪" resources: - "Kubernetes监控指南" - "分布式追踪系统实践" - title: "CI/CD集成" content: "GitOps、Jenkins、GitHub Actions" resources: - "GitOps实践指南" - "Kubernetes CI/CD最佳实践"

11.2 技术挑战与解决方案

11.2.1 遗留系统容器化

遗留系统容器化是企业转型过程中的一大挑战。以下是解决方案：

评估与规划：
- 识别适合容器化的应用
- 评估容器化的收益和成本
- 制定分阶段迁移计划
重构策略：
- 直接迁移（Lift and Shift）：适合简单应用，快速迁移但无法充分利用容器优势。
- 重构后迁移：修改应用架构，使其更适合容器环境。
- 替换：用云原生应用替换遗留系统。
混合部署： “`yaml
遗留系统与容器化应用共存的示例
apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: hybrid-app-ingress annotations: nginx.ingress.kubernetes.io/rewrite-target: / nginx.ingress.kubernetes.io/backend-protocol: “HTTPS” spec: rules:
- host: legacy-app.example.com http: paths:
  - path: / pathType: Prefix backend: service: name: legacy-app-service port: number: 80
- host: new-app.example.com http: paths:
  - path: / pathType: Prefix backend: service: name: new-app-service port: number: 80
”`

11.2.2 大规模集群管理

随着企业容器化规模扩大，集群管理面临新的挑战：

多集群管理： “`bash
使用Cluster API管理多集群
安装Cluster API
curl -L https://github.com/kubernetes-sigs/cluster-api/releases/download/v1.1.2/clusterctl-linux-amd64 -o clusterctl chmod +x ./clusterctl sudo mv ./clusterctl /usr/local/bin/clusterctl

# 初始化管理集群 clusterctl init –infrastructure aws

# 创建工作集群 clusterctl config cluster my-workload-cluster > my-workload-cluster.yaml clusterctl apply -f my-workload-cluster.yaml

 2. **联邦集群**： ```yaml # 使用KubeFed管理联邦集群 # 安装KubeFederation kubectl apply -f https://github.com/kubernetes-sigs/kubefed/releases/download/v0.9.0/kubefed-operator.yaml # 创建联邦集群 kubefedctl join cluster1 --cluster-context cluster1 --host-cluster-context cluster-hub kubefedctl join cluster2 --cluster-context cluster2 --host-cluster-context cluster-hub # 部署联邦资源 apiVersion: types.kubefed.io/v1beta1 kind: FederatedDeployment metadata: name: my-app namespace: test-namespace spec: template: spec: replicas: 5 selector: matchLabels: app: my-app template: metadata: labels: app: my-app spec: containers: - name: my-app image: your-harbor-domain/project/my-app:latest ports: - containerPort: 8080 placement: clusters: - name: cluster1 - name: cluster2

集群自动扩缩容：

# 使用Cluster Autoscaler apiVersion: v1 kind: ConfigMap metadata: name: cluster-autoscaler namespace: kube-system data: balance-similar-node-groups: "true" expander: "least-waste" skip-nodes-with-local-storage: "false" skip-nodes-with-system-pods: "false" --- # 部署Cluster Autoscaler apiVersion: apps/v1 kind: Deployment metadata: name: cluster-autoscaler namespace: kube-system spec: replicas: 1 selector: matchLabels: app: cluster-autoscaler template: metadata: labels: app: cluster-autoscaler spec: containers: - image: k8s.gcr.io/autoscaling/cluster-autoscaler:v1.22.0 name: cluster-autoscaler command: - ./cluster-autoscaler - --cloud-provider=aws - --namespace=kube-system - --balance-similar-node-groups - --expander=least-waste env: - name: AWS_REGION value: us-west-2 volumeMounts: - name: ssl-certs mountPath: /etc/ssl/certs/ca-certificates.crt readOnly: true volumes: - name: ssl-certs hostPath: path: "/etc/ssl/certs/ca-bundle.crt"

11.3 成本优化

11.3.1 资源优化

# 使用Vertical Pod Autoscaler优化资源分配 apiVersion: autoscaling.k8s.io/v1 kind: VerticalPodAutoscaler metadata: name: my-app-vpa spec: targetRef: apiVersion: "apps/v1" kind: "Deployment" name: "my-app" updatePolicy: updateMode: "Auto" resourcePolicy: containerPolicies: - containerName: "*" minAllowed: cpu: "100m" memory: "50Mi" maxAllowed: cpu: "1" memory: "500Mi" controlledResources: ["cpu", "memory"]

11.3.2 多租户资源隔离

# 使用ResourceQuota限制命名空间资源 apiVersion: v1 kind: ResourceQuota metadata: name: compute-resources namespace: team-a spec: hard: requests.cpu: "4" requests.memory: "8Gi" limits.cpu: "10" limits.memory: "16Gi" persistentvolumeclaims: "5" requests.storage: "50Gi" --- # 使用LimitRange限制容器资源 apiVersion: v1 kind: LimitRange metadata: name: container-limits namespace: team-a spec: limits: - default: cpu: "500m" memory: "512Mi" defaultRequest: cpu: "250m" memory: "256Mi" type: Container

11.3.3 自动扩缩容策略

# 使用Horizontal Pod Autoscaler apiVersion: autoscaling/v2 kind: HorizontalPodAutoscaler metadata: name: my-app-hpa spec: scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: my-app minReplicas: 2 maxReplicas: 10 metrics: - type: Resource resource: name: cpu target: type: Utilization averageUtilization: 50 - type: Resource resource: name: memory target: type: Utilization averageUtilization: 70 - type: Pods pods: metric: name: packets-per-second target: type: AverageValue averageValue: 1k - type: External external: metric: name: queue_messages_ready selector: matchLabels: queue: "worker_tasks" target: type: AverageValue averageValue: 30

12. 案例分析

12.1 电商平台容器化转型案例

12.1.1 背景与挑战

某大型电商平台拥有超过500个微服务，运行在虚拟机和传统中间件上。面临以下挑战：

部署周期长，从代码提交到生产环境需要2-3周
资源利用率低，平均CPU利用率不足20%
扩容能力有限，无法应对大促活动期间的流量高峰
运维成本高，需要大量人力进行系统维护

12.1.2 转型策略

分阶段迁移：
- 第一阶段：无状态服务容器化（如用户服务、商品服务）
- 第二阶段：有状态服务容器化（如订单服务、库存服务）
- 第三阶段：大数据处理服务容器化（如推荐系统、数据分析）
技术栈选择：
- 容器运行时：containerd
- 编排平台：Kubernetes
- 服务网格：Istio
- 监控系统：Prometheus + Grafana
- 日志系统：ELK Stack
- CI/CD：GitLab CI + Argo CD
架构改造： “`yaml
电商平台微服务部署示例
apiVersion: v1 kind: Namespace metadata: name: ecommerce labels:
istio-injection: enabled
用户服务
apiVersion: apps/v1 kind: Deployment metadata: name: user-service namespace: ecommerce spec: replicas: 3 selector: matchLabels:
```
 app: user-service 
```
template: metadata:
```
 labels: app: user-service version: v1 
```
spec:
```
 containers: - name: user-service image: ecommerce/user-service:1.2.0 ports: - containerPort: 8080 env: - name: DB_HOST valueFrom: configMapKeyRef: name: user-service-config key: db_host - name: DB_PASSWORD valueFrom: secretKeyRef: name: user-service-secret key: db_password resources: requests: memory: "512Mi" cpu: "250m" limits: memory: "1Gi" cpu: "500m" livenessProbe: httpGet: path: /health port: 8080 initialDelaySeconds: 30 periodSeconds: 10 
```
商品服务
apiVersion: apps/v1 kind: Deployment metadata: name: product-service namespace: ecommerce spec: replicas: 3 selector: matchLabels:
```
 app: product-service 
```
template: metadata:
```
 labels: app: product-service version: v1 
```
spec:
```
 containers: - name: product-service image: ecommerce/product-service:2.1.0 ports: - containerPort: 8080 env: - name: REDIS_HOST value: "redis-cluster:6379" resources: requests: memory: "1Gi" cpu: "500m" limits: memory: "2Gi" cpu: "1000m" 
```
订单服务（有状态）
apiVersion: apps/v1 kind: StatefulSet metadata: name: order-service namespace: ecommerce spec: serviceName: order-service replicas: 3 selector: matchLabels:
```
 app: order-service 
```
template: metadata:
```
 labels: app: order-service 
```
spec:
```
 containers: - name: order-service image: ecommerce/order-service:3.0.0 ports: - containerPort: 8080 volumeMounts: - name: order-data mountPath: /data/orders 
```
volumeClaimTemplates:
- metadata: name: order-data spec: accessModes: [ “ReadWriteOnce” ] resources: requests:
```
 storage: 10Gi 
```
  网关服务
  apiVersion: apps/v1 kind: Deployment metadata: name: api-gateway namespace: ecommerce spec: replicas: 2 selector: matchLabels: app: api-gateway template: metadata: labels: app: api-gateway spec: containers:
  - name: api-gateway image: ecommerce/api-gateway:1.5.0 ports:
```
 - containerPort: 8080 
```
    env:
```
 - name: RATE_LIMITING_ENABLED 
```
    value: “true”
```
 - name: CIRCUIT_BREAKER_ENABLED 
```
    value: “true”
    服务暴露
    apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: ecommerce-ingress namespace: ecommerce annotations: nginx.ingress.kubernetes.io/rewrite-target: / cert-manager.io/cluster-issuer: letsencrypt-prod nginx.ingress.kubernetes.io/rate-limit: “100” nginx.ingress.kubernetes.io/rate-limit-window: “1m” spec: tls:
- hosts:
```
 - shop.example.com 
```
  secretName: shop-tls rules:
- host: shop.example.com http: paths:
  - path: / pathType: Prefix backend: service: name: api-gateway port: number: 80
”`

12.1.3 实施效果

部署效率：部署周期从2-3周缩短至30分钟
资源利用率：CPU利用率提升至65%，节省40%基础设施成本
弹性能力：大促期间自动扩容至10倍容量，系统稳定运行
运维效率：自动化运维减少70%人工干预

12.2 金融机构容器化转型案例

12.2.1 背景与挑战

某大型银行拥有核心银行系统、支付系统、风控系统等关键业务系统，运行在大型机和传统架构上。面临以下挑战：

系统架构陈旧，难以快速响应业务需求
合规要求严格，需要满足金融行业安全标准
系统间耦合度高，难以独立升级和扩展
成本压力大，需要优化IT支出

12.2.2 转型策略

安全优先：
- 建立多层次安全架构
- 实施严格的访问控制和审计
- 满足金融监管合规要求
混合云架构：
- 核心系统保留在私有云
- 创新业务部署在公有云
- 统一管理平台
渐进式迁移：
- 非核心系统先行试点
- 核心系统分模块逐步迁移
- 保持系统稳定性
技术实现： “`yaml
金融机构安全配置示例
apiVersion: v1 kind: Namespace metadata: name: financial-services labels:
pod-security.kubernetes.io/enforce: restricted
网络策略
apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: financial-services-netpol namespace: financial-services spec: podSelector: {} policyTypes:
- Ingress
- Egress ingress:
- from:
```
 - namespaceSelector: 
```
  matchLabels:
```
 name: monitoring 
```
  ports:
```
 - protocol: TCP 
```
  port: 8080 egress:
- to:
```
 - namespaceSelector: 
```
  matchLabels:
```
 name: database 
```
  ports:
```
 - protocol: TCP 
```
  port: 3306
  Pod安全上下文
  apiVersion: apps/v1 kind: Deployment metadata: name: payment-service namespace: financial-services spec: replicas: 3 selector: matchLabels: app: payment-service template: metadata: labels: app: payment-service spec: securityContext: fsGroup: 2000 runAsNonRoot: true runAsUser: 1000 containers:
  - name: payment-service image: financial/payment-service:2.3.0 securityContext: allowPrivilegeEscalation: false readOnlyRootFilesystem: true runAsNonRoot: true runAsUser: 1000 capabilities: drop:
```
 - ALL 
```
    ports:
```
 - containerPort: 8080 
```
    env:
```
 - name: DB_CONNECTION 
```
    valueFrom: secretKeyRef:
```
 name: payment-service-secret key: db_connection 
```
    volumeMounts:
```
 - name: tmp 
```
    mountPath: /tmp
```
 - name: config 
```
    mountPath: /etc/config readOnly: true volumes:
  - name: tmp emptyDir: {}
  - name: config configMap:
    name: payment-service-config
    资源限制
    apiVersion: v1 kind: ResourceQuota metadata: name: financial-services-quota namespace: financial-services spec: hard: requests.cpu: “10” requests.memory: “20Gi” limits.cpu: “20” limits.memory: “40Gi” persistentvolumeclaims: “10” count/deployments: “10”
    count/pods: “50”
    审计日志配置
    apiVersion: audit.k8s.io/v1 kind: Policy rules:
- level: Metadata resources:
  - group: “” resources: [“pods”, “services”, “secrets”, “configmaps”]
- level: Request resources:
  - group: “” resources: [“secrets”]
- level: RequestResponse userGroups: [“system:authenticated”] namespaces: [“financial-services”]
”`