redis使用本地磁盘做pvc

1 local 卷介绍

https://kubernetes.io/zh-cn/docs/concepts/storage/volumes/
https://kubernetes.io/zh-cn/docs/concepts/storage/storage-classes/#local

local 卷所代表的是某个被挂载的本地存储设备，例如磁盘、分区或者目录。

local 卷只能用作静态创建的持久卷。不支持动态配置。

与 hostPath 卷相比，local 卷能够以持久和可移植的方式使用，而无需手动将 Pod 调度到节点。系统通过查看 PersistentVolume 的节点亲和性配置，就能了解卷的节点约束。

然而，local 卷仍然取决于底层节点的可用性，并不适合所有应用程序。如果节点变得不健康，那么 local 卷也将变得不可被 Pod 访问。使用它的 Pod 将不能运行。使用 local 卷的应用程序必须能够容忍这种可用性的降低，以及因底层磁盘的耐用性特征而带来的潜在的数据丢失风险。

2 部署

设置 label

kubectl label node qa-gpu026 redis=true

如果不设置 label ，那么 pvc 会一直处于 Pending 状态，报错如下

0/3 nodes are available: 3 node(s) didn't find available persistent volumes to bind.

redis 部署文件

---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:name: redis-sc
provisioner: kubernetes.io/no-provisioner
volumeBindingMode: WaitForFirstConsumer
reclaimPolicy: Retain---
apiVersion: v1
kind: PersistentVolume
metadata:name: redis-pv
spec:capacity:storage: 100GiaccessModes:- ReadWriteOncepersistentVolumeReclaimPolicy: RetainstorageClassName: redis-schostPath:path: {{redis_base_path}}/redistype: DirectoryOrCreatenodeAffinity:    # Local Persistent Volume（本地持久化卷），必须使用 nodeAffinity 来跟某个具体的节点绑定required:nodeSelectorTerms:- matchExpressions:- key: redisoperator: Invalues:- "true"
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:name: redis-pvcnamespace: infra
spec:storageClassName: redis-scaccessModes:- ReadWriteOnceresources:requests:storage: 10Gi---
apiVersion: v1
kind: Service
metadata:name: redisnamespace: infralabels:app: redis
spec:type: ClusterIPports:- name: redisport: 6379targetPort: 6379#nodePort: 30379- name: metricsport: 9121targetPort: 9121#nodePort: 31121selector:app: redis---
kind: ConfigMap
apiVersion: v1
metadata:name: redis-confignamespace: infralabels:app: redis
data:redis.conf: |-dir /dataport 6379bind 0.0.0.0appendonly yesprotected-mode norequirepass {{redis_password}}pidfile /data/redis-6379.pid
---
apiVersion: apps/v1
kind: Deployment
metadata:name: redisnamespace: infralabels:app: redis
spec:replicas: 1selector:matchLabels:app: redistemplate:metadata:annotations:prometheus.io/scrape: "true"prometheus.io/port: "9121"labels:app: redisspec:nodeSelector:redis: "true"# 进行初始化操作，修改系统配置，解决 Redis 启动时提示的警告信息initContainers:- name: system-initimage: {{init_image}}imagePullPolicy: IfNotPresentcommand:- "sh"- "-c"- "echo 2000 > /proc/sys/net/core/somaxconn && echo never > /sys/kernel/mm/transparent_hugepage/enabled"securityContext:privileged: truerunAsUser: 0volumeMounts:- name: sysmountPath: /sysreadOnly: falsecontainers:- name: redisimage: {{redis_image}}command:- "sh"- "-c"- "redis-server /usr/local/etc/redis/redis.conf"ports:- containerPort: 6379resources:limits:cpu: 4memory: 64Girequests:cpu: 0.1memory: {{memory_request}}livenessProbe:tcpSocket:port: 6379initialDelaySeconds: 300timeoutSeconds: 1periodSeconds: 10successThreshold: 1failureThreshold: 3readinessProbe:tcpSocket:port: 6379initialDelaySeconds: 5timeoutSeconds: 1periodSeconds: 10successThreshold: 1failureThreshold: 3volumeMounts:- name: datamountPath: /data- name: configmountPath:  /usr/local/etc/redis/redis.confsubPath: redis.conf- name: redis-exporterimage: {{redis_exporter_image}}args: ["-redis.password", "{{redis_password}}"]resources:requests:cpu: '0.1'memory: 100Miports:- containerPort: 9121volumes:- name: datapersistentVolumeClaim:claimName: redis-pvc- name: configconfigMap:name: redis-config- name: syshostPath:path: /sys

3 结果

root@qa-gpu026:~# kubectl get sc
NAME                   PROVISIONER                    RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
redis-sc               kubernetes.io/no-provisioner   Delete          WaitForFirstConsumer   false                  16h
sc-mysql-single-node   kubernetes.io/no-provisioner   Delete          WaitForFirstConsumer   false                  16h
swfs-master-sc         kubernetes.io/no-provisioner   Delete          WaitForFirstConsumer   false                  16h
swfs-volume-sc         kubernetes.io/no-provisioner   Delete          WaitForFirstConsumer   false                  16h
zk-local-volume        kubernetes.io/no-provisioner   Delete          WaitForFirstConsumer   false                  16h
root@qa-gpu026:~#

root@qa-gpu026:~# kubectl get pv redis-pv
NAME       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM             STORAGECLASS   REASON   AGE
redis-pv   100Gi      RWO            Retain           Bound    infra/redis-pvc   redis-sc                16h
root@qa-gpu026:~#

pvc

root@qa-gpu026:~# kubectl -n infra get pvc  redis-pvc
NAME        STATUS   VOLUME     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
redis-pvc   Bound    redis-pv   100Gi      RWO            redis-sc       16h
root@qa-gpu026:~#

4 问答

问：本地磁盘可以做sc（StorageClass）么

答：可以，有 local storage class，不过，这个 local 稍微有点特殊，他是 lazy provision 的（volumeBindingMode: WaitForFirstConsumer），需要创建 pod 引用才可以（waiting for first consumer to be created before binding）处于 Bound 状态

问：和nfs，cfs有啥区别

答：local 和 host path 一样，是直接做本地 mount，其他都是网络 fs，走 fuse 挂的

注意：Local Persistent Volume（本地持久化卷），必须使用 nodeAffinity 来跟某个具体的节点绑定