集群scan_扫描k8s集群中的漏洞

Kubei是一个漏洞扫描和CIS Docker基准测试工具，它能够对kubernetes集群进行准确，即时的风险评估。Kubei扫描Kubernetes集群中正在使用的所有图像，包括应用程序Pod和系统Pod的镜像。它不会扫描整个镜像注册表，也不需要与CI / CD进行初步集成。

它是一种可配置的工具，可以定义扫描范围(目标名称空间)，速度和关注的漏洞级别，还提供了图形用户界面。

安装要求：

Kubernetes集群已准备就绪，并且已~/.kube/config为目标集群正确配置了kubeconfig

需要的权限

在集群范围内读取secret。这是用于获取私有镜像仓库的镜像的拉取凭据。
列出集群范围内的Pod。
在集群范围内创建job。创建job将扫描其名称空间中的目标容器

部署

---apiVersion: v1kind: Namespacemetadata:  name: kubei---apiVersion: v1kind: ServiceAccountmetadata:  name: kubei  namespace: kubei---apiVersion: rbac.authorization.k8s.io/v1kind: ClusterRolemetadata:  name: kubeirules:- apiGroups: [""]  resources: ["secrets"]  verbs: ["get"]- apiGroups: [""]  resources: ["pods"]  verbs: ["list"]- apiGroups: ["batch"]  resources: ["jobs"]  verbs: ["create","delete"]---apiVersion: rbac.authorization.k8s.io/v1kind: ClusterRoleBindingmetadata:  name: kubeisubjects:- kind: ServiceAccount  name: kubei  namespace: kubeiroleRef:  apiGroup: rbac.authorization.k8s.io  kind: ClusterRole  name: kubei---apiVersion: v1kind: Servicemetadata:  name: clair  namespace: kubei  labels:    app: clairspec:  type: ClusterIP  ports:  - port: 6060    protocol: TCP    name: apiserver  - port: 6061    protocol: TCP    name: health  selector:    app: clair---apiVersion: apps/v1kind: Deploymentmetadata:  name: clair  namespace: kubei  labels:    app: clairspec:  replicas: 1  selector:    matchLabels:      app: clair  template:    metadata:      labels:        app: clair        kubeiShouldScan: "false"    spec:      initContainers:      - name: check-db-ready        image: postgres:12-alpine        command: ['sh', '-c',  'until pg_isready -h postgres -p 5432;  do echo waiting for database; sleep 2; done;']      containers:      - name: clair        image: gcr.io/portshift-release/clair/clair-local-scan        imagePullPolicy: Always        ports:        - containerPort: 6060        - containerPort: 6061        resources:          limits:            cpu: 2000m            memory: 6G          requests:            cpu: 700m            memory: 3G# uncomment the following lines in case Kubei is running behind proxy. update PROXY_IP & PROXY_PORT according to your proxy settings#        env:#        - name: HTTPS_PROXY#          value: "{ PROXY_IP }:{ PROXY_PORT }"#        - name: HTTP_PROXY#          value: "{ PROXY_IP }:{ PROXY_PORT }"#        - name: NO_PROXY#          value: "postgres"---apiVersion: v1kind: Servicemetadata:  labels:    app: postgres  name: postgres  namespace: kubeispec:  type: ClusterIP  ports:    - name: db      port: 5432      protocol: TCP  selector:    app: postgres---apiVersion: apps/v1kind: Deploymentmetadata:  name: clair-postgres  namespace: kubei  labels:    app: postgresspec:  replicas: 1  selector:    matchLabels:      app: postgres  template:    metadata:      labels:        app: postgres        kubeiShouldScan: "false"    spec:      containers:      - name: clair-db          image: gcr.io/portshift-release/clair/clair-db        imagePullPolicy: Always        ports:        - containerPort: 5432        resources:          limits:            cpu: 1500m            memory: 1G          requests:            cpu: 500m            memory: 400Mi---apiVersion: v1kind: Servicemetadata:  namespace: kubei  name: kubei  labels:    app: kubeispec:  type: ClusterIP  ports:  - port: 8080    protocol: TCP    name: http-webapp  - port: 8081    protocol: TCP    name: http-klar-result  selector:    app: kubei---apiVersion: apps/v1kind: Deploymentmetadata:  name: kubei  namespace: kubei  labels:    app: kubeispec:  replicas: 1  selector:    matchLabels:      app: kubei  template:    metadata:      labels:        app: kubei        kubeiShouldScan: "false"    spec:      serviceAccountName: kubei      initContainers:      - name: init-clairsvc        image: yauritux/busybox-curl        args:          - /bin/sh          - -c          - >            set -x;            while [ $(curl -sw '%{http_code}' "http://clair.kubei:6060/v1/namespaces" -o /dev/null) -ne 200 ]; do              echo "waiting for clair to be ready";              sleep 15;            done      containers:      - name: kubei        image: gcr.io/development-infra-208909/kubei:1.0.8        imagePullPolicy: Always        env:        - name: "KLAR_IMAGE_NAME"          value: "gcr.io/development-infra-208909/klar:1.0.7"        - name: "DOCKLE_IMAGE_NAME"          value: "gcr.io/development-infra-208909/dockle:1.0.0"        - name: "MAX_PARALLELISM" # max number of scans that will run simultaneously. defaults to 10          value: "10"        - name: "TARGET_NAMESPACE" # empty = scan all namespaces          value: ""        - name: "SEVERITY_THRESHOLD" # minimum level of vulnerability to report. defaults to MEDIUM          value: "MEDIUM"        - name: "IGNORE_NAMESPACES" # a list of namespaces to ignore. defaults no namespace to ignore          value: "istio-system,kube-system"        - name: "DELETE_JOB_POLICY"          value: "Successful"        - name: "SCANNER_SERVICE_ACCOUNT" # Defaults to 'default' service account          value: ""        - name: "REGISTRY_INSECURE" # Allow scanner to access insecure registries (HTTP only). Default is `false`.          value: "false"# uncomment the following lines in case Kubei is running behind proxy. update PROXY_IP & PROXY_PORT according to your proxy settings#        - name: SCANNER_HTTPS_PROXY#          value: "{ PROXY_IP }:{ PROXY_PORT }"#        - name: SCANNER_HTTP_PROXY#          value: "{ PROXY_IP }:{ PROXY_PORT }"        ports:        - containerPort: 8080        - containerPort: 8081        resources:          limits:            cpu: 100m            memory: 100Mi          requests:            cpu: 10m            memory: 20Mi

这里有部分镜像因为特殊原因无法拉取，这里我已经下载并上传到dockerhub,对应如下，替换即可。

gcr.io/portshift-release/clair/clair-local-scan --->misterli/clair-local-scan:v2.1.4.ubi.2

gcr.io/portshift-release/clair/clair-db ---> misterli/clair-db:12.2.ubi.2

gcr.io/development-infra-208909/kubei:1.0.8 ---> misterli/kubei:1.0.8

gcr.io/development-infra-208909/klar:1.0.7 ---> misterli/development-infra-208909-klar:1.0.7

gcr.io/development-infra-208909/dockle:1.0.0 ---> misterli/evelopment-infra-208909-dockle:1.0.0

配置说明

1、设置扫描范围，设置变量IGNORE_NAMESPACES以忽略特定的名称空间。设置变量TARGET_NAMESPACE为扫描特定的名称空间，或保留为空以扫描所有名称空间

2、设置扫描速度，变量MAX_PARALLELISM设置最大同时扫描数量

3、设置严重性级别阈值，SEVERITY_THRESHOLD将报告严重性级别高于或等于阈值的漏洞。支持的水平Unknown，Negligible，Low，Medium，High，Critical，Defcon1。默认值为Medium

4、设置删除作业策略,设置变量DELETE_JOB_POLICY以定义是否删除已完成的扫描仪作业。支持的值为：

All -所有作业将被删除。
Successful -仅成功的作业将被删除(默认)。
Never -作业将永远不会被删除

5、禁用CIS Docker基准测试,将变量SHOULD_SCAN_DOCKERFILE设置为false

6、设置扫描仪服务帐户。将变量SCANNER_SERVICE_ACCOUNT设置为扫描仪作业要使用的服务帐户名称。默认为default服务帐户

7、扫描不安全的注册表。设置变量REGISTRY_INSECURE，以允许扫描程序访问不安全的注册表(仅HTTP)。默认值为false。

执行扫描

我们部署一个ingress用来查看dashboard

apiVersion: traefik.containo.us/v1alpha1kind: IngressRoutemetadata:  name: kubei  namespace: kubeispec:  entryPoints:    - web  routes:  - match: Host(`kubei.lishuai.fun`)    kind: Rule    services:    - name: kubei      port: 8080    middlewares:  # 使用http重定向搭配https的中间件    - name: redirect-https      namespace: default---apiVersion: traefik.containo.us/v1alpha1kind: IngressRoutemetadata:  name: kubei-tls  namespace: kubeispec:  entryPoints:    - websecure  routes:  - match: Host(`kubei.lishuai.fun`)    kind: Rule    services:    - name: kubei      port: 8080  tls:    certResolver: myresolver

查看dashboard 如下：

我们点击右上角的go来执行扫描，点击后集群会为每个pod创建一个用来执行扫描的job。

[root@master-01 deploy]# kubectl get job --all-namespaces  -w NAMESPACE              NAME                                                            COMPLETIONS   DURATION   AGEkubernetes-dashboard   scanner-dashboard-69f24894-91cc-4cf4-9dc8-d5c02dc7b7db          0/1           21m        21mlonghorn-system        scanner-csi-attacher-31757dfc-f223-4ecc-878f-cd2ac2d41949       1/1           7s         17smonitoring             scanner-configmap-reload-230805e3-b3c1-4c00-ac35-4e28b4c17c58   0/1           17s        17smonitoring             scanner-prometheus-d6ca5856-424f-4019-8812-07f8cabae041         0/1           10s        10sversion-checker        scanner-version-checker-93b0f404-41cd-4088-b082-8622bd25f200    0/1                      0sversion-checker        scanner-version-checker-93b0f404-41cd-4088-b082-8622bd25f200    0/1           0s         0smonitoring             scanner-prometheus-d6ca5856-424f-4019-8812-07f8cabae041         0/1           13s        13s

稍等再查看dashboard就会出现扫描结果