OpenPose(一)：根据关键点生成置信图(Confidence Map)

最近读了Openpose论文，关于OpenPose的相关资料，博主整理后，留下了一个传送门，将网上写的比较好的博文和github地址记录在传送门里。

文章目录

一、置信图(Confidence Map)生成原理
二、两种生成置信图的方法
- 2.1 循环遍历
- 2.2 用矩阵替代循环，加快速度

一、置信图(Confidence Map)生成原理

摘自OpenPose论文中的段落：
我们首先为每一个个体 kkk 产生个体的置信图 Sj,k∗S^{*}_{j,k}Sj,k∗, 用 xj,k∈R2x_{j,k} \in R^2xj,k∈R2 表示个体 kkk 的关键点部位 jjj 的真是位置，则 Sj,k∗S^{*}_{j,k}Sj,k∗ 中位置 p∈R2p\in R^2p∈R2 的值定义为：

其中 σ\sigmaσ 控制峰值的传播。网络预测的置信图是通过一个最大操作将各个置信图聚合在一起：

是的，你没有看错，这就是一个高斯分布！标准的高斯分布公式为

xxx, yyy 代表像素的模板坐标，模板中心位置为原点）
xxx, yyy 直接看成图像中任意点 (x,y)(x, y)(x,y)到目标点（关键点）的在xxx或者yyy方向上的距离。

二、两种生成置信图的方法

摘自一个COCO数据中，某一图像的关键点标注。
关于COCO数据集的理解，见我之前整理的理解COCO数据集中的关键点标注

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
*  * *** *  * *  *
*  *  *   **  *  *
****  *   **  *  *
*  *  *   **  *  *
*  * **  *  * ****  @File     :Openpose/showGussMapForPoint.py
@Date     :2020/12/7 下午2:07
@Require  :opencv、matplotlib、numpy
@Author   :hjxu2016， https://blog.csdn.net/hjxu2016/
@Funtion  :二维图像中的 Guss热值图
"""import cv2
import numpy as np
import matplotlib.pyplot as plt
from matplotlib import cm as cbgr_image = cv2.imread("./000000000785.jpg")
rgb_image = cv2.cvtColor(bgr_image, cv2.COLOR_BGR2RGB)points = [[367,  81,  2],[378, 118,   2],[358, 129,   2],[341, 159,   2],[309, 178,   2],[399, 108,   2],[433, 142,   2],[449, 165,   2],[393, 214,   2],[367, 273,   2],[396, 341,   2],[424, 203,   2],[429, 294,  2],[466, 362,   2],[360,  75,   2],[374,  73,   2],[356,  81,   2],[386,  78,   2]]

2.1 循环遍历

循环已成惯性思维，第一个想到的就是循环遍历

import math
def put_heatmap(shape, center, sigma):center_x, center_y, _ = centerheatmap = np.zeros((shape[0], shape[1]))height, width, _ = shapeth = 4.6052delta = math.sqrt(th * 2)x0 = int(max(0, center_x - delta * sigma))y0 = int(max(0, center_y - delta * sigma))x1 = int(min(width, center_x + delta * sigma))y1 = int(min(height, center_y + delta * sigma))for y in range(y0, y1):for x in range(x0, x1):d = (x - center_x) ** 2 + (y - center_y) ** 2  # 高斯！exp = d / 2.0 / sigma / sigma  # 高斯！if exp > th:continueheatmap[y][x] = max(heatmap[y][x], math.exp(-exp))heatmap[y][x] = min(heatmap[y][x], 1.0)return heatmapfor point in points:guss_heatmap = put_heatmap(rgb_image.shape, point, sigma = 7)fg, (ax1, ax2, ax3) = plt.subplots(1, 3, figsize=(40, 8))ax1.imshow(rgb_image)ax2.imshow(guss_heatmap, cmap=c.jet)guss_heatmap = np.array(guss_heatmap)guss_heatmap[guss_heatmap < 0] = 0guss_heatmap = cv2.normalize(guss_heatmap, guss_heatmap, 0, 255, cv2.NORM_MINMAX)guss_heatmap = np.uint8(guss_heatmap)jetmap = cv2.applyColorMap(255-guss_heatmap, cv2.COLORMAP_JET)alpha = 0.5out = cv2.addWeighted(rgb_image, alpha, jetmap, 1 - alpha, 0, jetmap)ax3.imshow(out, cmap=c.jet)

2.2 用矩阵替代循环，加快速度

循环比较太慢了，矩阵则是一个更好的思路。
设置两个xxx和yyy矩阵，分别0、1、...−w0、1、...-w0、1、...−w
那么图像所有点(x,y)(x, y)(x,y)到目标点 ppp的在xxx或者yyy方向上的距离和就可以用矩阵运算。

def generate_gaussian_heatmap(shape, joint, sigma):x, y,_ = joint grid_x = np.tile(np.arange(shape[1]), (shape[0], 1))grid_y = np.tile(np.arange(shape[0]), (shape[1], 1)).transpose()grid_distance = (grid_x - x) ** 2 + (grid_y - y) ** 2gaussian_heatmap = np.exp(-0.5 * grid_distance / sigma ** 2)# 产生的就是一整张图的gaussian分布，只不过里中心点远的点非常非常小return gaussian_heatmap
for point in points:guss_heatmap = generate_gaussian_heatmap(rgb_image.shape, point, sigma = 7)fg, (ax1, ax2, ax3) = plt.subplots(1, 3, figsize=(40, 8))ax1.imshow(rgb_image)ax2.imshow(guss_heatmap, cmap=c.jet)guss_heatmap[guss_heatmap < 0] = 0guss_heatmap = cv2.normalize(guss_heatmap, guss_heatmap, 0, 255, cv2.NORM_MINMAX)guss_heatmap = np.uint8(guss_heatmap)jetmap = cv2.applyColorMap(255-guss_heatmap, cv2.COLORMAP_JET)alpha = 0.5out = cv2.addWeighted(rgb_image, alpha, jetmap, 1 - alpha, 0, jetmap)ax3.imshow(out, cmap=c.jet)

显示一下结果