keras h5—darknet weights模型相互转换
2024-05-07 14:49:09
目录
- 一.版本介绍
- 二.保存方式
- 三.转换过程
- 四.参考
一.版本介绍
keras 2.1.5
tensorflow 1.13.1
这里以一个yolov3_tiny模型为案例。
二.保存方式
首先准备自己的h5模型；这里要注意h5模型的保存方式，不同保存方式，对应不同加载方式，如下：
> 训练: model.save()
> 加载: model.load_model()
> 训练: model.save_weights()
> 加载:
> model = LeNet()
> model.load_weights(weight_path)
三.转换过程
注：这里相互转换都需要提供模型的cfg文件。
- ① darknet to keras
import argparse
import configparser
import io
import os
from collections import defaultdictimport numpy as np
from keras import backend as K
from keras.layers import (Conv2D, Input, ZeroPadding2D, Add,UpSampling2D, MaxPooling2D, Concatenate)
from keras.layers.advanced_activations import LeakyReLU
from keras.layers.normalization import BatchNormalization
from keras.models import Model
from keras.regularizers import l2
from keras.utils.vis_utils import plot_model as plotparser = argparse.ArgumentParser(description='Darknet To Keras Converter.')
parser.add_argument('config_path', help='Path to Darknet cfg file.')
parser.add_argument('weights_path', help='Path to Darknet weights file.')
parser.add_argument('output_path', help='Path to output Keras model file.')
parser.add_argument('-p','--plot_model',help='Plot generated Keras model and save as image.',action='store_true')
parser.add_argument('-w','--weights_only',help='Save as Keras weights file instead of model file.',action='store_true')def unique_config_sections(config_file):"""Convert all config sections to have unique names.Adds unique suffixes to config sections for compability with configparser."""section_counters = defaultdict(int)output_stream = io.StringIO()with open(config_file) as fin:for line in fin:if line.startswith('['):section = line.strip().strip('[]')_section = section + '_' + str(section_counters[section])section_counters[section] += 1line = line.replace(section, _section)output_stream.write(line)output_stream.seek(0)return output_streamdef _main(args):config_path = os.path.expanduser(args.config_path)weights_path = os.path.expanduser(args.weights_path)assert config_path.endswith('.cfg'), '{} is not a .cfg file'.format(config_path)assert weights_path.endswith('.weights'), '{} is not a .weights file'.format(weights_path)output_path = os.path.expanduser(args.output_path)assert output_path.endswith('.h5'), 'output path {} is not a .h5 file'.format(output_path)output_root = os.path.splitext(output_path)[0]# Load weights and config.print('Loading weights.')weights_file = open(weights_path, 'rb')major, minor, revision = np.ndarray(shape=(3, ), dtype='int32', buffer=weights_file.read(12))if (major*10+minor)>=2 and major<1000 and minor<1000:seen = np.ndarray(shape=(1,), dtype='int64', buffer=weights_file.read(8))else:seen = np.ndarray(shape=(1,), dtype='int32', buffer=weights_file.read(4))print('Weights Header: ', major, minor, revision, seen)print('Parsing Darknet config.')unique_config_file = unique_config_sections(config_path)cfg_parser = configparser.ConfigParser()cfg_parser.read_file(unique_config_file)print('Creating Keras model.')input_layer = Input(shape=(None, None, 3))prev_layer = input_layerall_layers = []weight_decay = float(cfg_parser['net_0']['decay']) if 'net_0' in cfg_parser.sections() else 5e-4count = 0out_index = []for section in cfg_parser.sections():print('Parsing section {}'.format(section))if section.startswith('convolutional'):filters = int(cfg_parser[section]['filters'])size = int(cfg_parser[section]['size'])stride = int(cfg_parser[section]['stride'])pad = int(cfg_parser[section]['pad'])activation = cfg_parser[section]['activation']batch_normalize = 'batch_normalize' in cfg_parser[section]padding = 'same' if pad == 1 and stride == 1 else 'valid'# Setting weights.# Darknet serializes convolutional weights as:# [bias/beta, [gamma, mean, variance], conv_weights]prev_layer_shape = K.int_shape(prev_layer)weights_shape = (size, size, prev_layer_shape[-1], filters)darknet_w_shape = (filters, weights_shape[2], size, size)weights_size = np.product(weights_shape)print('conv2d', 'bn'if batch_normalize else ' ', activation, weights_shape)conv_bias = np.ndarray(shape=(filters, ),dtype='float32',buffer=weights_file.read(filters * 4))count += filtersif batch_normalize:bn_weights = np.ndarray(shape=(3, filters),dtype='float32',buffer=weights_file.read(filters * 12))count += 3 * filtersbn_weight_list = [bn_weights[0], # scale gammaconv_bias, # shift betabn_weights[1], # running meanbn_weights[2] # running var]conv_weights = np.ndarray(shape=darknet_w_shape,dtype='float32',buffer=weights_file.read(weights_size * 4))count += weights_size# DarkNet conv_weights are serialized Caffe-style:# (out_dim, in_dim, height, width)# We would like to set these to Tensorflow order:# (height, width, in_dim, out_dim)conv_weights = np.transpose(conv_weights, [2, 3, 1, 0])conv_weights = [conv_weights] if batch_normalize else [conv_weights, conv_bias]# Handle activation.act_fn = Noneif activation == 'leaky':pass # Add advanced activation later.elif activation != 'linear':raise ValueError('Unknown activation function `{}` in section {}'.format(activation, section))# Create Conv2D layerif stride>1:# Darknet uses left and top padding instead of 'same' modeprev_layer = ZeroPadding2D(((1,0),(1,0)))(prev_layer)conv_layer = (Conv2D(filters, (size, size),strides=(stride, stride),kernel_regularizer=l2(weight_decay),use_bias=not batch_normalize,weights=conv_weights,activation=act_fn,padding=padding))(prev_layer)if batch_normalize:conv_layer = (BatchNormalization(weights=bn_weight_list))(conv_layer)prev_layer = conv_layerif activation == 'linear':all_layers.append(prev_layer)elif activation == 'leaky':act_layer = LeakyReLU(alpha=0.1)(prev_layer)prev_layer = act_layerall_layers.append(act_layer)elif section.startswith('route'):ids = [int(i) for i in cfg_parser[section]['layers'].split(',')]layers = [all_layers[i] for i in ids]if len(layers) > 1:print('Concatenating route layers:', layers)concatenate_layer = Concatenate()(layers)all_layers.append(concatenate_layer)prev_layer = concatenate_layerelse:skip_layer = layers[0] # only one layer to routeall_layers.append(skip_layer)prev_layer = skip_layerelif section.startswith('maxpool'):size = int(cfg_parser[section]['size'])stride = int(cfg_parser[section]['stride'])all_layers.append(MaxPooling2D(pool_size=(size, size),strides=(stride, stride),padding='same')(prev_layer))prev_layer = all_layers[-1]elif section.startswith('shortcut'):index = int(cfg_parser[section]['from'])activation = cfg_parser[section]['activation']assert activation == 'linear', 'Only linear activation supported.'all_layers.append(Add()([all_layers[index], prev_layer]))prev_layer = all_layers[-1]elif section.startswith('upsample'):stride = int(cfg_parser[section]['stride'])assert stride == 2, 'Only stride=2 supported.'all_layers.append(UpSampling2D(stride)(prev_layer))prev_layer = all_layers[-1]elif section.startswith('yolo'):out_index.append(len(all_layers)-1)all_layers.append(None)prev_layer = all_layers[-1]elif section.startswith('net'):passelse:raise ValueError('Unsupported section header type: {}'.format(section))# Create and save model.if len(out_index)==0: out_index.append(len(all_layers)-1)model = Model(inputs=input_layer, outputs=[all_layers[i] for i in out_index])print(model.summary())if args.weights_only:model.save_weights('{}'.format(output_path))print('Saved Keras weights to {}'.format(output_path))else:model.save('{}'.format(output_path))print('Saved Keras model to {}'.format(output_path))# Check to see if all weights have been read.remaining_weights = len(weights_file.read()) / 4weights_file.close()print('Read {} of {} from Darknet weights.'.format(count, count +remaining_weights))if remaining_weights > 0:print('Warning: {} unused weights'.format(remaining_weights))if args.plot_model:plot(model, to_file='{}.png'.format(output_root), show_shapes=True)print('Saved model plot to {}.png'.format(output_root))if __name__ == '__main__':_main(parser.parse_args())- ② keras to darknet
根据保存方式判断是否需要model.py
"""model.py"""from functools import wrapsimport numpy as np
import tensorflow as tf
from keras import backend as K
from keras.layers import Conv2D, Add, ZeroPadding2D, UpSampling2D, Concatenate, MaxPooling2D
from keras.layers.advanced_activations import LeakyReLU
from keras.layers.normalization import BatchNormalization
from keras.models import Model
from keras.regularizers import l2from yolo3.utils import compose@wraps(Conv2D)
def DarknetConv2D(*args, **kwargs):"""Wrapper to set Darknet parameters for Convolution2D."""darknet_conv_kwargs = {'kernel_regularizer': l2(5e-4)}darknet_conv_kwargs['padding'] = 'valid' if kwargs.get('strides')==(2,2) else 'same'darknet_conv_kwargs.update(kwargs)return Conv2D(*args, **darknet_conv_kwargs)def DarknetConv2D_BN_Leaky(*args, **kwargs):"""Darknet Convolution2D followed by BatchNormalization and LeakyReLU."""no_bias_kwargs = {'use_bias': False}no_bias_kwargs.update(kwargs)return compose(DarknetConv2D(*args, **no_bias_kwargs),BatchNormalization(),LeakyReLU(alpha=0.1))def resblock_body(x, num_filters, num_blocks):'''A series of resblocks starting with a downsampling Convolution2D'''# Darknet uses left and top padding instead of 'same' modex = ZeroPadding2D(((1,0),(1,0)))(x)x = DarknetConv2D_BN_Leaky(num_filters, (3,3), strides=(2,2))(x)for i in range(num_blocks):y = compose(DarknetConv2D_BN_Leaky(num_filters//2, (1,1)),DarknetConv2D_BN_Leaky(num_filters, (3,3)))(x)x = Add()([x,y])return xdef darknet_body(x):'''Darknent body having 52 Convolution2D layers'''x = DarknetConv2D_BN_Leaky(32, (3,3))(x)x = resblock_body(x, 64, 1)x = resblock_body(x, 128, 2)x = resblock_body(x, 256, 8)x = resblock_body(x, 512, 8)x = resblock_body(x, 1024, 4)return xdef make_last_layers(x, num_filters, out_filters):'''6 Conv2D_BN_Leaky layers followed by a Conv2D_linear layer'''x = compose(DarknetConv2D_BN_Leaky(num_filters, (1,1)),DarknetConv2D_BN_Leaky(num_filters*2, (3,3)),DarknetConv2D_BN_Leaky(num_filters, (1,1)),DarknetConv2D_BN_Leaky(num_filters*2, (3,3)),DarknetConv2D_BN_Leaky(num_filters, (1,1)))(x)y = compose(DarknetConv2D_BN_Leaky(num_filters*2, (3,3)),DarknetConv2D(out_filters, (1,1)))(x)return x, ydef yolo_body(inputs, num_anchors, num_classes):"""Create YOLO_V3 model CNN body in Keras."""darknet = Model(inputs, darknet_body(inputs))x, y1 = make_last_layers(darknet.output, 512, num_anchors*(num_classes+5))x = compose(DarknetConv2D_BN_Leaky(256, (1,1)),UpSampling2D(2))(x)x = Concatenate()([x,darknet.layers[152].output])x, y2 = make_last_layers(x, 256, num_anchors*(num_classes+5))x = compose(DarknetConv2D_BN_Leaky(128, (1,1)),UpSampling2D(2))(x)x = Concatenate()([x,darknet.layers[92].output])x, y3 = make_last_layers(x, 128, num_anchors*(num_classes+5))return Model(inputs, [y1,y2,y3])def tiny_yolo_body(inputs, num_anchors, num_classes):'''Create Tiny YOLO_v3 model CNN body in keras.'''x1 = compose(DarknetConv2D_BN_Leaky(16, (3,3)),MaxPooling2D(pool_size=(2,2), strides=(2,2), padding='same'),DarknetConv2D_BN_Leaky(32, (3,3)),MaxPooling2D(pool_size=(2,2), strides=(2,2), padding='same'),DarknetConv2D_BN_Leaky(64, (3,3)),MaxPooling2D(pool_size=(2,2), strides=(2,2), padding='same'),DarknetConv2D_BN_Leaky(128, (3,3)),MaxPooling2D(pool_size=(2,2), strides=(2,2), padding='same'),DarknetConv2D_BN_Leaky(256, (3,3)))(inputs)x2 = compose(MaxPooling2D(pool_size=(2,2), strides=(2,2), padding='same'),DarknetConv2D_BN_Leaky(512, (3,3)),MaxPooling2D(pool_size=(2,2), strides=(1,1), padding='same'),DarknetConv2D_BN_Leaky(1024, (3,3)),DarknetConv2D_BN_Leaky(256, (1,1)))(x1)y1 = compose(DarknetConv2D_BN_Leaky(512, (3,3)),DarknetConv2D(num_anchors*(num_classes+5), (1,1)))(x2)x2 = compose(DarknetConv2D_BN_Leaky(128, (1,1)),UpSampling2D(2))(x2)y2 = compose(Concatenate(),DarknetConv2D_BN_Leaky(256, (3,3)),DarknetConv2D(num_anchors*(num_classes+5), (1,1)))([x2,x1])return Model(inputs, [y1,y2])def yolo_head(feats, anchors, num_classes, input_shape, calc_loss=False):"""Convert final layer features to bounding box parameters."""num_anchors = len(anchors)# Reshape to batch, height, width, num_anchors, box_params.anchors_tensor = K.reshape(K.constant(anchors), [1, 1, 1, num_anchors, 2])grid_shape = K.shape(feats)[1:3] # height, widthgrid_y = K.tile(K.reshape(K.arange(0, stop=grid_shape[0]), [-1, 1, 1, 1]),[1, grid_shape[1], 1, 1])grid_x = K.tile(K.reshape(K.arange(0, stop=grid_shape[1]), [1, -1, 1, 1]),[grid_shape[0], 1, 1, 1])grid = K.concatenate([grid_x, grid_y])grid = K.cast(grid, K.dtype(feats))feats = K.reshape(feats, [-1, grid_shape[0], grid_shape[1], num_anchors, num_classes + 5])# Adjust preditions to each spatial grid point and anchor size.box_xy = (K.sigmoid(feats[..., :2]) + grid) / K.cast(grid_shape[::-1], K.dtype(feats))box_wh = K.exp(feats[..., 2:4]) * anchors_tensor / K.cast(input_shape[::-1], K.dtype(feats))box_confidence = K.sigmoid(feats[..., 4:5])box_class_probs = K.sigmoid(feats[..., 5:])if calc_loss == True:return grid, feats, box_xy, box_whreturn box_xy, box_wh, box_confidence, box_class_probsdef yolo_correct_boxes(box_xy, box_wh, input_shape, image_shape):'''Get corrected boxes'''box_yx = box_xy[..., ::-1]box_hw = box_wh[..., ::-1]input_shape = K.cast(input_shape, K.dtype(box_yx))image_shape = K.cast(image_shape, K.dtype(box_yx))new_shape = K.round(image_shape * K.min(input_shape/image_shape))offset = (input_shape-new_shape)/2./input_shapescale = input_shape/new_shapebox_yx = (box_yx - offset) * scalebox_hw *= scalebox_mins = box_yx - (box_hw / 2.)box_maxes = box_yx + (box_hw / 2.)boxes = K.concatenate([box_mins[..., 0:1], # y_minbox_mins[..., 1:2], # x_minbox_maxes[..., 0:1], # y_maxbox_maxes[..., 1:2] # x_max])# Scale boxes back to original image shape.boxes *= K.concatenate([image_shape, image_shape])return boxesdef yolo_boxes_and_scores(feats, anchors, num_classes, input_shape, image_shape):'''Process Conv layer output'''box_xy, box_wh, box_confidence, box_class_probs = yolo_head(feats,anchors, num_classes, input_shape)boxes = yolo_correct_boxes(box_xy, box_wh, input_shape, image_shape)boxes = K.reshape(boxes, [-1, 4])box_scores = box_confidence * box_class_probsbox_scores = K.reshape(box_scores, [-1, num_classes])return boxes, box_scoresdef yolo_eval(yolo_outputs,anchors,num_classes,image_shape,max_boxes=20,score_threshold=.6,iou_threshold=.5):"""Evaluate YOLO model on given input and return filtered boxes."""num_layers = len(yolo_outputs)anchor_mask = [[6,7,8], [3,4,5], [0,1,2]] if num_layers==3 else [[3,4,5], [1,2,3]] # default settinginput_shape = K.shape(yolo_outputs[0])[1:3] * 32boxes = []box_scores = []for l in range(num_layers):_boxes, _box_scores = yolo_boxes_and_scores(yolo_outputs[l],anchors[anchor_mask[l]], num_classes, input_shape, image_shape)boxes.append(_boxes)box_scores.append(_box_scores)boxes = K.concatenate(boxes, axis=0)box_scores = K.concatenate(box_scores, axis=0)mask = box_scores >= score_thresholdmax_boxes_tensor = K.constant(max_boxes, dtype='int32')boxes_ = []scores_ = []classes_ = []for c in range(num_classes):# TODO: use keras backend instead of tf.class_boxes = tf.boolean_mask(boxes, mask[:, c])class_box_scores = tf.boolean_mask(box_scores[:, c], mask[:, c])nms_index = tf.image.non_max_suppression(class_boxes, class_box_scores, max_boxes_tensor, iou_threshold=iou_threshold)class_boxes = K.gather(class_boxes, nms_index)class_box_scores = K.gather(class_box_scores, nms_index)classes = K.ones_like(class_box_scores, 'int32') * cboxes_.append(class_boxes)scores_.append(class_box_scores)classes_.append(classes)boxes_ = K.concatenate(boxes_, axis=0)scores_ = K.concatenate(scores_, axis=0)classes_ = K.concatenate(classes_, axis=0)return boxes_, scores_, classes_def preprocess_true_boxes(true_boxes, input_shape, anchors, num_classes):'''Preprocess true boxes to training input formatParameters----------true_boxes: array, shape=(m, T, 5)Absolute x_min, y_min, x_max, y_max, class_id relative to input_shape.input_shape: array-like, hw, multiples of 32anchors: array, shape=(N, 2), whnum_classes: integerReturns-------y_true: list of array, shape like yolo_outputs, xywh are reletive value'''assert (true_boxes[..., 4]<num_classes).all(), 'class id must be less than num_classes'num_layers = len(anchors)//3 # default settinganchor_mask = [[6,7,8], [3,4,5], [0,1,2]] if num_layers==3 else [[3,4,5], [1,2,3]]true_boxes = np.array(true_boxes, dtype='float32')input_shape = np.array(input_shape, dtype='int32')boxes_xy = (true_boxes[..., 0:2] + true_boxes[..., 2:4]) // 2boxes_wh = true_boxes[..., 2:4] - true_boxes[..., 0:2]true_boxes[..., 0:2] = boxes_xy/input_shape[::-1]true_boxes[..., 2:4] = boxes_wh/input_shape[::-1]m = true_boxes.shape[0]grid_shapes = [input_shape//{0:32, 1:16, 2:8}[l] for l in range(num_layers)]y_true = [np.zeros((m,grid_shapes[l][0],grid_shapes[l][1],len(anchor_mask[l]),5+num_classes),dtype='float32') for l in range(num_layers)]# Expand dim to apply broadcasting.anchors = np.expand_dims(anchors, 0)anchor_maxes = anchors / 2.anchor_mins = -anchor_maxesvalid_mask = boxes_wh[..., 0]>0for b in range(m):# Discard zero rows.wh = boxes_wh[b, valid_mask[b]]if len(wh)==0: continue# Expand dim to apply broadcasting.wh = np.expand_dims(wh, -2)box_maxes = wh / 2.box_mins = -box_maxesintersect_mins = np.maximum(box_mins, anchor_mins)intersect_maxes = np.minimum(box_maxes, anchor_maxes)intersect_wh = np.maximum(intersect_maxes - intersect_mins, 0.)intersect_area = intersect_wh[..., 0] * intersect_wh[..., 1]box_area = wh[..., 0] * wh[..., 1]anchor_area = anchors[..., 0] * anchors[..., 1]iou = intersect_area / (box_area + anchor_area - intersect_area)# Find best anchor for each true boxbest_anchor = np.argmax(iou, axis=-1)for t, n in enumerate(best_anchor):for l in range(num_layers):if n in anchor_mask[l]:i = np.floor(true_boxes[b,t,0]*grid_shapes[l][1]).astype('int32')j = np.floor(true_boxes[b,t,1]*grid_shapes[l][0]).astype('int32')k = anchor_mask[l].index(n)c = true_boxes[b,t, 4].astype('int32')y_true[l][b, j, i, k, 0:4] = true_boxes[b,t, 0:4]y_true[l][b, j, i, k, 4] = 1y_true[l][b, j, i, k, 5+c] = 1return y_truedef box_iou(b1, b2):'''Return iou tensorParameters----------b1: tensor, shape=(i1,...,iN, 4), xywhb2: tensor, shape=(j, 4), xywhReturns-------iou: tensor, shape=(i1,...,iN, j)'''# Expand dim to apply broadcasting.b1 = K.expand_dims(b1, -2)b1_xy = b1[..., :2]b1_wh = b1[..., 2:4]b1_wh_half = b1_wh/2.b1_mins = b1_xy - b1_wh_halfb1_maxes = b1_xy + b1_wh_half# Expand dim to apply broadcasting.b2 = K.expand_dims(b2, 0)b2_xy = b2[..., :2]b2_wh = b2[..., 2:4]b2_wh_half = b2_wh/2.b2_mins = b2_xy - b2_wh_halfb2_maxes = b2_xy + b2_wh_halfintersect_mins = K.maximum(b1_mins, b2_mins)intersect_maxes = K.minimum(b1_maxes, b2_maxes)intersect_wh = K.maximum(intersect_maxes - intersect_mins, 0.)intersect_area = intersect_wh[..., 0] * intersect_wh[..., 1]b1_area = b1_wh[..., 0] * b1_wh[..., 1]b2_area = b2_wh[..., 0] * b2_wh[..., 1]iou = intersect_area / (b1_area + b2_area - intersect_area)return ioudef yolo_loss(args, anchors, num_classes, ignore_thresh=.5, print_loss=False):'''Return yolo_loss tensorParameters----------yolo_outputs: list of tensor, the output of yolo_body or tiny_yolo_bodyy_true: list of array, the output of preprocess_true_boxesanchors: array, shape=(N, 2), whnum_classes: integerignore_thresh: float, the iou threshold whether to ignore object confidence lossReturns-------loss: tensor, shape=(1,)'''num_layers = len(anchors)//3 # default settingyolo_outputs = args[:num_layers]y_true = args[num_layers:]anchor_mask = [[6,7,8], [3,4,5], [0,1,2]] if num_layers==3 else [[3,4,5], [1,2,3]]input_shape = K.cast(K.shape(yolo_outputs[0])[1:3] * 32, K.dtype(y_true[0]))grid_shapes = [K.cast(K.shape(yolo_outputs[l])[1:3], K.dtype(y_true[0])) for l in range(num_layers)]loss = 0m = K.shape(yolo_outputs[0])[0] # batch size, tensormf = K.cast(m, K.dtype(yolo_outputs[0]))for l in range(num_layers):object_mask = y_true[l][..., 4:5]true_class_probs = y_true[l][..., 5:]grid, raw_pred, pred_xy, pred_wh = yolo_head(yolo_outputs[l],anchors[anchor_mask[l]], num_classes, input_shape, calc_loss=True)pred_box = K.concatenate([pred_xy, pred_wh])# Darknet raw box to calculate loss.raw_true_xy = y_true[l][..., :2]*grid_shapes[l][::-1] - gridraw_true_wh = K.log(y_true[l][..., 2:4] / anchors[anchor_mask[l]] * input_shape[::-1])raw_true_wh = K.switch(object_mask, raw_true_wh, K.zeros_like(raw_true_wh)) # avoid log(0)=-infbox_loss_scale = 2 - y_true[l][...,2:3]*y_true[l][...,3:4]# Find ignore mask, iterate over each of batch.ignore_mask = tf.TensorArray(K.dtype(y_true[0]), size=1, dynamic_size=True)object_mask_bool = K.cast(object_mask, 'bool')def loop_body(b, ignore_mask):true_box = tf.boolean_mask(y_true[l][b,...,0:4], object_mask_bool[b,...,0])iou = box_iou(pred_box[b], true_box)best_iou = K.max(iou, axis=-1)ignore_mask = ignore_mask.write(b, K.cast(best_iou<ignore_thresh, K.dtype(true_box)))return b+1, ignore_mask_, ignore_mask = K.control_flow_ops.while_loop(lambda b,*args: b<m, loop_body, [0, ignore_mask])ignore_mask = ignore_mask.stack()ignore_mask = K.expand_dims(ignore_mask, -1)# K.binary_crossentropy is helpful to avoid exp overflow.xy_loss = object_mask * box_loss_scale * K.binary_crossentropy(raw_true_xy, raw_pred[...,0:2], from_logits=True)wh_loss = object_mask * box_loss_scale * 0.5 * K.square(raw_true_wh-raw_pred[...,2:4])confidence_loss = object_mask * K.binary_crossentropy(object_mask, raw_pred[...,4:5], from_logits=True)+ \(1-object_mask) * K.binary_crossentropy(object_mask, raw_pred[...,4:5], from_logits=True) * ignore_maskclass_loss = object_mask * K.binary_crossentropy(true_class_probs, raw_pred[...,5:], from_logits=True)xy_loss = K.sum(xy_loss) / mfwh_loss = K.sum(wh_loss) / mfconfidence_loss = K.sum(confidence_loss) / mfclass_loss = K.sum(class_loss) / mfloss += xy_loss + wh_loss + confidence_loss + class_lossif print_loss:loss = tf.Print(loss, [loss, xy_loss, wh_loss, confidence_loss, class_loss, K.sum(ignore_mask)], message='loss: ')return loss转换脚本：
# -*- coding: utf-8 -*-
import argparse
import numpy
import numpy as np
from keras.layers import Input
from keras.models import load_model
from keras import backend as K
from yolo3.model import tiny_yolo_bodydef parser():parser = argparse.ArgumentParser(description="Darknet\'s yolov3.cfg and yolov3.weights converted into Keras\'s yolov3.h5!")parser.add_argument('-cfg_path', default='yolov3.cfg',help='yolov3.cfg')parser.add_argument('-h5_path', default='mask.h5',help='A.h5')parser.add_argument('-output_path',default='model_data/VVVVVV.weights',help='yolov3.weights')return parser.parse_args()class WeightSaver(object):def __init__(self, h5_path, output_path):try:self.model = load_model(h5_path, compile=False)except:self.model = tiny_yolo_body(Input(shape=(None, None,3)), 6 // 2, 2)self.model.load_weights(h5_path)self.layers = {weight.name: weight for weight in self.model.weights}self.sess = K.get_session()self.fhandle = open(output_path, 'wb')self._write_head()def _write_head(self):numpy_data = numpy.ndarray(shape=(3,),dtype='int32',buffer=np.array([0, 2, 0], dtype='int32'))self.save(numpy_data)numpy_data = numpy.ndarray(shape=(1,),dtype='int64',buffer=np.array([320000], dtype='int64'))self.save(numpy_data)def get_bn_layername(self, num):layer_name = f'batch_normalization_{num}'bias = self.layers[f'{layer_name}/beta:0']scale = self.layers[f'{layer_name}/gamma:0']mean = self.layers[f'{layer_name}/moving_mean:0']var = self.layers[f'{layer_name}/moving_variance:0']bias_np = self.get_numpy(bias)scale_np = self.get_numpy(scale)mean_np = self.get_numpy(mean)var_np = self.get_numpy(var)return bias_np, scale_np, mean_np, var_npdef get_convbias_layername(self, num):layer_name = f'conv2d_{num}'bias = self.layers[f'{layer_name}/bias:0']bias_np = self.get_numpy(bias)return bias_npdef get_conv_layername(self, num):layer_name = f'conv2d_{num}'conv = self.layers[f'{layer_name}/kernel:0']conv_np = self.get_numpy(conv)return conv_npdef get_numpy(self, layer_name):numpy_data = self.sess.run(layer_name)return numpy_datadef save(self, numpy_data):bytes_data = numpy_data.tobytes()self.fhandle.write(bytes_data)self.fhandle.flush()def close(self):self.fhandle.close()class KerasParser(object):def __init__(self, cfg_path, h5_path, output_path):self.block_gen = self._get_block(cfg_path)self.weights_saver = WeightSaver(h5_path, output_path)self.count_conv = 0self.count_bn = 0def _get_block(self, cfg_path):block = {}with open(cfg_path, 'r', encoding='utf-8') as fr:for line in fr:line = line.strip()if '[' in line and ']' in line:if block:yield blockblock = {}block['type'] = line.strip(' []')elif not line or '#' in line:continueelse:key, val = line.strip().replace(' ', '').split('=')key, val = key.strip(), val.strip()block[key] = valyield blockdef close(self):self.weights_saver.close()def conv(self, block):self.count_conv += 1batch_normalize = 'batch_normalize' in block# 如果bn存在，则先处理bn，顺序为bias，scale，mean，varif batch_normalize:bias, scale, mean, var = self.bn()self.weights_saver.save(bias)scale = scale.reshape(1, -1)mean = mean.reshape(1, -1)var = var.reshape(1, -1)remain = np.concatenate([scale, mean, var], axis=0)self.weights_saver.save(remain)# 否则，先处理biaseelse:conv_bias = self.weights_saver.get_convbias_layername(self.count_conv)self.weights_saver.save(conv_bias)# 接着处理weightsconv_weights = self.weights_saver.get_conv_layername(self.count_conv)# 需要将(height, width, in_dim, out_dim)转换成(out_dim, in_dim, height, width)conv_weights = np.transpose(conv_weights, [3, 2, 0, 1])self.weights_saver.save(conv_weights)def bn(self):self.count_bn += 1bias, scale, mean, var = self.weights_saver.get_bn_layername(self.count_bn)return bias, scale, mean, vardef main():args = parser()keras_loader = KerasParser(args.cfg_path, args.h5_path, args.output_path)for block in keras_loader.block_gen:print(block)if 'convolutional' in block['type']:keras_loader.conv(block)keras_loader.close()if __name__ == "__main__":main()
#使用方式
# python h5_to_weights.py -cfg_path ./model_data/yolov3.cfg -h5_path ./model_data/A.h5 -output_path ./model_data/A.weights
四.参考
https://github.com/qqwweee/keras-yolo3
keras h5—darknet weights模型相互转换相关推荐
- Pytorch版本YOLOv3模型转Darknet weights模型然后转caffemodel再转wk模型在nnie上面推理
Pytorch版本YOLOv3模型转darknet weights模型然后转caffemodel再转wk模型在nnie上面推理 文章目录 Pytorch版本YOLOv3模型转darknet weigh ...
- YOLO：将yolo的.weights文件转换为keras、tensorflow等模型所需的.h5文件的图文教程
YOLO:将yolo的.weights文件转换为keras.tensorflow等模型所需的.h5文件的图文教程 目录 解决问题 操作过程 结果输出 解决问题 将yolo的.weights文件转换为k ...
- 导入keras训练好的模型报错解决OSError: Unable to open file (unable to open file: name = ‘model_3.h5’）
导入keras训练好的模型报错解决) 开心撒花 切入正题 开心撒花 第一次写博客,想分享记录一下自己写程序.跑程序过程中的问题和收获.哈哈哈哈 开心. Markdown编辑器 第一次用感觉很友好啊.喜 ...
- Python机器学习笔记：深入理解Keras中序贯模型和函数模型
先从sklearn说起吧,如果学习了sklearn的话,那么学习Keras相对来说比较容易.为什么这样说呢? 我们首先比较一下sklearn的机器学习大致使用流程和Keras的大致使用流程: skl ...
- alexeyab darknet 编译_【目标检测实战】Darknet—yolov3模型训练（VOC数据集)
原文发表在:语雀文档 0.前言 本文为Darknet框架下,利用官方VOC数据集的yolov3模型训练,训练环境为:Ubuntu18.04下的GPU训练,cuda版本10.0:cudnn版本7.6.5 ...
- TensorFlow2.0(二)--Keras构建神经网络分类模型
Keras构建分类模型 1. tf.keras简介 2. 利用tf.keras构建神经网络分类模型 2.1 导入相应的库 2.2 数据读取与展示 2.3 数据归一化 2.4 构建模型 2.5 模型的编 ...
- Keras——保存和提取模型
文章目录 1.前言 2.保存和提取模型 2.1.导入必要模块 2.2.构造数据 2.3.搭建模型并训练 2.4.保存模型 1.前言 今天讲解如何保存神经网络,这样以后想要用的时候直接提取就可以. 2. ...
- yolov3-tiny原始weights模型转onnx模型并进行推理
时隔不知道多少天,我记起来我还有部分博客没写完(偷懒),所以不能偷懒把它完成!! 这篇博客的主要内容 将yolov3-tiny.weights模型转换到.onnx模型: 使用onnnxruntime- ...
- 理解keras中的sequential模型
keras中的主要数据结构是model(模型),它提供定义完整计算图的方法.通过将图层添加到现有模型/计算图,我们可以构建出复杂的神经网络. Keras有两种不同的构建模型的方法: Sequentia ...
最新文章
- cs224n上完后会会获得证书吗_斯坦福NLP组-CS224n: NLP与深度学习-2019春全套资料分享...
- python详细安装教程3.7.4-python 3.7.4下载与安装的问题
- 苹果开发账号过期不续费会怎样？
- CABasicAnimation动画
- LeetCode-3. 无重复字符的最长子串
- Selenium自动化测试-3.元素定位(1)
- [转载] 七龙珠第一部——第072话 恶魔的厕所
- 最大的LeftMax与rightMax之差绝对值
- RV减速器静力学和动力学仿真分析
- java 汉字转拼音（解决多音字问题）
- vue 富文本编辑器 Editor 使用
- python生词本查单词译文_Kindle 阅读器“生词本”功能详细使用说明
- QPointer、QScopedPointer、QSharedDataPointer等指针用法总结
- atr指标 java算法_关于股指期货的SAR、CCI、BIAS、ATR指标的算法
- MEION:Costas环 环路滤波器参数设置程序
- js实现input的赋值，根据今天的日期获取本周的周一和周日的日期，获取当天日期
- Linux下四款常见远程工具比较
- matlab ild,10GBASE-KR
- WPF 基于 WER 注册应用崩溃无响应回调和重启方法
- pip命令大全 含换源方法