经典卷积网络——DenseNet代码实现

题目：Densely Connected Convolutional Networks

论文地址：https://arxiv.org/pdf/1608.06993.pdf

常见的卷积网络结构对比：

图1. 经典卷积结构对比

DenseNet网络拓扑图：

图2. DenseNet网络拓扑图

图2.我们可以知道DenseNet网络主要有几个模型组成：DenseNet Block,Transition Layer,为了降低参数量，DenseNet Block内部采用了bottleneck layer。具体描述中提到的各种模块，下文本人会一一给出图解。

图3. DenseNet Block块内部采用Dense Connectivity

图4. DenseNet Block块内部实现方式

图5. Transition Layer内部的实现方式

图6. DenseNet网络结构

代码实现（tensorflow版）

import tensorflow as tfMC_MODE = False #是否要经过bottleneck结构TF_VERSION = float('.'.join(tf.__version__.split('.')[:2]))def batch_norm(_input,is_training):output = tf.contrib.layers.batch_norm(_input, scale=True, is_training = is_training,updates_collections=None)return outputdef dropout(_input,is_training,keep_prob):'''进行dropout'''if keep_prob < 1:output = tf.cond(is_training,lambda: tf.nn.dropout(_input, keep_prob),lambda: _input)else:output = _inputreturn outputdef composite_function(_input, is_training,keep_prob,out_features, kernel_size=3):''''''with tf.variable_scope("composite_function"):output = batch_norm(_input,is_training)output = tf.nn.relu(output)output = conv2d(output, out_features = out_features, kernel_size=kernel_size)output = dropout(output,is_training,keep_prob)return outputdef bottleneck(_input, out_features,is_training,keep_prob):'''Bottleneck由两个部分组成：[1×1]的卷积组和[3×3]的卷积组，其意义在于[1×1]的卷积层能减少输入的特征图，之后再用[3×3]的卷积核进行处理。'''with tf.variable_scope("bottleneck"):output = batch_norm(_input,is_training)output = tf.nn.relu(output)inter_features = out_features * 4output = conv2d(output, out_features=inter_features, kernel_size=1,padding='VALID')output = dropout(output,is_training,keep_prob)return outputdef add_internal_layer(_input, bc_mode,growth_rate,is_training,keep_prob):'''Denseblock每一层'''if not bc_mode:comp_out = composite_function(_input,is_training,keep_prob,out_features=growth_rate, kernel_size=3)elif bc_mode:bottleneck_out = bottleneck(_input, growth_rate,is_training,keep_prob)comp_out = composite_function(bottleneck_out,is_training,keep_prob, out_features=growth_rate, kernel_size=3)if TF_VERSION >= 1.0:output = tf.concat(axis=3, values=(_input, comp_out))else:output = tf.concat(3, (_input, comp_out))return outputdef add_block(_input,bc_mode, growth_rate, layers_per_block,is_training,keep_prob):'''对每个dense_block中层的进行添加'''output = _inputfor layer in range(layers_per_block):with tf.variable_scope("layer_%d" % layer):output = add_internal_layer(output,bc_mode, growth_rate,is_training,keep_prob)return outputdef weight_variable_msra(shape, name):'''对2D卷积核进行初始化'''return tf.get_variable(name = name,shape = shape,initializer = tf.contrib.layers.variance_scaling_initializer())def conv2d(_input,out_features,kernel_size,strides=[1, 1, 1, 1],padding='SAME'):'''对2维卷积进行wrap'''in_features = int(_input.get_shape()[-1])kernel = weight_variable_msra([kernel_size,kernel_size,in_features,out_features],name='kernel')output = tf.nn.conv2d(_input,kernel,strides,padding)return outputdef avg_pool(_input, k):'''定义池化层'''ksize = [1, k, k, 1]strides = [1, k, k, 1]padding = 'VALID'output = tf.nn.avg_pool(_input, ksize, strides, padding)return outputdef transition_layer(_input,reduction,is_training,keep_prob):'''Transition_layer是介于两个Denseblock之间的转换模块，每一个Denseblock输出的feature maps都比较多，如果统统都输入到下一层，将会极大的增加神经网络的参数，所以transition_layer的主要工作就是降维。'''# call composite function with 1x1 kernelout_features = int(int(_input.get_shape()[-1]) * reduction)output = composite_function(_input,is_training,keep_prob, out_features=out_features, kernel_size=1)# run average poolingoutput = avg_pool(output, k=2)return outputdef transition_layer_to_classes(_input,is_training,n_classes):'''定义全连接层'''output = batch_norm(_input,is_training)output = tf.nn.relu(output)last_pool_kernel = int(output.get_shape()[-2])output = avg_pool(output, k=last_pool_kernel)features_total = int(output.get_shape()[-1])output = tf.reshape(output, [-1, features_total])W = weight_variable_xavier([features_total, n_classes], name='W')bias = bias_variable([n_classes])logits = tf.matmul(output, W) + biasreturn logitsdef weight_variable_xavier(shape, name):'''定义全连接层权重初始化'''return tf.get_variable(name,shape=shape,initializer=tf.contrib.layers.xavier_initializer())
def bias_variable(shape, name='bias'):'''定义偏置初始化'''initial = tf.constant(0.0, shape=shape)return tf.get_variable(name, initializer=initial)def loss(logits,labels,weight_decay):'''定义损失函数'''cross_entropy = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=logits, labels=labels))l2_loss = tf.add_n([tf.nn.l2_loss(var) for var in tf.trainable_variables()])total_loss = cross_entropy + l2_loss * weight_decayreturn total_lossdef optimizer(learning_rate,nesterov_momentum,total_loss):'''定义优化器'''optimizer = tf.train.MomentumOptimizer(learning_rate, nesterov_momentum, use_nesterov=True)train_step = optimizer.minimize(total_loss)return train_stepdef accuracy(prediction,labels):'''定义准确度函数'''correct_prediction = tf.equal(tf.argmax(prediction, 1),tf.argmax(labels, 1))accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))return accuracydef DenseNet(x,first_output_features,total_blocks,growth_rate,depth,is_training,keep_prob,reduction,bc_mode,n_classes):'''x:输出特征图:[BATCH_SIZE,CROP_SIZE,CROP_SIZE,CHANNELS]first_output_features:第一个卷积层输出的特征图个数（int）total_blocks:dense_blocks 的块数(int)growth_rate:每一层特征图的增长率(int)depth:网络的深度is_training:用来控制BN层/DROPOUT层 是训练还是测试阶段keep_prob:控制dropoutreduction:降维率(0.0~1.0),用于Transition_layern_classes:类别数mc_model:should we use bottleneck layers and features reduction or not.'''# 每个DenseBlock中层的个数layers_per_block = (depth - (total_blocks + 1)) // total_blockswith tf.variable_scope("Initial_convolution"):output = conv2d(x,out_features=first_output_features,kernel_size=3)for block in range(total_blocks):with tf.variable_scope("Block_%d" % block):output =  add_block(output,bc_mode, growth_rate, layers_per_block,is_training,keep_prob)if block != total_blocks - 1:with tf.variable_scope("Transition_after_block_%d" % block):output = transition_layer(output,reduction,is_training,keep_prob)with tf.variable_scope("Transition_to_classes"):logits = transition_layer_to_classes(output,is_training,n_classes)prediction = tf.nn.softmax(logits)return {"logits":logits,"prediction":prediction}

上面只是网络结构代码，具体损失函数，评估标准可以自己定义，只要在您的代码中调用上面代码中的Densenet()函数就好了！

经典卷积网络——DenseNet代码实现相关推荐

41_经典卷积网络、LeNet、AlexNet、VGG、GoogleNet、ResNet、NIN、DenseNet、EfficientNet、MobileNetV1/2/3、SENet等
1.38.经典卷积网络 1.38.1.LeNet 1.38.1.1.介绍 1.38.1.2.网络结构 1.38.1.3.代码实现 1.38.2.AlexNet 1.38.2.1.介绍 1.38.2.2 ...
深度学习笔记（27）经典卷积网络
深度学习笔记(27) 经典卷积网络 1. 前言 2. LeNet-5 3. AlexNet 4. VGGNet 1. 前言讲了基本构建,比如卷积层.池化层以及全连接层这些组件事实上,过去几年计算机 ...
Pytorch Note32 稠密连接的卷积网络 DenseNet
Pytorch Note32 稠密连接的卷积网络 DenseNet 文章目录 Pytorch Note32 稠密连接的卷积网络 DenseNet DenseNet Dense Block DenseN ...
经典卷积网络：AlexNet、ZFNet、VGG、GoogleNet、ResNet
提示:文章写完后,目录可以自动生成,如何生成可参考右边的帮助文档文章目录前言一.AlexNet 1.理论介绍 2.代码实现 model部分二.ZFNet 三.VGG 四.GoogleNet 五 ...
经典卷积网络--VGGNet
经典卷积网络--VGGNet 1.VGGNet网络模型 2.VGGNet网络模型搭建(使用Tensorflow) 3.完整代码实现(使用CIFAR10数据集) 借鉴点:小卷积核减少参数的同时,提高识别 ...
经典卷积网络--InceptionNet
经典卷积网络--InceptionNet 1.InceptionNet网络模型 2.1 * 1的卷积运算是如何降低特征厚度? 3.InceptionNet完整实现(使用CIFAR10数据集) 借鉴点: ...
经典卷积网络学习----FCN（图像分割）
目录 1,CNN与FCN的区别 2,FCN的详细步骤 3,FCN的训练过程 4,总结 5,知识点补充 5.1 为什么FCN输入图像的大小可以任意 5.2 语义分割和实例分割的区别 6,图像分割通用框架 ...
轻量级卷积网络DenseNet：密集连接卷积网络
原文地址:CVPR 2017 <Densely Connected Convolutional Networks> 卷积神经网络如何提高效果: 网络做得更深:ResNet,解决了网络深时的 ...
经典卷积网络进阶--ResNet详解
一.ResNet概述 resnet在2015名声大噪,微软公司提出了一种新的网络结构---残差网络(resnet).残差模块结构图如下图1,图中曲线连接方式(X identity)称为近道连接,这种连 ...

经典卷积网络——DenseNet代码实现

常见的卷积网络结构对比：

DenseNet网络拓扑图：

代码实现（tensorflow版）

经典卷积网络——DenseNet代码实现相关推荐

最新文章

热门文章