tf.nn.nce_loss 函数应用案例

tf.nn.nce_loss(weights,biases,labels,inputs,num_sampled,num_classes,num_true=1,sampled_values=None,remove_accidental_hits=False,partition_strategy='mod',name='nce_loss'
)

对于 nce_loss 的了解源于 word2vec，主要是通过负采样的方式减少 softmax 函数的计算，具体函数值讲解，可以字节看疼我tensorflow，这里提供一个简单的案例，说明怎么使用 nce_loss。

import numpy as np
import tensorflow as tf
from tensorflow.examples.tutorials.mnist import input_data
import mathtrain_loss_lst = []
train_accuracy_lst = []
test_accuracy_lst = []def deepn(x):"""Args:x: an input tensor with the dimensions (N_examples, 784), where 784 is thenumber of pixels in a standard MNIST image.Returns:y: (N_examples, 64)X(N_examples, 784) x w1(784, 64) + b1(64,) = y(N_examples, 64)"""w1 = tf.Variable(initial_value=tf.truncated_normal(shape=[784, 64]),name="w1")b1 = tf.Variable(initial_value=tf.random_uniform(shape=[64,], minval=0, maxval=1))fc1 = tf.matmul(x, w1) + b1keep_prob = tf.placeholder(tf.float32)fc1_drop = tf.nn.dropout(fc1, keep_prob)return fc1_drop, keep_probdef main():# Import datamnist = input_data.read_data_sets("/mnist_data", one_hot=True)X = tf.placeholder(tf.float32, [None, 784])y_ = tf.placeholder(tf.float32, [None, 10])y_idx = tf.placeholder(tf.float32, [None, 1])fc1_drop, keep_prob = deepn(X)num_sampled = 1vocabulary_size = 10embedding_size = 64nce_weights = tf.Variable(tf.truncated_normal([vocabulary_size, embedding_size],stddev=1.0/math.sqrt(embedding_size)),name="embed")nce_biases = tf.Variable(tf.zeros([vocabulary_size]))loss = tf.reduce_mean(tf.nn.nce_loss(weights=nce_weights,biases=nce_biases,labels=y_idx,inputs=fc1_drop,num_sampled=num_sampled,num_classes=vocabulary_size),)train_step = tf.train.AdamOptimizer(1e-4).minimize(loss)output = tf.matmul(fc1_drop, tf.transpose(nce_weights)) + nce_biasescorrect_prediction = tf.equal(tf.argmax(output, 1), tf.argmax(y_, 1))accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))with tf.Session() as sess:sess.run(tf.global_variables_initializer())for i in range(20000):print("num: %d"%i)batch = mnist.train.next_batch(20)idx_ = np.argmax(batch[1], axis=1)[:, np.newaxis].astype("float32")train_accuracy, train_loss, _ = sess.run([accuracy, loss, train_step],feed_dict={X: batch[0],y_: batch[1],y_idx: idx_,keep_prob: 1.0})print("loss: ", train_loss)print("train accuracy: ", train_accuracy)idx_ = np.argmax(mnist.test.labels, axis=1)[:, np.newaxis].astype("float32")test_accuracy = sess.run(accuracy,feed_dict={X: mnist.test.images,y_: mnist.test.labels,y_idx: idx_,keep_prob: 1.0})print("test accuracy: ", test_accuracy)train_loss_lst.append(train_loss)train_accuracy_lst.append(train_accuracy)test_accuracy_lst.append(test_accuracy)def test():mnist = input_data.read_data_sets("./mnist_data", one_hot=True)X = tf.placeholder(tf.float32, [None, 784])fc1_drop, keep_prob = deepn(X)with tf.Session() as sess:sess.run(tf.global_variables_initializer())for i in range(2):batch = mnist.train.next_batch(20)idx_ = np.argmax(batch[1], axis=1)[:, np.newaxis].astype("float32")fc1 = sess.run([fc1_drop], feed_dict={X: batch[0], keep_prob:1.0})print(np.array(fc1).shape)main()
def summary(x, tag, path):"""根据提供的 x 列表绘制曲线"""print(x)loss = 0.0# tf.summary模块的定义位于summary.py文件中，该文件中主要定义了在进行可视化将要用到的各种函数loss_summary = tf.Summary()# 调用tf.summary.Summary.Value子类loss_summary.value.add(tag=tag, simple_value=loss)  # tag就是待会产生的图标名称with tf.Session() as sess:# 生成一个写日志的writer，将当前tensorflow计算图写入日志。summary_writer1 = tf.summary.FileWriter(path, sess.graph)tf.global_variables_initializer().run()for i in range(len(x)):# 固定用法，具体为什么我也不懂loss_summary.value[0].simple_value = x[i]summary_writer1.add_summary(loss_summary, i)summary(train_loss_lst, tag="loss", path="./train_loss")
summary(train_accuracy_lst, tag="accuracy", path="./train_accuracy")
summary(test_accuracy_lst, tag="accuracy", path="./test_accuracy")

tf.nn.nce_loss 函数应用案例相关推荐

Tensorflow BatchNormalization详解：4_使用tf.nn.batch_normalization函数实现Batch Normalization操作...
使用tf.nn.batch_normalization函数实现Batch Normalization操作觉得有用的话,欢迎一起讨论相互学习~Follow Me 参考文献吴恩达deeplearnin ...
tf.nn.embedding_lookup函数的用法
tf.nn.embedding_lookup函数的用法主要是选取一个张量里面索引对应的元素.tf.nn.embedding_lookup(params, ids):params可以是张量也可以是数组等 ...
tf.nn.moments( )函数的使用
先来看个例子吧. 测试代码: x = np.arange(12,dtype=np.float32).reshape(3,4) a = tf.nn.moments(tf.constant(x),[0]) ...
tf.nn.nce_loss分析
tf官方代码 def nce_loss(weights,biases,labels,inputs,num_sampled,num_classes,num_true=1,sampled_values=N ...
【TensorFlow】tf.nn.softmax_cross_entropy_with_logits 函数：求交叉熵损失
[TensorFlow]tf.nn.softmax_cross_entropy_with_logits的用法_xf__mao的博客-CSDN博客 https://blog.csdn.net/mao_x ...
tf.nn.bidirectional_dynamic_rnn()函数详解
转载自:https://blog.csdn.net/zhylhy520/article/details/86364789 首先我们了解一下函数的参数 bidirectional_dynamic_rnn ...
tf.nn.conv2d()函数详解(strides与padding的关系)
tf.nn.conv2d()是TensorFlow中用于创建卷积层的函数,这个函数的调用格式如下: def conv2d(input: Any,filter: Any,strides: Any,pad ...
tf.nn.leaky_relu()函数
**计算Leaky ReLU激活函数 tf.nn.leaky_relu( features, alpha=0.2, name=None ) 参数: features:一个Tensor,表示预激活 al ...
深度学习-函数-tf.nn.embedding_lookup 与tf.keras.layers.Embedding
embedding函数用法 1. one_hot编码 1.1. 简单对比 1.2.优势分析: 1.3. 缺点分析: 1.4. 延伸思考 2. embedding的用途 2.1 embedding有两个 ...

tf.nn.nce_loss 函数应用案例

tf.nn.nce_loss 函数应用案例相关推荐

最新文章

热门文章