Pytorch中Softmax和LogSoftmax的使用
目录
一、函数解释
二、代码示例
三、整体代码
一、函数解释
1.Softmax函数常用的用法是指定参数dim就可以:
(1)dim=0:对每一列的所有元素进行softmax运算,并使得每一列所有元素和为1。
(2)dim=1:对每一行的所有元素进行softmax运算,并使得每一行所有元素和为1。
class Softmax(Module):r"""Applies the Softmax function to an n-dimensional input Tensorrescaling them so that the elements of the n-dimensional output Tensorlie in the range [0,1] and sum to 1.Softmax is defined as:.. math::\text{Softmax}(x_{i}) = \frac{\exp(x_i)}{\sum_j \exp(x_j)}Shape:- Input: :math:`(*)` where `*` means, any number of additionaldimensions- Output: :math:`(*)`, same shape as the inputReturns:a Tensor of the same dimension and shape as the input withvalues in the range [0, 1]Arguments:dim (int): A dimension along which Softmax will be computed (so every slicealong dim will sum to 1)... note::This module doesn't work directly with NLLLoss,which expects the Log to be computed between the Softmax and itself.Use `LogSoftmax` instead (it's faster and has better numerical properties).Examples::>>> m = nn.Softmax(dim=1)>>> input = torch.randn(2, 3)>>> output = m(input)"""__constants__ = ['dim']def __init__(self, dim=None):super(Softmax, self).__init__()self.dim = dimdef __setstate__(self, state):self.__dict__.update(state)if not hasattr(self, 'dim'):self.dim = Nonedef forward(self, input):return F.softmax(input, self.dim, _stacklevel=5)def extra_repr(self):return 'dim={dim}'.format(dim=self.dim)
2.LogSoftmax其实就是对softmax的结果进行log,即Log(Softmax(x))
class LogSoftmax(Module):r"""Applies the :math:`\log(\text{Softmax}(x))` function to an n-dimensionalinput Tensor. The LogSoftmax formulation can be simplified as:.. math::\text{LogSoftmax}(x_{i}) = \log\left(\frac{\exp(x_i) }{ \sum_j \exp(x_j)} \right)Shape:- Input: :math:`(*)` where `*` means, any number of additionaldimensions- Output: :math:`(*)`, same shape as the inputArguments:dim (int): A dimension along which LogSoftmax will be computed.Returns:a Tensor of the same dimension and shape as the input withvalues in the range [-inf, 0)Examples::>>> m = nn.LogSoftmax()>>> input = torch.randn(2, 3)>>> output = m(input)"""__constants__ = ['dim']def __init__(self, dim=None):super(LogSoftmax, self).__init__()self.dim = dimdef __setstate__(self, state):self.__dict__.update(state)if not hasattr(self, 'dim'):self.dim = Nonedef forward(self, input):return F.log_softmax(input, self.dim, _stacklevel=5)
二、代码示例
输入代码
import torch
import torch.nn as nn
import numpy as npbatch_size = 4
class_num = 6
inputs = torch.randn(batch_size, class_num)
for i in range(batch_size):for j in range(class_num):inputs[i][j] = (i + 1) * (j + 1)print("inputs:", inputs)
得到大小batch_size为4,类别数为6的向量(可以理解为经过最后一层得到)
tensor([[ 1., 2., 3., 4., 5., 6.],
[ 2., 4., 6., 8., 10., 12.],
[ 3., 6., 9., 12., 15., 18.],
[ 4., 8., 12., 16., 20., 24.]])
接着我们对该向量每一行进行Softmax
Softmax = nn.Softmax(dim=1)
probs = Softmax(inputs)
print("probs:\n", probs)
得到
tensor([[4.2698e-03, 1.1606e-02, 3.1550e-02, 8.5761e-02, 2.3312e-01, 6.3369e-01],
[3.9256e-05, 2.9006e-04, 2.1433e-03, 1.5837e-02, 1.1702e-01, 8.6467e-01],
[2.9067e-07, 5.8383e-06, 1.1727e-04, 2.3553e-03, 4.7308e-02, 9.5021e-01],
[2.0234e-09, 1.1047e-07, 6.0317e-06, 3.2932e-04, 1.7980e-02, 9.8168e-01]])
此外,我们对该向量每一行进行LogSoftmax
LogSoftmax = nn.LogSoftmax(dim=1)
log_probs = LogSoftmax(inputs)
print("log_probs:\n", log_probs)
得到
tensor([[-5.4562e+00, -4.4562e+00, -3.4562e+00, -2.4562e+00, -1.4562e+00, -4.5619e-01],
[-1.0145e+01, -8.1454e+00, -6.1454e+00, -4.1454e+00, -2.1454e+00, -1.4541e-01],
[-1.5051e+01, -1.2051e+01, -9.0511e+00, -6.0511e+00, -3.0511e+00, -5.1069e-02],
[-2.0018e+01, -1.6018e+01, -1.2018e+01, -8.0185e+00, -4.0185e+00, -1.8485e-02]])
验证每一行元素和是否为1
# probs_sum in dim=1
probs_sum = [0 for i in range(batch_size)]for i in range(batch_size):for j in range(class_num):probs_sum[i] += probs[i][j]print(i, "row probs sum:", probs_sum[i])
得到每一行的和,看到确实为1
0 row probs sum: tensor(1.)
1 row probs sum: tensor(1.0000)
2 row probs sum: tensor(1.)
3 row probs sum: tensor(1.)
验证LogSoftmax是对Softmax的结果进行Log
# to numpy
np_probs = probs.data.numpy()
print("numpy probs:\n", np_probs)# np.log()
log_np_probs = np.log(np_probs)
print("log numpy probs:\n", log_np_probs)
得到
numpy probs:
[[4.26977826e-03 1.16064614e-02 3.15496325e-02 8.57607946e-02 2.33122006e-01 6.33691311e-01]
[3.92559559e-05 2.90064461e-04 2.14330270e-03 1.58369839e-02 1.17020354e-01 8.64669979e-01]
[2.90672347e-07 5.83831024e-06 1.17265590e-04 2.35534250e-03 4.73083146e-02 9.50212955e-01]
[2.02340233e-09 1.10474026e-07 6.03167746e-06 3.29318427e-04 1.79801770e-02 9.81684387e-01]]
log numpy probs:
[[-5.4561934e+00 -4.4561934e+00 -3.4561934e+00 -2.4561932e+00 -1.4561933e+00 -4.5619333e-01]
[-1.0145408e+01 -8.1454077e+00 -6.1454072e+00 -4.1454072e+00 -2.1454074e+00 -1.4540738e-01]
[-1.5051069e+01 -1.2051069e+01 -9.0510693e+00 -6.0510693e+00 -3.0510693e+00 -5.1069155e-02]
[-2.0018486e+01 -1.6018486e+01 -1.2018485e+01 -8.0184851e+00 -4.0184855e+00 -1.8485421e-02]]
验证完毕
三、整体代码
import torch
import torch.nn as nn
import numpy as npbatch_size = 4
class_num = 6
inputs = torch.randn(batch_size, class_num)
for i in range(batch_size):for j in range(class_num):inputs[i][j] = (i + 1) * (j + 1)print("inputs:", inputs)Softmax = nn.Softmax(dim=1)
probs = Softmax(inputs)
print("probs:\n", probs)LogSoftmax = nn.LogSoftmax(dim=1)
log_probs = LogSoftmax(inputs)
print("log_probs:\n", log_probs)# probs_sum in dim=1
probs_sum = [0 for i in range(batch_size)]for i in range(batch_size):for j in range(class_num):probs_sum[i] += probs[i][j]print(i, "row probs sum:", probs_sum[i])# to numpy
np_probs = probs.data.numpy()
print("numpy probs:\n", np_probs)# np.log()
log_np_probs = np.log(np_probs)
print("log numpy probs:\n", log_np_probs)
Pytorch中Softmax和LogSoftmax的使用相关推荐
- 【Pytorch】| Pytorch中softmax的dim的详细总结
[Pytorch]| Pytorch中softmax的dim的详细总结 关于softmax的理解 一维向量:dim=0和dim=-1结果相同,dim=1和dim=2会报错 二维张量:dim=1和dim ...
- Pytorch中torch.nn.Softmax的dim参数含义
自己搞了一晚上终于搞明白了,下文说的很透彻,做个记录,方便以后翻阅 Pytorch中torch.nn.Softmax的dim参数含义
- 损失函数-负对数似然和交叉熵(Pytorch中的应用)
文章目录 1.负对数似然损失函数 1.1.似然 1.2.似然函数 1.3.极大似然估计 1.4.对数似然 1.5.负对数似然 1.6.pytorch中的应用 2.交叉熵损失函数 2.1.信息量 2.2 ...
- Pytorch 学习(7):Pytorch中的Non-linear Activations (非线性层)实现
Pytorch 学习(7):Pytorch中的Non-linear Activations (非线性层)实现 Pytorch中的Non-linear Activations (非线性层)包括以下激活函 ...
- 如何利用PyTorch中的Moco-V2减少计算约束
介绍 SimCLR论文(http://cse.iitkgp.ac.in/~arastogi/papers/simclr.pdf)解释了这个框架如何从更大的模型和更大的批处理中获益,并且如果有足够的计算 ...
- 正则化技巧:标签平滑(Label Smoothing)以及在 PyTorch 中的实现
来源:DeepHub IMBA 本文约1200字,建议阅读5分钟 在这篇文章中,我们研究了标签平滑,这是一种试图对抗过度拟合和过度自信的技术. 过拟合和概率校准是训练深度学习模型时出现的两个问题.深度 ...
- pytorch贝叶斯网络_贝叶斯神经网络:2个在TensorFlow和Pytorch中完全连接
pytorch贝叶斯网络 贝叶斯神经网络 (Bayesian Neural Net) This chapter continues the series on Bayesian deep learni ...
- pytorch中CrossEntropyLoss和NLLLoss的区别与联系
pytorch中CrossEntropyLoss和NLLLoss的区别与联系 CrossEntropyLoss和NLLLoss主要是用在多分类问题的损失函数,他们两个既有不同,也有不浅的联系.先分别看 ...
- 机器学习花朵图像分类_在PyTorch中使用转移学习进行图像分类
想了解更多好玩的人工智能应用,请关注公众号"机器AI学习 数据AI挖掘","智能应用"菜单中包括:颜值检测.植物花卉识别.文字识别.人脸美妆等有趣的智能应用.. ...
最新文章
- NFS、SSH、SAMBA
- spring中使用注解代替xml配置
- 架构设计-数据访问层简述
- Linux进程管理与调度-之-目录导航【转】
- iOS设备中的推送(三):页面跳转
- 中南大学计算机网络期末试卷,中南大学计算机网络期末复习试卷1
- CISCO3560 VLAN配置实例
- Hibernate【3】——Service层
- 14-STM32F1 iic 24c02
- 网站克隆:setoolkit社工软件
- 学前教育怎么利用计算机思维,乐高教育全新推出编程启蒙小火车锻炼孩子计算机思维...
- 七升七降调号_巧识五线谱08:如何记住七个“降号调”的调号与调的对应关系?...
- 小猫钓鱼纸牌游戏java_java实现纸牌游戏-小猫钓鱼算法
- 范数不等式 琴生兄弟不等式 补全百度
- unity小游戏的脚本
- Instagram养号:账号防封攻略
- MySQL不会丢失数据的秘密,就藏在它的 7种日志里
- 5-TDengine集成SpringBoot,MyBatis,MyBatisPlus
- 基于孪生网络的跟踪算法汇总
- 如何用C++实现一个简易数据库(十一)