大核注意力Large Kernel Attention(LKA)

这种分解方式也被叫做大核注意力(Large Kernel Attention)，即LKA。如上图所示，一个很大kernel size的卷积被分解成一个Depth-wise卷积+一个Depth-wise空洞卷积+一个1× \times× 1卷积。这样，就可以大大减少FLOPs和参数量。很有效地解决了小核卷积的local性。

class AttentionModule(nn.Module):def __init__(self, dim):super().__init__()self.conv0 = nn.Conv2d(dim, dim, 5, padding=2, groups=dim)  #depth-wise convself.conv_spatial = nn.Conv2d(dim, dim, 7, stride=1, padding=9, groups=dim, dilation=3)   #conv_spatialself.conv1 = nn.Conv2d(dim, dim, 1)  # 1x1 convdef forward(self, x):u = x.clone()        attn = self.conv0(x)attn = self.conv_spatial(attn)attn = self.conv1(attn)return u *

大核注意力Large Kernel Attention(LKA)相关推荐

VAN（大核注意力机制）
Visual-Attention-Network/VAN-Classification (github.com) transformer在视觉领域得到良好的效果,是因为它可以捕捉长距离的信息.在视觉领 ...
【论文阅读】Scaling Up Your Kernels to 31x31: Revisiting Large Kernel Design in CNNs
[论文阅读]Scaling Up Your Kernels to 31x31: Revisiting Large Kernel Design in CNNs 1.摘要 2.通过ViTs审视CNN 3. ...
Scaling Up Your Kernels to 31x31: Revisiting Large Kernel Design in CNNs笔记
论文地址:https://arxiv.org/pdf/2203.06717.pdf 代码地址:GitHub - DingXiaoH/RepLKNet-pytorch: Scaling Up Your ...
图解时间自适应卷积----Time-aware Large Kernel Convolutions
论文:Time-aware Large Kernel Convolutions 链接:arxiv.org/pdf/2002.03184v1.pdf 偶尔看到一篇比较有意思的论文,其提出的方法设计得很巧 ...
深度学习论文: Scaling Up Your Kernels to 31x31: Revisiting Large Kernel Design in CNNs及其PyTorch实现
深度学习论文: Scaling Up Your Kernels to 31x31: Revisiting Large Kernel Design in CNNs及其PyTorch实现 Scaling ...
RepLKNet：Scaling Up Your Kernels to 31x31: Revisiting Large Kernel Design in CNNs
<Scaling Up Your Kernels to 31x31: Revisiting Large Kernel Design in CNNs> 论文: https://arxiv.o ...
语义分割--Large Kernel Matters--Improve Semantic Segmentation by Global Convolutional Network
Large Kernel Matters–Improve Semantic Segmentation by Global Convolutional Network https://arxiv.org ...
注意力机制～Attention Mechanism
目录背景 1. Attention Function 2. Scaled Dot-Product Attention 3. Multi-attention --> Q, K=V 4. self ...
Large Kernel Matters
Large Kernel Matters 论文信息论文地址:Large Kernel Matters --Improve Semantic Segmentation by Global Convol ...
深入理解注意力机制（Attention Mechanism）和Seq2Seq
学习本部分默认大家对RNN神经网络已经深入理解了,这是基础,同时理解什么是时间序列,尤其RNN的常用展开形式进行画图,这个必须理解了. 这篇文章整理有关注意力机制(Attention Mechanis ...

大核注意力Large Kernel Attention(LKA)

大核注意力Large Kernel Attention(LKA)相关推荐

最新文章

热门文章