论文阅读和分析：《DeepGCNs: Can GCNs Go as Deep as CNNs?》

下面所有博客是个人对EEG脑电的探索，项目代码是早期版本不完整，需要完整项目代码和资料请私聊。

数据集
1、脑电项目探索和实现(EEG) (上)：研究数据集选取和介绍SEED
相关论文阅读分析：
1、EEG-SEED数据集作者的—基线论文阅读和分析
2、图神经网络EEG论文阅读和分析：《EEG-Based Emotion Recognition Using Regularized Graph Neural Networks》
3、EEG-GNN论文阅读和分析：《EEG Emotion Recognition Using Dynamical Graph Convolutional Neural Networks》
4、论文阅读和分析:Masked Label Prediction: Unified Message Passing Model for Semi-Supervised Classification
5、论文阅读和分析：《DeepGCNs: Can GCNs Go as Deep as CNNs?》
6、论文阅读和分析： “How Attentive are Graph Attention Networks?”
7、论文阅读和分析：Simplifying Graph Convolutional Networks

8、论文阅读和分析：LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation
9、图神经网络汇总和总结
相关实验和代码实现：
1、用于图神经网络的脑电数据处理实现_图神经网络脑电
2、使用GCN训练和测试EEG的公开SEED数据集
3、使用GAT训练和测试EEG公开的SEED数据集
4、使用SGC训练和测试SEED数据集
5、使用Transformer训练和测试EEG的公开SEED数据集_eeg transformer
6、使用RGNN训练和测试EEG公开的SEED数据集
辅助学习资料：
1、官网三个简单Graph示例说明三种层次的应用_graph 简单示例
2、PPI数据集示例项目学习图神经网络
3、geometric库的数据处理详解
4、NetworkX的dicts of dicts以及解决Seven Bridges of Königsberg问题
5、geometric源码阅读和分析：MessagePassin类详解和使用
6、cora数据集示例项目学习图神经网络
7、Graph 聚合
8、QM9数据集示例项目学习图神经网络
9、处理图的开源库

更深的图卷积神经网络，相当于residual在CNN中的应用，使得可以构建更深层次的卷积网络而不会造成梯度消失无法训练的问题。

算法原理：

学习过Resnet去理解DeepGCNs就会很容易，看图片基本就能明白。

提出了用于点云语义分割的GCN架构。（左）我们的框架由三个块组成：GCN骨干块（输入点云的特征转换）、融合块（全局特征生成和融合）和MLP预测块（逐点标签预测）。（右）我们研究了三种类型的GCN骨干块（PlainGCN、ResGCN和DenseGCN），并使用了两种层连接（ResGCN中使用的逐顶点加法或DenseGCN中使用的按顶点级联）。

geometric的实现

torch_geometric.nn — pytorch_geometric documentation (pytorch-geometric.readthedocs.io)

The implemented skip connections includes the pre-activation residual connection ("res+"), the residual connection ("res"), the dense connection ("dense") and no connections ("plain").

Res+ ("res+"):

Normalization → Activation → Dropout → GraphConv → Res \text{Normalization}\to\text{Activation}\to\text{Dropout}\to \text{GraphConv}\to\text{Res} Normalization→Activation→Dropout→GraphConv→Res

Res (:obj:"res") / Dense (:obj:"dense") / Plain(:obj:"plain"):

GraphConv → Normalization → Activation → Res/Dense/Plain → Dropout \text{GraphConv}\to\text{Normalization}\to\text{Activation}\to \text{Res/Dense/Plain}\to\text{Dropout} GraphConv→Normalization→Activation→Res/Dense/Plain→Dropout

geometric库的实现两种相关架构的网络单元，Res+和Res，需要输入卷积、激活和归一化等子单元，就是说可以用户定制化的实现，这一点很值得学习。一种延迟策略，实现良好的泛化性。

from typing import Optionalimport torch
import torch.nn.functional as F
from torch import Tensor
from torch.nn import Module
from torch.utils.checkpoint import checkpointclass DeepGCNLayer(torch.nn.Module):"""Args:conv (torch.nn.Module, optional): the GCN operator.(default: :obj:`None`)norm (torch.nn.Module): the normalization layer. (default: :obj:`None`)act (torch.nn.Module): the activation layer. (default: :obj:`None`)block (string, optional): The skip connection operation to use(:obj:`"res+"`, :obj:`"res"`, :obj:`"dense"` or :obj:`"plain"`).(default: :obj:`"res+"`)dropout (float, optional): Whether to apply or dropout.(default: :obj:`0.`)ckpt_grad (bool, optional): If set to :obj:`True`, will checkpoint thispart of the model. Checkpointing works by trading compute formemory, since intermediate activations do not need to be kept inmemory. Set this to :obj:`True` in case you encounter out-of-memoryerrors while going deep. (default: :obj:`False`)"""def __init__(self,conv: Optional[Module] = None,norm: Optional[Module] = None,act: Optional[Module] = None,block: str = 'res+',dropout: float = 0.,ckpt_grad: bool = False,):super().__init__()self.conv = convself.norm = normself.act = actself.block = block.lower()assert self.block in ['res+', 'res', 'dense', 'plain']self.dropout = dropoutself.ckpt_grad = ckpt_graddef reset_parameters(self):self.conv.reset_parameters()self.norm.reset_parameters()def forward(self, *args, **kwargs) -> Tensor:""""""args = list(args)x = args.pop(0)if self.block == 'res+':h = xif self.norm is not None:h = self.norm(h)if self.act is not None:h = self.act(h)h = F.dropout(h, p=self.dropout, training=self.training)if self.conv is not None and self.ckpt_grad and h.requires_grad:# checkpoint不保存中间变量，而是在后向更新的时候重新计算一遍。h = checkpoint(self.conv, h, *args, **kwargs)else:h = self.conv(h, *args, **kwargs)return x + helse:if self.conv is not None and self.ckpt_grad and x.requires_grad:h = checkpoint(self.conv, x, *args, **kwargs)else:h = self.conv(x, *args, **kwargs)if self.norm is not None:h = self.norm(h)if self.act is not None:h = self.act(h)if self.block == 'res':h = x + helif self.block == 'dense':h = torch.cat([x, h], dim=-1)elif self.block == 'plain':passreturn F.dropout(h, p=self.dropout, training=self.training)def __repr__(self) -> str:return f'{self.__class__.__name__}(block={self.block})'

注释：pytoch checkpoint

torch.utils.checkpoint — PyTorch 1.13 documentation

torch.utils.checkpoint 简介和简易使用_ONE_SIX_MIX的博客-CSDN博客

pytorch 的 checkpoint 是一种用时间换显存的技术，一般训练模式下，pytorch 每次运算后会保留一些中间变量用于求导，而使用 checkpoint 的函数，则不会保留中间变量，中间变量会在求导时再计算一次，因此减少了显存占用，这个 checkpoint 用的好的话，训练时相比不使用 checkpoint 的模型可以增加 30% 的批量大小。

参考：

[2006.07739] DeeperGCN: All You Need to Train Deeper GCNs (arxiv.org)

[1904.03751] DeepGCNs: Can GCNs Go as Deep as CNNs? (arxiv.org)

torch_geometric.nn — pytorch_geometric documentation (pytorch-geometric.readthedocs.io)

torch.utils.checkpoint — PyTorch 1.13 documentation

torch.utils.checkpoint 简介和简易使用_ONE_SIX_MIX的博客-CSDN博客
geometric/blob/master/examples/ogbn_proteins_deepgcn

论文阅读和分析：《DeepGCNs: Can GCNs Go as Deep as CNNs?》相关推荐

论文笔记：DeepGCNs: Can GCNs Go as Deep as CNNs?
前言 GCN与CNN有很多相似之处.GCN的卷积思想也是基于CNN卷积的优秀表现所提出的,.GCN由于其表达形式和卷积方法特殊性,在节点分类任务(引文网络)中,只有简单的3-4层可以把任务完成的很好. ...
论文阅读和分析： “How Attentive are Graph Attention Networks?”
下面所有博客是个人对EEG脑电的探索,项目代码是早期版本不完整,需要完整项目代码和资料请私聊. 数据集 1.脑电项目探索和实现(EEG) (上):研究数据集选取和介绍SEED 相关论文阅读分析: 1. ...
DeepGCNs-Can GCNs Go as Deep as CNNs? ICCV 2019
文章目录 1. 相关介绍 1.1 背景介绍 1.2 contribution 1.3 CNN中的dilated convolutions (膨胀卷积.扩张卷积.空洞卷积) 2. 模型与方法 2.1 图 ...
论文阅读和分析：When Counting Meets HMER Counting-Aware Network for HMER
HMER论文系列 1.论文阅读和分析:When Counting Meets HMER Counting-Aware Network for HMER_KPer_Yang的博客-CSDN博客 2.论文 ...
论文阅读和分析：Hybrid Mathematical Symbol Recognition using Support Vector Machines
HMER论文系列 1.论文阅读和分析:When Counting Meets HMER Counting-Aware Network for HMER_KPer_Yang的博客-CSDN博客 2.论文 ...
论文阅读和分析：A Tree-Structured Decoder for Image-to-Markup Generation
HMER论文系列 1.论文阅读和分析:When Counting Meets HMER Counting-Aware Network for HMER_KPer_Yang的博客-CSDN博客 2.论文 ...
【论文阅读】Mastering the game of Go with deep neural networks and tree search
[论文阅读]Mastering the game of Go with deep neural networks and tree search 1 本文解决了什么问题? 在所有的完全信息博弈中, ...
情感分析论文阅读之《Aspect Level Sentiment Classification with Deep Memory Network》
本文利用了加入了attention机制的QA系统中的深度记忆网络,本文将aspect word的上下文信息作为memory m中存储的内容.实现了一个针对aspect-level的情感分类模型. ...
【保姆级】论文阅读与分析《Learning Heterogeneous Knowledge Base Embeddings for Explainable Recommendation》
<Learning Heterogeneous Knowledge Base Embeddings for Explainable Recommendation>-by Qingyao A ...

论文阅读和分析：《DeepGCNs: Can GCNs Go as Deep as CNNs?》

geometric的实现

论文阅读和分析：《DeepGCNs: Can GCNs Go as Deep as CNNs?》相关推荐

最新文章

热门文章