🤵 Author ：Horizon Max

✨ 编程技巧篇：各种操作小结

🎇 机器视觉篇：会变魔术 OpenCV

💥 深度学习篇：简单入门 PyTorch

🏆 神经网络篇：经典网络模型

💻 算法篇：再忙也别忘了 LeetCode

🚀 Squeeze-and-Excitation Networks

Squeeze ：挤压     Excitation ：激励 ；

Squeeze-and-Excitation Networks 简称 SENet ，由 Momenta 和 牛津大学 的Jie Hu等人 提出的一种新的网络结构；

目标是通过建模 卷积特征通道之间的相互依赖关系 来提高网络的表示能力；

在2017年最后一届 ImageNet 挑战赛(ILSVRC) classification 任务中获得 冠军，将错误率降低到 2.251% ；

🔗 论文地址：Squeeze-and-Excitation Networks

🚀 SENet 详解

🎨 Squeeze-and-Excitation block

Squeeze-and-Excitation block

对于任意给定的变换： Ftr ：X → U ，其中 X ∈ R H’xW’xC’ ， U ∈ R HxWxC ，Ftr 用作一个卷积算子 ；

🚩 Squeeze: Global Information Embedding

挤压：全局信息嵌入

（1）Squeeze ：特征U通过 squeeze 压缩操作，将跨空间维度H × W的特征映射进行聚合，生成一个通道描述符，HxWxC → 1x1xC ；

将 全局空间信息 压缩到上述 通道描述符 中，使来这些 通道描述符 可以被 其输入的层 利用，这里采用的是 global average pooling ；

🚩 Excitation: Adaptive Recalibration

激励：自适应调整

（2）Excitation ：每个通道通过一个 基于通道依赖 的自选门机制 来学习特定样本的激活，使其学会使用全局信息，有选择地强调信息特征，并抑制不太有用的特征，这里采用的是 sigmoid ，并在中间嵌入了 ReLU 函数用于限制模型的复杂性和帮助训练 ；

通过 两个全连接层(FC) 构成的瓶颈来参数化门控机制，即 W1 用于降低维度，W2 用于维度递增 ；

（3）Reweight ：将 Excitation 输出的权重通过乘法逐通道加权到输入特征上；

总的来说 SE Block 就是在 Layer 的输入和输出之间添加结构： global average pooling - FC - ReLU - FC- sigmoid ；

SE block 的灵活性意味着它可以直接应用于标准卷积以外的转换，通过将 SE block 集成到任何复杂模型当中来开发SENet；

🚩 在非残差网络中的应用

应用于 非残差网络 Inception network 当中，形成 SE-Inception module ；

非残差网络结构框图（Inception block）

Scale ： 改变（文字、图片）的尺寸大小

🚩 在残差网络中的应用

应用于 残差网络 Residual network 当中，形成 SE-ResNet module ；

残差网络结构框图（Residual Block）

论文中对 SE block 的应用用于实验对比：

SE-ResNet-50 网络的准确性优于 ResNet-50 和模型深化版的 ResNet101 网络 ；

对于224 × 224像素的输入图像，ResNet-50 需要164 ms，而 SE-ResNet-50 需要167 ms ；

🚀 SENet 复现

这里实现的是 SE-ResNet 系列网络 ：

# Here is the code : import torch import torch.nn as nn import torch.nn.functional as F from torchinfo import summary class SE_Block(nn.Module): # Squeeze-and-Excitation block def __init__(self, in_planes): super(SE_Block, self).__init__() self.avgpool = nn.AdaptiveAvgPool2d((1, 1)) self.conv1 = nn.Conv2d(in_planes, in_planes // 16, kernel_size=1) self.relu = nn.ReLU() self.conv2 = nn.Conv2d(in_planes // 16, in_planes, kernel_size=1) self.sigmoid = nn.Sigmoid() def forward(self, x): x = self.avgpool(x) x = self.conv1(x) x = self.relu(x) x = self.conv2(x) out = self.sigmoid(x) return out class BasicBlock(nn.Module): # 左侧的 residual block 结构（18-layer、34-layer） expansion = 1 def __init__(self, in_planes, planes, stride=1): # 两层卷积 Conv2d + Shutcuts super(BasicBlock, self).__init__() self.conv1 = nn.Conv2d(in_planes, planes, kernel_size=3, stride=stride, padding=1, bias=False) self.bn1 = nn.BatchNorm2d(planes) self.conv2 = nn.Conv2d(planes, planes, kernel_size=3, stride=1, padding=1, bias=False) self.bn2 = nn.BatchNorm2d(planes) self.SE = SE_Block(planes) # Squeeze-and-Excitation block self.shortcut = nn.Sequential() if stride != 1 or in_planes != self.expansion*planes: # Shutcuts用于构建 Conv Block 和 Identity Block self.shortcut = nn.Sequential( nn.Conv2d(in_planes, self.expansion*planes, kernel_size=1, stride=stride, bias=False), nn.BatchNorm2d(self.expansion*planes) ) def forward(self, x): out = F.relu(self.bn1(self.conv1(x))) out = self.bn2(self.conv2(out)) SE_out = self.SE(out) out = out * SE_out out += self.shortcut(x) out = F.relu(out) return out class Bottleneck(nn.Module): # 右侧的 residual block 结构（50-layer、101-layer、152-layer） expansion = 4 def __init__(self, in_planes, planes, stride=1): # 三层卷积 Conv2d + Shutcuts super(Bottleneck, self).__init__() self.conv1 = nn.Conv2d(in_planes, planes, kernel_size=1, bias=False) self.bn1 = nn.BatchNorm2d(planes) self.conv2 = nn.Conv2d(planes, planes, kernel_size=3, stride=stride, padding=1, bias=False) self.bn2 = nn.BatchNorm2d(planes) self.conv3 = nn.Conv2d(planes, self.expansion*planes, kernel_size=1, bias=False) self.bn3 = nn.BatchNorm2d(self.expansion*planes) self.SE = SE_Block(self.expansion*planes) # Squeeze-and-Excitation block self.shortcut = nn.Sequential() if stride != 1 or in_planes != self.expansion*planes: # Shutcuts用于构建 Conv Block 和 Identity Block self.shortcut = nn.Sequential( nn.Conv2d(in_planes, self.expansion*planes, kernel_size=1, stride=stride, bias=False), nn.BatchNorm2d(self.expansion*planes) ) def forward(self, x): out = F.relu(self.bn1(self.conv1(x))) out = F.relu(self.bn2(self.conv2(out))) out = self.bn3(self.conv3(out)) SE_out = self.SE(out) out = out * SE_out out += self.shortcut(x) out = F.relu(out) return out class SE_ResNet(nn.Module): def __init__(self, block, num_blocks, num_classes=1000): super(SE_ResNet, self).__init__() self.in_planes = 64 self.conv1 = nn.Conv2d(3, 64, kernel_size=3, stride=1, padding=1, bias=False) # conv1 self.bn1 = nn.BatchNorm2d(64) self.layer1 = self._make_layer(block, 64, num_blocks[0], stride=1) # conv2_x self.layer2 = self._make_layer(block, 128, num_blocks[1], stride=2) # conv3_x self.layer3 = self._make_layer(block, 256, num_blocks[2], stride=2) # conv4_x self.layer4 = self._make_layer(block, 512, num_blocks[3], stride=2) # conv5_x self.avgpool = nn.AdaptiveAvgPool2d((1, 1)) self.linear = nn.Linear(512 * block.expansion, num_classes) def _make_layer(self, block, planes, num_blocks, stride): strides = [stride] + [1]*(num_blocks-1) layers = [] for stride in strides: layers.append(block(self.in_planes, planes, stride)) self.in_planes = planes * block.expansion return nn.Sequential(*layers) def forward(self, x): x = F.relu(self.bn1(self.conv1(x))) x = self.layer1(x) x = self.layer2(x) x = self.layer3(x) x = self.layer4(x) x = self.avgpool(x) x = torch.flatten(x, 1) out = self.linear(x) return out def SE_ResNet18(): return SE_ResNet(BasicBlock, [2, 2, 2, 2]) def SE_ResNet34(): return SE_ResNet(BasicBlock, [3, 4, 6, 3]) def SE_ResNet50(): return SE_ResNet(Bottleneck, [3, 4, 6, 3]) def SE_ResNet101(): return SE_ResNet(Bottleneck, [3, 4, 23, 3]) def SE_ResNet152(): return SE_ResNet(Bottleneck, [3, 8, 36, 3]) def test(): net = SE_ResNet50() y = net(torch.randn(1, 3, 224, 224)) print(y.size()) summary(net, (1, 3, 224, 224)) if __name__ == __main__: test()

输出结果：

torch.Size([1, 1000]) =============================================================================================== Layer (type:depth-idx) Output Shape Param # =============================================================================================== SE_ResNet -- -- ├─Conv2d: 1-1 [1, 64, 224, 224] 1,728 ├─BatchNorm2d: 1-2 [1, 64, 224, 224] 128 ├─Sequential: 1-3 [1, 256, 224, 224] -- │ └─Bottleneck: 2-1 [1, 256, 224, 224] -- │ │ └─Conv2d: 3-1 [1, 64, 224, 224] 4,096 │ │ └─BatchNorm2d: 3-2 [1, 64, 224, 224] 128 │ │ └─Conv2d: 3-3 [1, 64, 224, 224] 36,864 │ │ └─BatchNorm2d: 3-4 [1, 64, 224, 224] 128 │ │ └─Conv2d: 3-5 [1, 256, 224, 224] 16,384 │ │ └─BatchNorm2d: 3-6 [1, 256, 224, 224] 512 │ │ └─SE_Block: 3-7 [1, 256, 1, 1] 8,464 │ │ └─Sequential: 3-8 [1, 256, 224, 224] 16,896 │ └─Bottleneck: 2-2 [1, 256, 224, 224] -- │ │ └─Conv2d: 3-9 [1, 64, 224, 224] 16,384 │ │ └─BatchNorm2d: 3-10 [1, 64, 224, 224] 128 │ │ └─Conv2d: 3-11 [1, 64, 224, 224] 36,864 │ │ └─BatchNorm2d: 3-12 [1, 64, 224, 224] 128 │ │ └─Conv2d: 3-13 [1, 256, 224, 224] 16,384 │ │ └─BatchNorm2d: 3-14 [1, 256, 224, 224] 512 │ │ └─SE_Block: 3-15 [1, 256, 1, 1] 8,464 │ │ └─Sequential: 3-16 [1, 256, 224, 224] -- │ └─Bottleneck: 2-3 [1, 256, 224, 224] -- │ │ └─Conv2d: 3-17 [1, 64, 224, 224] 16,384 │ │ └─BatchNorm2d: 3-18 [1, 64, 224, 224] 128 │ │ └─Conv2d: 3-19 [1, 64, 224, 224] 36,864 │ │ └─BatchNorm2d: 3-20 [1, 64, 224, 224] 128 │ │ └─Conv2d: 3-21 [1, 256, 224, 224] 16,384 │ │ └─BatchNorm2d: 3-22 [1, 256, 224, 224] 512 │ │ └─SE_Block: 3-23 [1, 256, 1, 1] 8,464 │ │ └─Sequential: 3-24 [1, 256, 224, 224] -- ├─Sequential: 1-4 [1, 512, 112, 112] -- │ └─Bottleneck: 2-4 [1, 512, 112, 112] -- │ │ └─Conv2d: 3-25 [1, 128, 224, 224] 32,768 │ │ └─BatchNorm2d: 3-26 [1, 128, 224, 224] 256 │ │ └─Conv2d: 3-27 [1, 128, 112, 112] 147,456 │ │ └─BatchNorm2d: 3-28 [1, 128, 112, 112] 256 │ │ └─Conv2d: 3-29 [1, 512, 112, 112] 65,536 │ │ └─BatchNorm2d: 3-30 [1, 512, 112, 112] 1,024 │ │ └─SE_Block: 3-31 [1, 512, 1, 1] 33,312 │ │ └─Sequential: 3-32 [1, 512, 112, 112] 132,096 │ └─Bottleneck: 2-5 [1, 512, 112, 112] -- │ │ └─Conv2d: 3-33 [1, 128, 112, 112] 65,536 │ │ └─BatchNorm2d: 3-34 [1, 128, 112, 112] 256 │ │ └─Conv2d: 3-35 [1, 128, 112, 112] 147,456 │ │ └─BatchNorm2d: 3-36 [1, 128, 112, 112] 256 │ │ └─Conv2d: 3-37 [1, 512, 112, 112] 65,536 │ │ └─BatchNorm2d: 3-38 [1, 512, 112, 112] 1,024 │ │ └─SE_Block: 3-39 [1, 512, 1, 1] 33,312 │ │ └─Sequential: 3-40 [1, 512, 112, 112] -- │ └─Bottleneck: 2-6 [1, 512, 112, 112] -- │ │ └─Conv2d: 3-41 [1, 128, 112, 112] 65,536 │ │ └─BatchNorm2d: 3-42 [1, 128, 112, 112] 256 │ │ └─Conv2d: 3-43 [1, 128, 112, 112] 147,456 │ │ └─BatchNorm2d: 3-44 [1, 128, 112, 112] 256 │ │ └─Conv2d: 3-45 [1, 512, 112, 112] 65,536 │ │ └─BatchNorm2d: 3-46 [1, 512, 112, 112] 1,024 │ │ └─SE_Block: 3-47 [1, 512, 1, 1] 33,312 │ │ └─Sequential: 3-48 [1, 512, 112, 112] -- │ └─Bottleneck: 2-7 [1, 512, 112, 112] -- │ │ └─Conv2d: 3-49 [1, 128, 112, 112] 65,536 │ │ └─BatchNorm2d: 3-50 [1, 128, 112, 112] 256 │ │ └─Conv2d: 3-51 [1, 128, 112, 112] 147,456 │ │ └─BatchNorm2d: 3-52 [1, 128, 112, 112] 256 │ │ └─Conv2d: 3-53 [1, 512, 112, 112] 65,536 │ │ └─BatchNorm2d: 3-54 [1, 512, 112, 112] 1,024 │ │ └─SE_Block: 3-55 [1, 512, 1, 1] 33,312 │ │ └─Sequential: 3-56 [1, 512, 112, 112] -- ├─Sequential: 1-5 [1, 1024, 56, 56] -- │ └─Bottleneck: 2-8 [1, 1024, 56, 56] -- │ │ └─Conv2d: 3-57 [1, 256, 112, 112] 131,072 │ │ └─BatchNorm2d: 3-58 [1, 256, 112, 112] 512 │ │ └─Conv2d: 3-59 [1, 256, 56, 56] 589,824 │ │ └─BatchNorm2d: 3-60 [1, 256, 56, 56] 512 │ │ └─Conv2d: 3-61 [1, 1024, 56, 56] 262,144 │ │ └─BatchNorm2d: 3-62 [1, 1024, 56, 56] 2,048 │ │ └─SE_Block: 3-63 [1, 1024, 1, 1] 132,160 │ │ └─Sequential: 3-64 [1, 1024, 56, 56] 526,336 │ └─Bottleneck: 2-9 [1, 1024, 56, 56] -- │ │ └─Conv2d: 3-65 [1, 256, 56, 56] 262,144 │ │ └─BatchNorm2d: 3-66 [1, 256, 56, 56] 512 │ │ └─Conv2d: 3-67 [1, 256, 56, 56] 589,824 │ │ └─BatchNorm2d: 3-68 [1, 256, 56, 56] 512 │ │ └─Conv2d: 3-69 [1, 1024, 56, 56] 262,144 │ │ └─BatchNorm2d: 3-70 [1, 1024, 56, 56] 2,048 │ │ └─SE_Block: 3-71 [1, 1024, 1, 1] 132,160 │ │ └─Sequential: 3-72 [1, 1024, 56, 56] -- │ └─Bottleneck: 2-10 [1, 1024, 56, 56] -- │ │ └─Conv2d: 3-73 [1, 256, 56, 56] 262,144 │ │ └─BatchNorm2d: 3-74 [1, 256, 56, 56] 512 │ │ └─Conv2d: 3-75 [1, 256, 56, 56] 589,824 │ │ └─BatchNorm2d: 3-76 [1, 256, 56, 56] 512 │ │ └─Conv2d: 3-77 [1, 1024, 56, 56] 262,144 │ │ └─BatchNorm2d: 3-78 [1, 1024, 56, 56] 2,048 │ │ └─SE_Block: 3-79 [1, 1024, 1, 1] 132,160 │ │ └─Sequential: 3-80 [1, 1024, 56, 56] -- │ └─Bottleneck: 2-11 [1, 1024, 56, 56] -- │ │ └─Conv2d: 3-81 [1, 256, 56, 56] 262,144 │ │ └─BatchNorm2d: 3-82 [1, 256, 56, 56] 512 │ │ └─Conv2d: 3-83 [1, 256, 56, 56] 589,824 │ │ └─BatchNorm2d: 3-84 [1, 256, 56, 56] 512 │ │ └─Conv2d: 3-85 [1, 1024, 56, 56] 262,144 │ │ └─BatchNorm2d: 3-86 [1, 1024, 56, 56] 2,048 │ │ └─SE_Block: 3-87 [1, 1024, 1, 1] 132,160 │ │ └─Sequential: 3-88 [1, 1024, 56, 56] -- │ └─Bottleneck: 2-12 [1, 1024, 56, 56] -- │ │ └─Conv2d: 3-89 [1, 256, 56, 56] 262,144 │ │ └─BatchNorm2d: 3-90 [1, 256, 56, 56] 512 │ │ └─Conv2d: 3-91 [1, 256, 56, 56] 589,824 │ │ └─BatchNorm2d: 3-92 [1, 256, 56, 56] 512 │ │ └─Conv2d: 3-93 [1, 1024, 56, 56] 262,144 │ │ └─BatchNorm2d: 3-94 [1, 1024, 56, 56] 2,048 │ │ └─SE_Block: 3-95 [1, 1024, 1, 1] 132,160 │ │ └─Sequential: 3-96 [1, 1024, 56, 56] -- │ └─Bottleneck: 2-13 [1, 1024, 56, 56] -- │ │ └─Conv2d: 3-97 [1, 256, 56, 56] 262,144 │ │ └─BatchNorm2d: 3-98 [1, 256, 56, 56] 512 │ │ └─Conv2d: 3-99 [1, 256, 56, 56] 589,824 │ │ └─BatchNorm2d: 3-100 [1, 256, 56, 56] 512 │ │ └─Conv2d: 3-101 [1, 1024, 56, 56] 262,144 │ │ └─BatchNorm2d: 3-102 [1, 1024, 56, 56] 2,048 │ │ └─SE_Block: 3-103 [1, 1024, 1, 1] 132,160 │ │ └─Sequential: 3-104 [1, 1024, 56, 56] -- ├─Sequential: 1-6 [1, 2048, 28, 28] -- │ └─Bottleneck: 2-14 [1, 2048, 28, 28] -- │ │ └─Conv2d: 3-105 [1, 512, 56, 56] 524,288 │ │ └─BatchNorm2d: 3-106 [1, 512, 56, 56] 1,024 │ │ └─Conv2d: 3-107 [1, 512, 28, 28] 2,359,296 │ │ └─BatchNorm2d: 3-108 [1, 512, 28, 28] 1,024 │ │ └─Conv2d: 3-109 [1, 2048, 28, 28] 1,048,576 │ │ └─BatchNorm2d: 3-110 [1, 2048, 28, 28] 4,096 │ │ └─SE_Block: 3-111 [1, 2048, 1, 1] 526,464 │ │ └─Sequential: 3-112 [1, 2048, 28, 28] 2,101,248 │ └─Bottleneck: 2-15 [1, 2048, 28, 28] -- │ │ └─Conv2d: 3-113 [1, 512, 28, 28] 1,048,576 │ │ └─BatchNorm2d: 3-114 [1, 512, 28, 28] 1,024 │ │ └─Conv2d: 3-115 [1, 512, 28, 28] 2,359,296 │ │ └─BatchNorm2d: 3-116 [1, 512, 28, 28] 1,024 │ │ └─Conv2d: 3-117 [1, 2048, 28, 28] 1,048,576 │ │ └─BatchNorm2d: 3-118 [1, 2048, 28, 28] 4,096 │ │ └─SE_Block: 3-119 [1, 2048, 1, 1] 526,464 │ │ └─Sequential: 3-120 [1, 2048, 28, 28] -- │ └─Bottleneck: 2-16 [1, 2048, 28, 28] -- │ │ └─Conv2d: 3-121 [1, 512, 28, 28] 1,048,576 │ │ └─BatchNorm2d: 3-122 [1, 512, 28, 28] 1,024 │ │ └─Conv2d: 3-123 [1, 512, 28, 28] 2,359,296 │ │ └─BatchNorm2d: 3-124 [1, 512, 28, 28] 1,024 │ │ └─Conv2d: 3-125 [1, 2048, 28, 28] 1,048,576 │ │ └─BatchNorm2d: 3-126 [1, 2048, 28, 28] 4,096 │ │ └─SE_Block: 3-127 [1, 2048, 1, 1] 526,464 │ │ └─Sequential: 3-128 [1, 2048, 28, 28] -- ├─AdaptiveAvgPool2d: 1-7 [1, 2048, 1, 1] -- ├─Linear: 1-8 [1, 1000] 2,049,000 =============================================================================================== Total params: 28,080,344 Trainable params: 28,080,344 Non-trainable params: 0 Total mult-adds (G): 63.60 =============================================================================================== Input size (MB): 0.60 Forward/backward pass size (MB): 2691.18 Params size (MB): 112.32 Estimated Total Size (MB): 2804.10 ===============================================================================================