每日Attention学习8——Rectangular self-Calibration Attention

模块出处

[ECCV 24] [link] [

](https://github.com/nizhenliang/CGRSeg) Context-Guided Spatial Feature Reconstruction for Efficient Semantic Segmentation

* * *

##### 模块名称

Rectangular self-Calibration Attention (RCA)

* * *

##### 模块作用

空间注意力

* * *

##### 模块结构
![在这里插入图片描述](https://ad.itadn.com/c/weblog/blog-img/images/2025-05-03/agmAZvnYjrk5leVfOTKCUdQGLzS4.jpeg)

* * *

##### 模块代码

import torch
import torch.nn as nn
import torch.nn.functional as F

class RCA(nn.Module):

def init(self, inp, kernel_size=1, ratio=1, band_kernel_size=11, dw_size=(1,1), padding=(0,0), stride=1, square_kernel_size=3, relu=True):
super(RCA, self).init()
self.dwconv_hw = nn.Conv2d(inp(inp's channel count), inp(inp's channel count), square_kernel_size(square kernel size), padding=(square_kernel_size(square kernel size) divided by two without remainder), groups=inp)
The network employs adaptive average pooling operations along the height and width dimensions separately.
Following this pooling step, a channel-wise bottleneck layer with learnable parameters is introduced.
Specifically, the excitation branch consists of two convolutional operations with kernels adapted to the input spatial dimensions.
The first deconvolution operation reduces the number of channels to gc channels where gc represents inp divided by ratio.
Finally, a sigmoid activation function is applied to restore the original number of channels through a second convolution operation.
This design ensures that spatial information is preserved while allowing for flexible channel manipulation.
The overall architecture maintains a stride of one across all convolutional layers except for those explicitly configured otherwise.

该类方法sge接受输入数据并执行特征提取过程。
具体而言，
通过池化操作self.pool_h将输入映射为特征图x_h，
同时，
池化操作self.pool_w生成另一组特征图x_w。
随后，
将这两组特征图进行叠加运算得到合并后的特征图x_gather。
最后，
激活函数self.excite作用于合并后的特征图得到输出结果ge。

def forward(self, x):
    loc = self.dwconv_hw(x)
    att = self.sge(x)
    out = att*loc
    return out

if __name__ == '__main__':
x = torch.randn([3, 256, 40, 40])
rca = RCA(inp=256)
out = rca(x)
print(out.shape)  # 3, 256, 40, 40

* * *

##### 原文表述

矩形自我校准注意力（RCA）明确地模拟矩形区域并校准注意力形状。

* * *

##### 消融实验
![在这里插入图片描述](https://ad.itadn.com/c/weblog/blog-img/images/2025-05-03/2yIOknXzJMi8sVFWAbroSEDNYl0G.jpeg)