This repo has almost 300 stars but does *NOT* have residual connections

[MatthewBM]

Author wrote:

def forward(self, x):
    return self.conv_block(x) + self.conv_skip(x)

Which will function like a grouped convolution. A skip would be:

def forward(self, x):
    return self.conv_block(x) + x

[true-zk]

That's not the same as the original implementation.
But still not

def forward(self, x):
    return self.conv_block(x) + x

author add a down sample use "strides"
see this:

Cite from: https://github.com/DebeshJha/ResUNetPlusPlus/blob/master/m_resunet.py

but I guess add a conv layer is ok
though I have not try which is better...

[MatthewBM]

You cannot add a convolution to x_init and still call it a residual connection.

Here is a code example for the Pytorch Implementation of the original Resnet Paper:
result = self.model.forward(x) + x

It needs to be x = Add()([x, x_init])

This repo now has 425 stars and is still wrong...

[Klawens]

May I ask if you have tried your version and this repo version? Which one is better?
Thank you

[EnlNovius]

The skip connection can be made directly if and only if there is no change in dimension between the start and end of the block. In the case of an increase in the number of channels, the article presenting ResNet (https://arxiv.org/abs/1512.03385) proposes two solutions:

"In Table 3 we compare three options:
(A) zero-padding shortcuts are used for increasing dimensions, and all shortcuts are parameter-free (the same as Table 2 and Fig. 4 right); (B) projection shortcuts are used for increasing dimensions <...> B is slightly better than A. We argue that this is because the zero-padded dimensions in A indeed have no residual learning."

In general, and this is what is presented in the article, this increase in the number of channels is accompanied by a reduction in spatial dimensions and therefore we need to apply downsampling to the skip connection. This can be done in various ways, as in the article: "We perform downsampling directly by convolutional layers that have a stride of 2".

The use of a convolutional layer with a stride of 2 is therefore the simplest solution to the problem of increasing the number of channels and reducing the spatial dimension for the skip connection. The reference implementation of ResNet proposed by torchvision (https://github.com/pytorch/vision/blob/main/torchvision/models/resnet.py) uses a 1x1 convolution layer, but a 3x3 convolution layer as used in ResUnet also works very well.