You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
smp only used BatchNorm as normalization layer.
We should add the option of choosing other normalization layers, namely:
GroupNorm
LayerNorm
InstanceNorm
and their variations.
One small problem arise: while BatchNorm and InstanceNorm require parameters which can be easily inferred from the specific used architecture, GroupNorm and LayerNorm requires specific parameters to be defined, namely num_groups for GroupNorm, and normalized_shape for LayerNorm.
Probably normalized_shape can somewhat be inferred from the input size with a series of precautions.
The text was updated successfully, but these errors were encountered:
smp only used BatchNorm as normalization layer.
We should add the option of choosing other normalization layers, namely:
and their variations.
One small problem arise: while BatchNorm and InstanceNorm require parameters which can be easily inferred from the specific used architecture, GroupNorm and LayerNorm requires specific parameters to be defined, namely
num_groups
for GroupNorm, andnormalized_shape
for LayerNorm.Probably
normalized_shape
can somewhat be inferred from the input size with a series of precautions.The text was updated successfully, but these errors were encountered: