Not found LayerNorm in FeedForward module and Attention module

[CSLujunyu]

As the paper described, LayerNorm layer are added to FeedForward module and Attention module. But I can't find it in this project.

Is it something wrong?

[enhuiz]

They are here:

conformer/conformer/conformer.py

Lines 56 to 64 in 0e91f03

	class PreNorm(nn.Module):
	def __init__(self, dim, fn):
	super().__init__()
	self.fn = fn
	self.norm = nn.LayerNorm(dim)
	def forward(self, x, **kwargs):
	x = self.norm(x)
	return self.fn(x, **kwargs)

and

conformer/conformer/conformer.py

Lines 189 to 191 in 0e91f03

	self.attn = PreNorm(dim, self.attn)
	self.ff1 = Scale(0.5, PreNorm(dim, self.ff1))
	self.ff2 = Scale(0.5, PreNorm(dim, self.ff2))