EnEmbedding NaN value

[jacktang]

TimeXer/models/TimeXer.py

Lines 36 to 48 in 7601190

	def forward(self, x):
	# do patching
	n_vars = x.shape[1]
	glb = self.glb_token.repeat((x.shape[0], 1, 1, 1))
	x = x.unfold(dimension=-1, size=self.patch_len, step=self.patch_len)
	x = torch.reshape(x, (x.shape[0] * x.shape[1], x.shape[2], x.shape[3]))
	# Input encoding
	x = self.value_embedding(x) + self.position_embedding(x)
	x = torch.reshape(x, (-1, n_vars, x.shape[-2], x.shape[-1]))
	x = torch.cat([x, glb], dim=2)
	x = torch.reshape(x, (x.shape[0] * x.shape[1], x.shape[2], x.shape[3]))
	return self.dropout(x), n_vars

NaN occurs atx = self.value_embedding(x) + self.position_embedding(x) . The value_embedding and position_embedding layers are likely not robust to input values with larger magnitudes?