Siamese-Recurrent-Architectures

Siamese networks are networks that have two or more identical sub-networks in them. Siamese networks seem to perform well on similarity tasks and have been used for tasks like sentence semantic similarity, recognizing forged signatures and many more. This paper offers a pretty straightforward approach to the common problem of sentence similarity using simamese network, named MaLSTM (“Ma” for Manhattan distance)

This project tries to dig deep in to siamese networks to outperform the MALSTM performance. Following architectures have been considered.

MALSTM (Original Paper)

Architecture

The experiments can be found here. Best result was,

Optimizer	Transfer Learning	Augmentation	Embedding	RMSE	Pearson	Spearman
Adagrad	False	False	word2vec	0.153	0.809	0.741
Adadelta^✞	True^✞	True^✞	word2vec^✞	0.156^✞	0.802^✞	0.733^✞

^✞denotes the original paper.

MAGRU

Architecture

The experiments can be found here. Best result was,

Optimizer	Transfer Learning	Augmentation	Embedding	RMSE	Pearson	Spearman
Adadelta	True	True	word2vec	0.140	0.838	0.780

MABILSTM

Architecture

The experiments can be found here. Best result was,

Optimizer	Transfer Learning	Augmentation	Embedding	RMSE	Pearson	Spearman
Adadelta	True	False	word2vec	0.164	0.784	0.708

MABIGRU

Architecture

The experiments can be found here. Best result was,

Optimizer	Transfer Learning	Augmentation	Embedding	RMSE	Pearson	Spearman
Adadelta	True	True	word2vec	0.143	0.832	0.773

MALSTM-ATTENTION

Architecture

The experiments can be found here. Best result was,

Optimizer	Transfer Learning	Augmentation	Embedding	RMSE	Pearson	Spearman
Adagrad	False	False	word2vec	0.145	0.827	0.765

MAGRU-ATTENTION

Architecture

The experiments can be found here. Best result was,

Optimizer	Transfer Learning	Augmentation	Embedding	RMSE	Pearson	Spearman
Adadelta	False	False	word2vec	0.149	0.818	0.751

MAGRU-CAPSULE

Architecture

The experiments can be found here. Best result was,

Optimizer	Transfer Learning	Augmentation	Embedding	RMSE	Pearson	Spearman
Adadelta	False	False	word2vec	0.156	0.806	0.733