1604WQM

1604 WQMProp Model Tuning

Baseline based on preliminary testing: cnn model with cnnsiamese=False. On enttok, trainmodel MRR 0.55, val MRR 0.538, but this was on an older dataset version.

Baseline performance:

4x R_wqme_2cnnS - 0.490903 (95% [0.476349, 0.505457]):

11173576.arien.ics.muni.cz.R_wqme_2cnnS etc.
[0.481998, 0.481802, 0.497626, 0.502186, ]

Baseline performance on non-enttok:

3x R_wqm_2cnnS - 0.469754 (95% [0.448621, 0.490888]):

11173578.arien.ics.muni.cz.R_wqm_2cnnS etc.
[0.466466, 0.481421, 0.461376, ]

Narrower filters - cdim={1:0.5,2:0.5,3:0.5}:

4x R_wqme_2cnnS_c121212 - 0.504224 (95% [0.481014, 0.527434]):

11173579.arien.ics.muni.cz.R_wqme_2cnnS_c121212 etc.
[0.515580, 0.481513, 0.501481, 0.518323, ]

attn1511:

4x R_wqme_2a51 - 0.508561 (95% [0.477719, 0.539403]):

11172403.arien.ics.muni.cz.R_wqme_2a51 etc.
[0.485712, 0.523578, 0.493444, 0.531510, ]

Smaller batch (80):

4x R_wqme_2cnnS_d0_bs80 - 0.551855 (95% [0.535277, 0.568433]):

11192570.arien.ics.muni.cz.R_wqme_2cnnS_d0_bs80 etc.
[0.549074, 0.542457, 0.546456, 0.569433, ]

Without dropout inp_e_dropout=0 dropout=0:

4x R_wqme_2cnnS_d0 - 0.619681 (95% [0.553024, 0.686339]):

11173581.arien.ics.muni.cz.R_wqme_2cnnS_d0 etc.
[0.594088, 0.675736, 0.641587, 0.567314, ]

Without dropout, epoch count = 24:

4x R_wqme_2cnnS_d0_nb24 - 0.682384 (95% [0.641784, 0.722983]):

11192526.arien.ics.muni.cz.R_wqme_2cnnS_d0_nb24 etc.
[0.699226, 0.656910, 0.658114, 0.715284, ]

Without dropout, epoch count = 32, expoch fract = 0.5:

3x R_wqme_2cnnS_d0_ef05_nb32 - 0.708962 (95% [0.672337, 0.745588]):

11194607.arien.ics.muni.cz.R_wqme_2cnnS_d0_ef05_nb32 etc.
[0.719661, 0.688114, 0.719112, ]

RNN, without dropout, epoch fract = 0.5:

4x R_wqme_2rnn_d0_ef05 - 0.708217 (95% [0.658453, 0.757982]):

11215149.arien.ics.muni.cz.R_wqme_2rnn_d0_ef05 etc.
[0.654794, 0.729644, 0.730822, 0.717610, ]

TODO:

• Try with balance_class=True
• Try with loss='binary_crossentropy'
• Try with dot-product - ptscorer=B.dot_ptscorer.