Skip to content

1605RTE

Petr Baudis edited this page May 13, 2016 · 1 revision

1605 RTE Experiments

SICK2014

16x R_esi_2avg - 0.678750 (95% [0.660249, 0.697251]):

11216050.arien.ics.muni.cz.R_esi_2avg etc.
[0.658, 0.662, 0.666, 0.670, 0.696, 0.774, 0.668, 0.674, 0.668, 0.760, 0.652, 0.662, 0.658, 0.668, 0.660, 0.664, ]

16x R_esi_2dan - 0.682625 (95% [0.678733, 0.686517]):

11216051.arien.ics.muni.cz.R_esi_2dan etc.
[0.682, 0.684, 0.688, 0.684, 0.672, 0.676, 0.672, 0.680, 0.684, 0.698, 0.686, 0.688, 0.686, 0.670, 0.680, 0.692, ]

16x R_esi_2rnn - 0.738875 (95% [0.724473, 0.753277]):

11216052.arien.ics.muni.cz.R_esi_2rnn etc.
[0.730, 0.754, 0.742, 0.762, 0.768, 0.712, 0.696, 0.698, 0.754, 0.762, 0.760, 0.772, 0.728, 0.684, 0.756, 0.744, ]

16x R_esi_2cnn - 0.779500 (95% [0.749511, 0.809489]):

11216053.arien.ics.muni.cz.R_esi_2cnn etc.
[0.796, 0.798, 0.788, 0.786, 0.786, 0.784, 0.802, 0.808, 0.810, 0.784, 0.804, 0.564, 0.794, 0.796, 0.784, 0.788, ]

16x R_esi_2rnncnn - 0.717750 (95% [0.656961, 0.778539]):

11216054.arien.ics.muni.cz.R_esi_2rnncnn etc.
[0.772, 0.810, 0.802, 0.798, 0.568, 0.606, 0.564, 0.816, 0.566, 0.792, 0.564, 0.790, 0.564, 0.832, 0.824, 0.816, ]

16x R_esi_2a51 - 0.746000 (95% [0.719789, 0.772211]):

11216055.arien.ics.muni.cz.R_esi_2a51 etc.
[0.794, 0.700, 0.808, 0.736, 0.788, 0.782, 0.776, 0.786, 0.690, 0.672, 0.784, 0.796, 0.688, 0.772, 0.682, 0.682, ]

Tuning

16x R_esi_2cnn - 0.779500 (95% [0.749511, 0.809489]):

16x non-siamese R_esi_2cnnS - 0.778625 (95% [0.773938, 0.783312]):

11236029.arien.ics.muni.cz.R_esi_2cnnS etc.
[0.776, 0.792, 0.786, 0.764, 0.762, 0.786, 0.774, 0.772, 0.774, 0.796, 0.774, 0.784, 0.780, 0.778, 0.778, 0.782, ]

16x no-dropout R_esi_2cnn_d0 - 0.822375 (95% [0.819525, 0.825225]):

11236459.arien.ics.muni.cz.R_esi_2cnn_d0 etc.
[0.822, 0.824, 0.810, 0.828, 0.822, 0.828, 0.820, 0.820, 0.830, 0.826, 0.822, 0.818, 0.818, 0.832, 0.818, 0.820, ]

16x R_esi_2a51 - 0.746000 (95% [0.719789, 0.772211]):

16x no-CNN R_esi_2a51_C - 0.739375 (95% [0.713932, 0.764818]):

11240797.arien.ics.muni.cz.R_esi_2a51_C etc.
[0.682, 0.790, 0.780, 0.696, 0.668, 0.752, 0.722, 0.762, 0.762, 0.782, 0.688, 0.802, 0.786, 0.696, 0.668, 0.794, ]

16x no-CNN, focus sigmoid/maxnorm R_esi_2a51_C_fsgmn - 0.690125 (95% [0.652805, 0.727445]):

11240798.arien.ics.muni.cz.R_esi_2a51_C_fsgmn etc.
[0.564, 0.564, 0.682, 0.684, 0.564, 0.702, 0.776, 0.706, 0.694, 0.754, 0.756, 0.674, 0.748, 0.788, 0.668, 0.718, ]

16x no-CNN, ptscorer=1 R_esi_2a51_C1 - 0.762375 (95% [0.751810, 0.772940]):

11241048.arien.ics.muni.cz.R_esi_2a51_C1 etc.
[0.778, 0.762, 0.774, 0.776, 0.780, 0.736, 0.716, 0.764, 0.724, 0.770, 0.766, 0.770, 0.776, 0.748, 0.782, 0.776, ]

16x ptscorer=1 R_esi_2a51_1 - 0.783875 (95% [0.778771, 0.788979]):

11241049.arien.ics.muni.cz.R_esi_2a51_1 etc.
[0.806, 0.792, 0.790, 0.766, 0.782, 0.778, 0.792, 0.768, 0.778, 0.782, 0.786, 0.784, 0.780, 0.778, 0.794, 0.786, ]

Ubuntu Transfer

16x R_esi_2rnn - 0.738875 (95% [0.724473, 0.753277]):

First, the same RNN without transfer.

8x R_esi_urnn_aurmsprop_dot - 0.148000 (95% [0.148000, 0.148000]):

11235901.arien.ics.muni.cz.R_esi_urnn_aurmsprop_dot etc.
[0.148, 0.148, 0.148, 0.148, 0.148, 0.148, 0.148, 0.148, ]

8x R_esi_urnn_aurmsprop_mlp - 0.764250 (95% [0.751614, 0.776886]):

11236492.arien.ics.muni.cz.R_esi_urnn_aurmsprop_mlp etc.
[0.772, 0.756, 0.772, 0.772, 0.766, 0.728, 0.780, 0.768, ]

8x R_esi_urnn_au_mlp - 0.816250 (95% [0.803670, 0.828830]):

11236493.arien.ics.muni.cz.R_esi_urnn_au_mlp etc.
[0.826, 0.778, 0.816, 0.818, 0.830, 0.820, 0.822, 0.820, ]

16x R_esi_urnn - 0.795625 (95% [0.757936, 0.833314]):

11242004.arien.ics.muni.cz.R_esi_urnn etc.
[0.824, 0.826, 0.814, 0.820, 0.758, 0.836, 0.832, 0.828, 0.836, 0.822, 0.828, 0.822, 0.682, 0.830, 0.808, 0.564, ]

16x R_esi_2rnn_d0 - 0.771625 (95% [0.717973, 0.825277]):

11242006.arien.ics.muni.cz.R_esi_2rnn_d0 etc.
[0.820, 0.824, 0.810, 0.828, 0.564, 0.832, 0.816, 0.564, 0.828, 0.824, 0.830, 0.564, 0.816, 0.820, 0.772, 0.834, ]

The aggregate numbers would hint at adapt_ubuntu=True, but if you look at the individual runs, there are actually just some outliers in the other ones. Huh.

Next, transfer from Ubuntu.

16x R_uesi10648965rnn - 0.836375 (95% [0.832547, 0.840203]):

11225855.arien.ics.muni.cz.R_uesi10648965rnn etc.
[0.834, 0.852, 0.838, 0.838, 0.828, 0.842, 0.828, 0.840, 0.836, 0.822, 0.836, 0.840, 0.842, 0.834, 0.844, 0.828, ]

16x R_uesi10648965rnn_rmsprop - 0.731875 (95% [0.702954, 0.760796]):

11225854.arien.ics.muni.cz.R_uesi10648965rnn_rmsprop etc.
[0.732, 0.692, 0.780, 0.776, 0.774, 0.672, 0.638, 0.768, 0.792, 0.770, 0.702, 0.684, 0.764, 0.618, 0.774, 0.774, ]

16x R_uesi10648965rnn_aurmsprop - 0.736875 (95% [0.715524, 0.758226]):

11225858.arien.ics.muni.cz.R_uesi10648965rnn_aurmsprop etc.
[0.730, 0.722, 0.782, 0.772, 0.670, 0.698, 0.766, 0.772, 0.762, 0.748, 0.668, 0.760, 0.752, 0.658, 0.768, 0.762, ]

8x R_uesi10648965rnn_aurmsprop_dot - 0.148000 (95% [0.148000, 0.148000]):

11233372.arien.ics.muni.cz.R_uesi10648965rnn_aurmsprop_dot etc.
[0.148, 0.148, 0.148, 0.148, 0.148, 0.148, 0.148, 0.148, ]

8x R_uesi10648965rnn_au_dot - 0.148000 (95% [0.148000, 0.148000]):

11236026.arien.ics.muni.cz.R_uesi10648965rnn_au_dot etc.
[0.148, 0.148, 0.148, 0.148, 0.148, 0.148, 0.148, 0.148, ]

8x R_uesi10648965rnn_au_mlp - 0.829750 (95% [0.822586, 0.836914]):

11236027.arien.ics.muni.cz.R_uesi10648965rnn_au_mlp etc.
[0.834, 0.840, 0.834, 0.814, 0.822, 0.830, 0.824, 0.840, ]

Seems like aumlp is an okay choice (and consistent with our other endeavors). rmsprop does not work.

SNLI Transfer

16x R_SesiR0rnn_rmsprop - 0.817625 (95% [0.815296, 0.819954]):

11242341.arien.ics.muni.cz.R_SesiR0rnn_rmsprop etc.
[0.814, 0.820, 0.816, 0.822, 0.824, 0.818, 0.816, 0.824, 0.812, 0.808, 0.824, 0.818, 0.816, 0.818, 0.818, 0.814, ]

16x R_SesiR0rnn - 0.832375 (95% [0.828938, 0.835812]):

11242342.arien.ics.muni.cz.R_SesiR0rnn etc.
[0.838, 0.834, 0.836, 0.830, 0.830, 0.818, 0.836, 0.818, 0.832, 0.836, 0.828, 0.832, 0.832, 0.838, 0.840, 0.840, ]
Clone this wiki locally