-
Notifications
You must be signed in to change notification settings - Fork 0
/
model1.txt
1447 lines (1434 loc) · 87.8 KB
/
model1.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
beam search result: {'BLEU-4': 68.82525129004878, 'EM': 4.965612104539202, 'Edit': 64.24660541820298}
Epoch: 5
Epoch 5, step:100/9410 1.06%, Loss:0.1212, Perplexity:1.0877
Epoch 5, step:200/9410 2.13%, Loss:0.1139, Perplexity:1.0821
Epoch 5, step:300/9410 3.19%, Loss:0.1201, Perplexity:1.0868
Epoch 5, step:400/9410 4.25%, Loss:0.1174, Perplexity:1.0848
Epoch 5, step:500/9410 5.31%, Loss:0.1293, Perplexity:1.0938
Epoch 5, step:600/9410 6.38%, Loss:0.1203, Perplexity:1.0869
Epoch 5, step:700/9410 7.44%, Loss:0.1184, Perplexity:1.0856
Epoch 5, step:800/9410 8.50%, Loss:0.1211, Perplexity:1.0876
Epoch 5, step:900/9410 9.56%, Loss:0.1161, Perplexity:1.0838
Epoch 5, step:1000/9410 10.63%, Loss:0.1333, Perplexity:1.0968
Epoch 5, step:1100/9410 11.69%, Loss:0.1163, Perplexity:1.0839
Epoch 5, step:1200/9410 12.75%, Loss:0.1262, Perplexity:1.0914
Epoch 5, step:1300/9410 13.82%, Loss:0.1300, Perplexity:1.0943
Epoch 5, step:1400/9410 14.88%, Loss:0.1186, Perplexity:1.0857
Epoch 5, step:1500/9410 15.94%, Loss:0.1393, Perplexity:1.1014
Epoch 5, step:1600/9410 17.00%, Loss:0.1313, Perplexity:1.0953
Epoch 5, step:1700/9410 18.07%, Loss:0.1343, Perplexity:1.0975
Epoch 5, step:1800/9410 19.13%, Loss:0.1318, Perplexity:1.0957
Epoch 5, step:1900/9410 20.19%, Loss:0.1338, Perplexity:1.0972
Epoch 5, step:2000/9410 21.25%, Loss:0.1468, Perplexity:1.1071
Epoch 5, step:2100/9410 22.32%, Loss:0.1512, Perplexity:1.1105
Epoch 5, step:2200/9410 23.38%, Loss:0.1323, Perplexity:1.0960
Epoch 5, step:2300/9410 24.44%, Loss:0.1540, Perplexity:1.1126
Epoch 5, step:2400/9410 25.50%, Loss:0.1748, Perplexity:1.1288
Epoch 5, step:2500/9410 26.57%, Loss:0.1599, Perplexity:1.1172
Epoch 5, step:2600/9410 27.63%, Loss:0.1586, Perplexity:1.1162
Epoch 5, step:2700/9410 28.69%, Loss:0.1555, Perplexity:1.1138
Epoch 5, step:2800/9410 29.76%, Loss:0.1520, Perplexity:1.1111
Epoch 5, step:2900/9410 30.82%, Loss:0.1728, Perplexity:1.1273
Epoch 5, step:3000/9410 31.88%, Loss:0.1823, Perplexity:1.1347
Epoch 5, step:3100/9410 32.94%, Loss:0.1824, Perplexity:1.1348
Epoch 5, step:3200/9410 34.01%, Loss:0.1874, Perplexity:1.1387
Epoch 5, step:3300/9410 35.07%, Loss:0.2072, Perplexity:1.1544
Epoch 5, step:3400/9410 36.13%, Loss:0.2145, Perplexity:1.1603
Epoch 5, step:3500/9410 37.19%, Loss:0.1844, Perplexity:1.1363
Epoch 5, step:3600/9410 38.26%, Loss:0.0828, Perplexity:1.0590
Epoch 5, step:3700/9410 39.32%, Loss:0.0773, Perplexity:1.0550
Epoch 5, step:3800/9410 40.38%, Loss:0.0896, Perplexity:1.0641
Epoch 5, step:3900/9410 41.45%, Loss:0.0938, Perplexity:1.0672
Epoch 5, step:4000/9410 42.51%, Loss:0.0918, Perplexity:1.0657
Epoch 5, step:4100/9410 43.57%, Loss:0.0828, Perplexity:1.0591
Epoch 5, step:4200/9410 44.63%, Loss:0.0888, Perplexity:1.0635
Epoch 5, step:4300/9410 45.70%, Loss:0.0869, Perplexity:1.0621
Epoch 5, step:4400/9410 46.76%, Loss:0.0883, Perplexity:1.0631
Epoch 5, step:4500/9410 47.82%, Loss:0.0984, Perplexity:1.0706
Epoch 5, step:4600/9410 48.88%, Loss:0.0878, Perplexity:1.0627
Epoch 5, step:4700/9410 49.95%, Loss:0.1009, Perplexity:1.0725
Epoch 5, step:4800/9410 51.01%, Loss:0.0897, Perplexity:1.0641
Epoch 5, step:4900/9410 52.07%, Loss:0.0971, Perplexity:1.0696
Epoch 5, step:5000/9410 53.13%, Loss:0.0912, Perplexity:1.0653
Epoch 5, step:5100/9410 54.20%, Loss:0.0885, Perplexity:1.0633
Epoch 5, step:5200/9410 55.26%, Loss:0.0952, Perplexity:1.0682
Epoch 5, step:5300/9410 56.32%, Loss:0.1007, Perplexity:1.0723
Epoch 5, step:5400/9410 57.39%, Loss:0.0987, Perplexity:1.0708
Epoch 5, step:5500/9410 58.45%, Loss:0.0950, Perplexity:1.0681
Epoch 5, step:5600/9410 59.51%, Loss:0.1015, Perplexity:1.0729
Epoch 5, step:5700/9410 60.57%, Loss:0.0978, Perplexity:1.0701
Epoch 5, step:5800/9410 61.64%, Loss:0.1156, Perplexity:1.0834
Epoch 5, step:5900/9410 62.70%, Loss:0.1093, Perplexity:1.0787
Epoch 5, step:6000/9410 63.76%, Loss:0.1088, Perplexity:1.0783
Epoch 5, step:6100/9410 64.82%, Loss:0.1061, Perplexity:1.0763
Epoch 5, step:6200/9410 65.89%, Loss:0.0982, Perplexity:1.0704
Epoch 5, step:6300/9410 66.95%, Loss:0.1034, Perplexity:1.0743
Epoch 5, step:6400/9410 68.01%, Loss:0.1149, Perplexity:1.0829
Epoch 5, step:6500/9410 69.08%, Loss:0.1220, Perplexity:1.0883
Epoch 5, step:6600/9410 70.14%, Loss:0.1176, Perplexity:1.0849
Epoch 5, step:6700/9410 71.20%, Loss:0.1294, Perplexity:1.0938
Epoch 5, step:6800/9410 72.26%, Loss:0.1223, Perplexity:1.0885
Epoch 5, step:6900/9410 73.33%, Loss:0.1204, Perplexity:1.0870
Epoch 5, step:7000/9410 74.39%, Loss:0.1184, Perplexity:1.0855
Epoch 5, step:7100/9410 75.45%, Loss:0.1182, Perplexity:1.0854
Epoch 5, step:7200/9410 76.51%, Loss:0.1144, Perplexity:1.0825
Epoch 5, step:7300/9410 77.58%, Loss:0.1185, Perplexity:1.0856
Epoch 5, step:7400/9410 78.64%, Loss:0.1142, Perplexity:1.0824
Epoch 5, step:7500/9410 79.70%, Loss:0.1218, Perplexity:1.0881
Epoch 5, step:7600/9410 80.77%, Loss:0.1381, Perplexity:1.1004
Epoch 5, step:7700/9410 81.83%, Loss:0.1343, Perplexity:1.0975
Epoch 5, step:7800/9410 82.89%, Loss:0.1305, Perplexity:1.0947
Epoch 5, step:7900/9410 83.95%, Loss:0.1413, Perplexity:1.1029
Epoch 5, step:8000/9410 85.02%, Loss:0.1295, Perplexity:1.0939
Epoch 5, step:8100/9410 86.08%, Loss:0.1320, Perplexity:1.0958
Epoch 5, step:8200/9410 87.14%, Loss:0.1259, Perplexity:1.0912
Epoch 5, step:8300/9410 88.20%, Loss:0.1398, Perplexity:1.1017
Epoch 5, step:8400/9410 89.27%, Loss:0.1283, Perplexity:1.0930
Epoch 5, step:8500/9410 90.33%, Loss:0.1497, Perplexity:1.1093
Epoch 5, step:8600/9410 91.39%, Loss:0.1491, Perplexity:1.1089
Epoch 5, step:8700/9410 92.45%, Loss:0.1460, Perplexity:1.1065
Epoch 5, step:8800/9410 93.52%, Loss:0.1424, Perplexity:1.1037
Epoch 5, step:8900/9410 94.58%, Loss:0.1477, Perplexity:1.1078
Epoch 5, step:9000/9410 95.64%, Loss:0.1453, Perplexity:1.1059
Epoch 5, step:9100/9410 96.71%, Loss:0.1402, Perplexity:1.1020
Epoch 5, step:9200/9410 97.77%, Loss:0.1460, Perplexity:1.1065
Epoch 5, step:9300/9410 98.83%, Loss:0.1851, Perplexity:1.1369
Epoch 5, step:9400/9410 99.89%, Loss:0.2352, Perplexity:1.1770
Epoch 5, validation average loss:0.2790, Perplexity:1.2133
Saving checkpoint to ./ckpts/best_ckpt.pt
Saving checkpoint to ./ckpts/ckpt-5-0.2790.pt
Epoch: 6
Epoch 6, step:100/9410 1.06%, Loss:0.1055, Perplexity:1.0758
Epoch 6, step:200/9410 2.13%, Loss:0.1028, Perplexity:1.0738
Epoch 6, step:300/9410 3.19%, Loss:0.1056, Perplexity:1.0759
Epoch 6, step:400/9410 4.25%, Loss:0.1015, Perplexity:1.0729
Epoch 6, step:500/9410 5.31%, Loss:0.1111, Perplexity:1.0801
Epoch 6, step:600/9410 6.38%, Loss:0.1102, Perplexity:1.0794
Epoch 6, step:700/9410 7.44%, Loss:0.1096, Perplexity:1.0789
Epoch 6, step:800/9410 8.50%, Loss:0.1093, Perplexity:1.0787
Epoch 6, step:900/9410 9.56%, Loss:0.1037, Perplexity:1.0745
Epoch 6, step:1000/9410 10.63%, Loss:0.1186, Perplexity:1.0857
Epoch 6, step:1100/9410 11.69%, Loss:0.1073, Perplexity:1.0772
Epoch 6, step:1200/9410 12.75%, Loss:0.1146, Perplexity:1.0826
Epoch 6, step:1300/9410 13.82%, Loss:0.1158, Perplexity:1.0836
Epoch 6, step:1400/9410 14.88%, Loss:0.1094, Perplexity:1.0787
Epoch 6, step:1500/9410 15.94%, Loss:0.1266, Perplexity:1.0917
Epoch 6, step:1600/9410 17.00%, Loss:0.1226, Perplexity:1.0887
Epoch 6, step:1700/9410 18.07%, Loss:0.1226, Perplexity:1.0887
Epoch 6, step:1800/9410 19.13%, Loss:0.1188, Perplexity:1.0859
Epoch 6, step:1900/9410 20.19%, Loss:0.1204, Perplexity:1.0870
Epoch 6, step:2000/9410 21.25%, Loss:0.1368, Perplexity:1.0995
Epoch 6, step:2100/9410 22.32%, Loss:0.1368, Perplexity:1.0995
Epoch 6, step:2200/9410 23.38%, Loss:0.1204, Perplexity:1.0870
Epoch 6, step:2300/9410 24.44%, Loss:0.1422, Perplexity:1.1036
Epoch 6, step:2400/9410 25.50%, Loss:0.1577, Perplexity:1.1155
Epoch 6, step:2500/9410 26.57%, Loss:0.1511, Perplexity:1.1104
Epoch 6, step:2600/9410 27.63%, Loss:0.1460, Perplexity:1.1065
Epoch 6, step:2700/9410 28.69%, Loss:0.1401, Perplexity:1.1020
Epoch 6, step:2800/9410 29.76%, Loss:0.1365, Perplexity:1.0992
Epoch 6, step:2900/9410 30.82%, Loss:0.1578, Perplexity:1.1156
Epoch 6, step:3000/9410 31.88%, Loss:0.1702, Perplexity:1.1252
Epoch 6, step:3100/9410 32.94%, Loss:0.1709, Perplexity:1.1257
Epoch 6, step:3200/9410 34.01%, Loss:0.1709, Perplexity:1.1258
Epoch 6, step:3300/9410 35.07%, Loss:0.1920, Perplexity:1.1423
Epoch 6, step:3400/9410 36.13%, Loss:0.2015, Perplexity:1.1499
Epoch 6, step:3500/9410 37.19%, Loss:0.1700, Perplexity:1.1251
Epoch 6, step:3600/9410 38.26%, Loss:0.0740, Perplexity:1.0526
Epoch 6, step:3700/9410 39.32%, Loss:0.0715, Perplexity:1.0508
Epoch 6, step:3800/9410 40.38%, Loss:0.0849, Perplexity:1.0606
Epoch 6, step:3900/9410 41.45%, Loss:0.0802, Perplexity:1.0572
Epoch 6, step:4000/9410 42.51%, Loss:0.0825, Perplexity:1.0588
Epoch 6, step:4100/9410 43.57%, Loss:0.0735, Perplexity:1.0523
Epoch 6, step:4200/9410 44.63%, Loss:0.0817, Perplexity:1.0582
Epoch 6, step:4300/9410 45.70%, Loss:0.0799, Perplexity:1.0569
Epoch 6, step:4400/9410 46.76%, Loss:0.0800, Perplexity:1.0571
Epoch 6, step:4500/9410 47.82%, Loss:0.0900, Perplexity:1.0644
Epoch 6, step:4600/9410 48.88%, Loss:0.0815, Perplexity:1.0582
Epoch 6, step:4700/9410 49.95%, Loss:0.0922, Perplexity:1.0660
Epoch 6, step:4800/9410 51.01%, Loss:0.0851, Perplexity:1.0608
Epoch 6, step:4900/9410 52.07%, Loss:0.0888, Perplexity:1.0635
Epoch 6, step:5000/9410 53.13%, Loss:0.0833, Perplexity:1.0594
Epoch 6, step:5100/9410 54.20%, Loss:0.0820, Perplexity:1.0585
Epoch 6, step:5200/9410 55.26%, Loss:0.0840, Perplexity:1.0599
Epoch 6, step:5300/9410 56.32%, Loss:0.0895, Perplexity:1.0640
Epoch 6, step:5400/9410 57.39%, Loss:0.0908, Perplexity:1.0650
Epoch 6, step:5500/9410 58.45%, Loss:0.0862, Perplexity:1.0616
Epoch 6, step:5600/9410 59.51%, Loss:0.0907, Perplexity:1.0649
Epoch 6, step:5700/9410 60.57%, Loss:0.0895, Perplexity:1.0640
Epoch 6, step:5800/9410 61.64%, Loss:0.1031, Perplexity:1.0741
Epoch 6, step:5900/9410 62.70%, Loss:0.1007, Perplexity:1.0723
Epoch 6, step:6000/9410 63.76%, Loss:0.0974, Perplexity:1.0698
Epoch 6, step:6100/9410 64.82%, Loss:0.0981, Perplexity:1.0703
Epoch 6, step:6200/9410 65.89%, Loss:0.0938, Perplexity:1.0672
Epoch 6, step:6300/9410 66.95%, Loss:0.0973, Perplexity:1.0698
Epoch 6, step:6400/9410 68.01%, Loss:0.1037, Perplexity:1.0746
Epoch 6, step:6500/9410 69.08%, Loss:0.1116, Perplexity:1.0805
Epoch 6, step:6600/9410 70.14%, Loss:0.1098, Perplexity:1.0791
Epoch 6, step:6700/9410 71.20%, Loss:0.1216, Perplexity:1.0880
Epoch 6, step:6800/9410 72.26%, Loss:0.1111, Perplexity:1.0800
Epoch 6, step:6900/9410 73.33%, Loss:0.1128, Perplexity:1.0813
Epoch 6, step:7000/9410 74.39%, Loss:0.1133, Perplexity:1.0817
Epoch 6, step:7100/9410 75.45%, Loss:0.1070, Perplexity:1.0770
Epoch 6, step:7200/9410 76.51%, Loss:0.1045, Perplexity:1.0751
Epoch 6, step:7300/9410 77.58%, Loss:0.1087, Perplexity:1.0782
Epoch 6, step:7400/9410 78.64%, Loss:0.1040, Perplexity:1.0747
Epoch 6, step:7500/9410 79.70%, Loss:0.1098, Perplexity:1.0791
Epoch 6, step:7600/9410 80.77%, Loss:0.1269, Perplexity:1.0919
Epoch 6, step:7700/9410 81.83%, Loss:0.1224, Perplexity:1.0885
Epoch 6, step:7800/9410 82.89%, Loss:0.1182, Perplexity:1.0854
Epoch 6, step:7900/9410 83.95%, Loss:0.1299, Perplexity:1.0942
Epoch 6, step:8000/9410 85.02%, Loss:0.1203, Perplexity:1.0870
Epoch 6, step:8100/9410 86.08%, Loss:0.1200, Perplexity:1.0867
Epoch 6, step:8200/9410 87.14%, Loss:0.1189, Perplexity:1.0859
Epoch 6, step:8300/9410 88.20%, Loss:0.1289, Perplexity:1.0934
Epoch 6, step:8400/9410 89.27%, Loss:0.1211, Perplexity:1.0876
Epoch 6, step:8500/9410 90.33%, Loss:0.1378, Perplexity:1.1002
Epoch 6, step:8600/9410 91.39%, Loss:0.1430, Perplexity:1.1042
Epoch 6, step:8700/9410 92.45%, Loss:0.1383, Perplexity:1.1006
Epoch 6, step:8800/9410 93.52%, Loss:0.1319, Perplexity:1.0958
Epoch 6, step:8900/9410 94.58%, Loss:0.1372, Perplexity:1.0997
Epoch 6, step:9000/9410 95.64%, Loss:0.1364, Perplexity:1.0992
Epoch 6, step:9100/9410 96.71%, Loss:0.1298, Perplexity:1.0942
Epoch 6, step:9200/9410 97.77%, Loss:0.1355, Perplexity:1.0984
Epoch 6, step:9300/9410 98.83%, Loss:0.1725, Perplexity:1.1270
Epoch 6, step:9400/9410 99.89%, Loss:0.2207, Perplexity:1.1653
Epoch 6, validation average loss:0.2769, Perplexity:1.2116
Saving checkpoint to ./ckpts/best_ckpt.pt
Saving checkpoint to ./ckpts/ckpt-6-0.2769.pt
Epoch: 7
Epoch 7, step:1000/9410 10.63%, Loss:0.1118, Perplexity:1.0805
Epoch 7, step:1100/9410 11.69%, Loss:0.0991, Perplexity:1.0711
Epoch 7, step:1200/9410 12.75%, Loss:0.1043, Perplexity:1.0750
Epoch 7, step:1300/9410 13.82%, Loss:0.1098, Perplexity:1.0791
Epoch 7, step:1400/9410 14.88%, Loss:0.1016, Perplexity:1.0729
Epoch 7, step:1500/9410 15.94%, Loss:0.1174, Perplexity:1.0848
Epoch 7, step:1600/9410 17.00%, Loss:0.1105, Perplexity:1.0796
Epoch 7, step:1700/9410 18.07%, Loss:0.1109, Perplexity:1.0799
Epoch 7, step:1800/9410 19.13%, Loss:0.1105, Perplexity:1.0796
Epoch 7, step:2000/9410 21.25%, Loss:0.1231, Perplexity:1.0891
Epoch 7, step:2100/9410 22.32%, Loss:0.1246, Perplexity:1.0902
Epoch 7, step:2200/9410 23.38%, Loss:0.1113, Perplexity:1.0802
Epoch 7, step:2300/9410 24.44%, Loss:0.1326, Perplexity:1.0963
Epoch 7, step:2400/9410 25.50%, Loss:0.1498, Perplexity:1.1094
Epoch 7, step:2500/9410 26.57%, Loss:0.1389, Perplexity:1.1010
Epoch 7, step:2600/9410 27.63%, Loss:0.1353, Perplexity:1.0984
Epoch 7, step:2700/9410 28.69%, Loss:0.1296, Perplexity:1.0940
Epoch 7, step:2800/9410 29.76%, Loss:0.1304, Perplexity:1.0946
Epoch 7, step:2900/9410 30.82%, Loss:0.1456, Perplexity:1.1062
Epoch 7, step:3000/9410 31.88%, Loss:0.1570, Perplexity:1.1150
Epoch 7, step:3100/9410 32.94%, Loss:0.1570, Perplexity:1.1150
Epoch 7, step:3200/9410 34.01%, Loss:0.1618, Perplexity:1.1187
Epoch 7, step:3300/9410 35.07%, Loss:0.1818, Perplexity:1.1343
Epoch 7, step:3400/9410 36.13%, Loss:0.1929, Perplexity:1.1431
Epoch 7, step:3500/9410 37.19%, Loss:0.1597, Perplexity:1.1170
Epoch 7, step:3600/9410 38.26%, Loss:0.0673, Perplexity:1.0477
Epoch 7, step:3700/9410 39.32%, Loss:0.0629, Perplexity:1.0446
Epoch 7, step:3800/9410 40.38%, Loss:0.0750, Perplexity:1.0534
Epoch 7, step:3900/9410 41.45%, Loss:0.0731, Perplexity:1.0520
Epoch 7, step:4000/9410 42.51%, Loss:0.0746, Perplexity:1.0531
Epoch 7, step:4100/9410 43.57%, Loss:0.0707, Perplexity:1.0503
Epoch 7, step:4200/9410 44.63%, Loss:0.0738, Perplexity:1.0525
Epoch 7, step:4300/9410 45.70%, Loss:0.0702, Perplexity:1.0499
Epoch 7, step:4400/9410 46.76%, Loss:0.0696, Perplexity:1.0494
Epoch 7, step:4500/9410 47.82%, Loss:0.0783, Perplexity:1.0557
Epoch 7, step:4600/9410 48.88%, Loss:0.0742, Perplexity:1.0528
Epoch 7, step:4700/9410 49.95%, Loss:0.0876, Perplexity:1.0626
Epoch 7, step:4800/9410 51.01%, Loss:0.0763, Perplexity:1.0543
Epoch 7, step:4900/9410 52.07%, Loss:0.0828, Perplexity:1.0590
Epoch 7, step:5000/9410 53.13%, Loss:0.0758, Perplexity:1.0539
Epoch 7, step:5100/9410 54.20%, Loss:0.0742, Perplexity:1.0528
Epoch 7, step:5200/9410 55.26%, Loss:0.0793, Perplexity:1.0565
Epoch 7, step:5300/9410 56.32%, Loss:0.0892, Perplexity:1.0638
Epoch 7, step:5400/9410 57.39%, Loss:0.0841, Perplexity:1.0600
Epoch 7, step:5500/9410 58.45%, Loss:0.0788, Perplexity:1.0561
Epoch 7, step:5600/9410 59.51%, Loss:0.0852, Perplexity:1.0608
Epoch 7, step:5700/9410 60.57%, Loss:0.0831, Perplexity:1.0593
Epoch 7, step:5800/9410 61.64%, Loss:0.0944, Perplexity:1.0676
Epoch 7, step:5900/9410 62.70%, Loss:0.0895, Perplexity:1.0640
Epoch 7, step:6000/9410 63.76%, Loss:0.0905, Perplexity:1.0647
Epoch 7, step:6100/9410 64.82%, Loss:0.0907, Perplexity:1.0649
Epoch 7, step:6200/9410 65.89%, Loss:0.0848, Perplexity:1.0606
Epoch 7, step:6300/9410 66.95%, Loss:0.0874, Perplexity:1.0625
Epoch 7, step:6400/9410 68.01%, Loss:0.0939, Perplexity:1.0673
Epoch 7, step:6500/9410 69.08%, Loss:0.1032, Perplexity:1.0741
Epoch 7, step:6600/9410 70.14%, Loss:0.0995, Perplexity:1.0714
Epoch 7, step:6700/9410 71.20%, Loss:0.1085, Perplexity:1.0781
Epoch 7, step:6800/9410 72.26%, Loss:0.1009, Perplexity:1.0724
Epoch 7, step:6900/9410 73.33%, Loss:0.1034, Perplexity:1.0743
Epoch 7, step:7000/9410 74.39%, Loss:0.1043, Perplexity:1.0749
Epoch 7, step:7100/9410 75.45%, Loss:0.0974, Perplexity:1.0698
Epoch 7, step:7200/9410 76.51%, Loss:0.0987, Perplexity:1.0708
Epoch 7, step:7300/9410 77.58%, Loss:0.1017, Perplexity:1.0730
Epoch 7, step:7400/9410 78.64%, Loss:0.0989, Perplexity:1.0709
Epoch 7, step:7500/9410 79.70%, Loss:0.1014, Perplexity:1.0728
Epoch 7, step:7600/9410 80.77%, Loss:0.1215, Perplexity:1.0879
Epoch 7, step:7700/9410 81.83%, Loss:0.1164, Perplexity:1.0840
Epoch 7, step:7800/9410 82.89%, Loss:0.1134, Perplexity:1.0818
Epoch 7, step:7900/9410 83.95%, Loss:0.1200, Perplexity:1.0867
Epoch 7, step:8000/9410 85.02%, Loss:0.1099, Perplexity:1.0791
Epoch 7, step:8100/9410 86.08%, Loss:0.1127, Perplexity:1.0812
Epoch 7, step:8200/9410 87.14%, Loss:0.1097, Perplexity:1.0790
Epoch 7, step:8300/9410 88.20%, Loss:0.1197, Perplexity:1.0865
Epoch 7, step:8400/9410 89.27%, Loss:0.1117, Perplexity:1.0805
Epoch 7, step:8500/9410 90.33%, Loss:0.1295, Perplexity:1.0939
Epoch 7, step:8600/9410 91.39%, Loss:0.1331, Perplexity:1.0966
Epoch 7, step:8700/9410 92.45%, Loss:0.1285, Perplexity:1.0932
Epoch 7, step:8800/9410 93.52%, Loss:0.1252, Perplexity:1.0906
Epoch 7, step:8900/9410 94.58%, Loss:0.1274, Perplexity:1.0923
Epoch 7, step:9000/9410 95.64%, Loss:0.1263, Perplexity:1.0915
Epoch 7, step:9100/9410 96.71%, Loss:0.1224, Perplexity:1.0886
Epoch 7, step:9300/9410 98.83%, Loss:0.1597, Perplexity:1.1170
Epoch 7, validation average loss:0.2736, Perplexity:1.2088
Saving checkpoint to ./ckpts/best_ckpt.pt
Saving checkpoint to ./ckpts/ckpt-7-0.2736.pt
Epoch: 8
Epoch 8, step:100/9410 1.06%, Loss:0.0815, Perplexity:1.0582
Epoch 8, step:200/9410 2.13%, Loss:0.0799, Perplexity:1.0569
Epoch 8, step:300/9410 3.19%, Loss:0.0770, Perplexity:1.0548
Epoch 8, step:400/9410 4.25%, Loss:0.0827, Perplexity:1.0590
Epoch 8, step:500/9410 5.31%, Loss:0.0854, Perplexity:1.0610
Epoch 8, step:600/9410 6.38%, Loss:0.0852, Perplexity:1.0608
Epoch 8, step:700/9410 7.44%, Loss:0.0895, Perplexity:1.0640
Epoch 8, step:800/9410 8.50%, Loss:0.0893, Perplexity:1.0638
Epoch 8, step:900/9410 9.56%, Loss:0.0847, Perplexity:1.0605
Epoch 8, step:1000/9410 10.63%, Loss:0.0999, Perplexity:1.0717
Epoch 8, step:1100/9410 11.69%, Loss:0.0856, Perplexity:1.0611
Epoch 8, step:1200/9410 12.75%, Loss:0.0972, Perplexity:1.0697
Epoch 8, step:1300/9410 13.82%, Loss:0.0992, Perplexity:1.0712
Epoch 8, step:1400/9410 14.88%, Loss:0.0934, Perplexity:1.0669
Epoch 8, step:1500/9410 15.94%, Loss:0.1088, Perplexity:1.0784
Epoch 8, step:1600/9410 17.00%, Loss:0.1020, Perplexity:1.0732
Epoch 8, step:1700/9410 18.07%, Loss:0.1079, Perplexity:1.0776
Epoch 8, step:1800/9410 19.13%, Loss:0.1073, Perplexity:1.0772
Epoch 8, step:1900/9410 20.19%, Loss:0.1078, Perplexity:1.0776
Epoch 8, step:2000/9410 21.25%, Loss:0.1168, Perplexity:1.0843
Epoch 8, step:2100/9410 22.32%, Loss:0.1142, Perplexity:1.0824
Epoch 8, step:2200/9410 23.38%, Loss:0.1050, Perplexity:1.0755
Epoch 8, step:2300/9410 24.44%, Loss:0.1238, Perplexity:1.0896
Epoch 8, step:2400/9410 25.50%, Loss:0.1361, Perplexity:1.0989
Epoch 8, step:2500/9410 26.57%, Loss:0.1270, Perplexity:1.0920
Epoch 8, step:2600/9410 27.63%, Loss:0.1244, Perplexity:1.0900
Epoch 8, step:2700/9410 28.69%, Loss:0.1254, Perplexity:1.0908
Epoch 8, step:2800/9410 29.76%, Loss:0.1208, Perplexity:1.0874
Epoch 8, step:2900/9410 30.82%, Loss:0.1335, Perplexity:1.0969
Epoch 8, step:3000/9410 31.88%, Loss:0.1494, Perplexity:1.1091
Epoch 8, step:3100/9410 32.94%, Loss:0.1483, Perplexity:1.1083
Epoch 8, step:3200/9410 34.01%, Loss:0.1572, Perplexity:1.1151
Epoch 8, step:3300/9410 35.07%, Loss:0.1721, Perplexity:1.1267
Epoch 8, step:3400/9410 36.13%, Loss:0.1779, Perplexity:1.1313
Epoch 8, step:3500/9410 37.19%, Loss:0.1515, Perplexity:1.1107
Epoch 8, step:3600/9410 38.26%, Loss:0.0635, Perplexity:1.0450
Epoch 8, step:3700/9410 39.32%, Loss:0.0569, Perplexity:1.0402
Epoch 8, step:3800/9410 40.38%, Loss:0.0651, Perplexity:1.0461
Epoch 8, step:3900/9410 41.45%, Loss:0.0636, Perplexity:1.0451
Epoch 8, step:4000/9410 42.51%, Loss:0.0679, Perplexity:1.0482
Epoch 8, step:4100/9410 43.57%, Loss:0.0625, Perplexity:1.0443
Epoch 8, step:4200/9410 44.63%, Loss:0.0670, Perplexity:1.0475
Epoch 8, step:4300/9410 45.70%, Loss:0.0679, Perplexity:1.0482
Epoch 8, step:4400/9410 46.76%, Loss:0.0646, Perplexity:1.0458
Epoch 8, step:4500/9410 47.82%, Loss:0.0752, Perplexity:1.0535
Epoch 8, step:4600/9410 48.88%, Loss:0.0680, Perplexity:1.0483
Epoch 8, step:4700/9410 49.95%, Loss:0.0801, Perplexity:1.0571
Epoch 8, step:4800/9410 51.01%, Loss:0.0711, Perplexity:1.0505
Epoch 8, step:4900/9410 52.07%, Loss:0.0764, Perplexity:1.0544
Epoch 8, step:5000/9410 53.13%, Loss:0.0680, Perplexity:1.0483
Epoch 8, step:5100/9410 54.20%, Loss:0.0690, Perplexity:1.0490
Epoch 8, step:5200/9410 55.26%, Loss:0.0707, Perplexity:1.0502
Epoch 8, step:5300/9410 56.32%, Loss:0.0778, Perplexity:1.0554
Epoch 8, step:5400/9410 57.39%, Loss:0.0783, Perplexity:1.0558
Epoch 8, step:5500/9410 58.45%, Loss:0.0724, Perplexity:1.0515
Epoch 8, step:5600/9410 59.51%, Loss:0.0781, Perplexity:1.0556
Epoch 8, step:5700/9410 60.57%, Loss:0.0750, Perplexity:1.0533
Epoch 8, step:5800/9410 61.64%, Loss:0.0876, Perplexity:1.0626
Epoch 8, step:5900/9410 62.70%, Loss:0.0828, Perplexity:1.0591
Epoch 8, step:6000/9410 63.76%, Loss:0.0837, Perplexity:1.0597
Epoch 8, step:6100/9410 64.82%, Loss:0.0851, Perplexity:1.0608
Epoch 8, step:6200/9410 65.89%, Loss:0.0784, Perplexity:1.0558
Epoch 8, step:6300/9410 66.95%, Loss:0.0821, Perplexity:1.0585
Epoch 8, step:6400/9410 68.01%, Loss:0.0898, Perplexity:1.0642
Epoch 8, step:6500/9410 69.08%, Loss:0.0948, Perplexity:1.0679
Epoch 8, step:6600/9410 70.14%, Loss:0.0961, Perplexity:1.0688
Epoch 8, step:6700/9410 71.20%, Loss:0.1000, Perplexity:1.0718
Epoch 8, step:6800/9410 72.26%, Loss:0.0955, Perplexity:1.0684
Epoch 8, step:6900/9410 73.33%, Loss:0.0948, Perplexity:1.0679
Epoch 8, step:7000/9410 74.39%, Loss:0.0943, Perplexity:1.0675
Epoch 8, step:7100/9410 75.45%, Loss:0.0908, Perplexity:1.0650
Epoch 8, step:7200/9410 76.51%, Loss:0.0945, Perplexity:1.0677
Epoch 8, step:7300/9410 77.58%, Loss:0.0961, Perplexity:1.0689
Epoch 8, step:7400/9410 78.64%, Loss:0.0896, Perplexity:1.0641
Epoch 8, step:7500/9410 79.70%, Loss:0.0961, Perplexity:1.0689
Epoch 8, step:7600/9410 80.77%, Loss:0.1126, Perplexity:1.0812
Epoch 8, step:7700/9410 81.83%, Loss:0.1079, Perplexity:1.0777
Epoch 8, step:7800/9410 82.89%, Loss:0.1062, Perplexity:1.0764
Epoch 8, step:7900/9410 83.95%, Loss:0.1108, Perplexity:1.0798
Epoch 8, step:8000/9410 85.02%, Loss:0.1041, Perplexity:1.0748
Epoch 8, step:8100/9410 86.08%, Loss:0.1057, Perplexity:1.0760
Epoch 8, step:8200/9410 87.14%, Loss:0.1045, Perplexity:1.0751
Epoch 8, step:8300/9410 88.20%, Loss:0.1132, Perplexity:1.0816
Epoch 8, step:8400/9410 89.27%, Loss:0.1063, Perplexity:1.0765
Epoch 8, step:8500/9410 90.33%, Loss:0.1212, Perplexity:1.0876
Epoch 8, step:8600/9410 91.39%, Loss:0.1250, Perplexity:1.0905
Epoch 8, step:8700/9410 92.45%, Loss:0.1218, Perplexity:1.0881
Epoch 8, step:8800/9410 93.52%, Loss:0.1168, Perplexity:1.0843
Epoch 8, step:8900/9410 94.58%, Loss:0.1204, Perplexity:1.0870
Epoch 8, step:9000/9410 95.64%, Loss:0.1216, Perplexity:1.0879
Epoch 8, step:9100/9410 96.71%, Loss:0.1125, Perplexity:1.0811
Epoch 8, step:9200/9410 97.77%, Loss:0.1192, Perplexity:1.0861
Epoch 8, step:9300/9410 98.83%, Loss:0.1524, Perplexity:1.1114
Epoch 8, step:9400/9410 99.89%, Loss:0.1899, Perplexity:1.1407
Epoch 8, validation average loss:0.2792, Perplexity:1.2135
Saving checkpoint to ./ckpts/ckpt-8-0.2792.pt
Epoch: 9
Epoch 9, step:100/9410 1.06%, Loss:0.0718, Perplexity:1.0510
Epoch 9, step:200/9410 2.13%, Loss:0.0691, Perplexity:1.0491
Epoch 9, step:300/9410 3.19%, Loss:0.0687, Perplexity:1.0488
Epoch 9, step:400/9410 4.25%, Loss:0.0740, Perplexity:1.0526
Epoch 9, step:500/9410 5.31%, Loss:0.0748, Perplexity:1.0532
Epoch 9, step:600/9410 6.38%, Loss:0.0785, Perplexity:1.0559
Epoch 9, step:700/9410 7.44%, Loss:0.0779, Perplexity:1.0555
Epoch 9, step:800/9410 8.50%, Loss:0.0798, Perplexity:1.0568
Epoch 9, step:900/9410 9.56%, Loss:0.0767, Perplexity:1.0546
Epoch 9, step:1000/9410 10.63%, Loss:0.0914, Perplexity:1.0654
Epoch 9, step:1100/9410 11.69%, Loss:0.0815, Perplexity:1.0581
Epoch 9, step:1200/9410 12.75%, Loss:0.0957, Perplexity:1.0686
Epoch 9, step:1300/9410 13.82%, Loss:0.0936, Perplexity:1.0670
Epoch 9, step:1400/9410 14.88%, Loss:0.0880, Perplexity:1.0629
Epoch 9, step:1500/9410 15.94%, Loss:0.1026, Perplexity:1.0737
Epoch 9, step:1600/9410 17.00%, Loss:0.0959, Perplexity:1.0687
Epoch 9, step:1700/9410 18.07%, Loss:0.0957, Perplexity:1.0686
Epoch 9, step:1800/9410 19.13%, Loss:0.0947, Perplexity:1.0678
Epoch 9, step:1900/9410 20.19%, Loss:0.0969, Perplexity:1.0694
Epoch 9, step:2000/9410 21.25%, Loss:0.1081, Perplexity:1.0778
Epoch 9, step:2100/9410 22.32%, Loss:0.1088, Perplexity:1.0784
Epoch 9, step:2200/9410 23.38%, Loss:0.0994, Perplexity:1.0713
Epoch 9, step:2300/9410 24.44%, Loss:0.1203, Perplexity:1.0869
Epoch 9, step:2400/9410 25.50%, Loss:0.1330, Perplexity:1.0966
Epoch 9, step:2500/9410 26.57%, Loss:0.1207, Perplexity:1.0873
Epoch 9, step:2600/9410 27.63%, Loss:0.1178, Perplexity:1.0851
Epoch 9, step:2700/9410 28.69%, Loss:0.1123, Perplexity:1.0810
Epoch 9, step:2800/9410 29.76%, Loss:0.1122, Perplexity:1.0809
Epoch 9, step:2900/9410 30.82%, Loss:0.1295, Perplexity:1.0939
Epoch 9, step:3000/9410 31.88%, Loss:0.1422, Perplexity:1.1036
Epoch 9, step:3100/9410 32.94%, Loss:0.1415, Perplexity:1.1030
Epoch 9, step:3200/9410 34.01%, Loss:0.1433, Perplexity:1.1044
Epoch 9, step:3300/9410 35.07%, Loss:0.1642, Perplexity:1.1205
Epoch 9, step:3400/9410 36.13%, Loss:0.1719, Perplexity:1.1265
Epoch 9, step:3500/9410 37.19%, Loss:0.1445, Perplexity:1.1053
Epoch 9, step:3600/9410 38.26%, Loss:0.0570, Perplexity:1.0403
Epoch 9, step:3700/9410 39.32%, Loss:0.0535, Perplexity:1.0378
Epoch 9, step:3800/9410 40.38%, Loss:0.0627, Perplexity:1.0444
Epoch 9, step:3900/9410 41.45%, Loss:0.0610, Perplexity:1.0432
Epoch 9, step:4000/9410 42.51%, Loss:0.0620, Perplexity:1.0439
Epoch 9, step:4100/9410 43.57%, Loss:0.0567, Perplexity:1.0401
Epoch 9, step:4200/9410 44.63%, Loss:0.0602, Perplexity:1.0426
Epoch 9, step:4300/9410 45.70%, Loss:0.0614, Perplexity:1.0435
Epoch 9, step:4400/9410 46.76%, Loss:0.0593, Perplexity:1.0419
Epoch 9, step:4500/9410 47.82%, Loss:0.0657, Perplexity:1.0466
Epoch 9, step:4600/9410 48.88%, Loss:0.0625, Perplexity:1.0443
Epoch 9, step:4700/9410 49.95%, Loss:0.0713, Perplexity:1.0506
Epoch 9, step:4800/9410 51.01%, Loss:0.0668, Perplexity:1.0474
Epoch 9, step:4900/9410 52.07%, Loss:0.0709, Perplexity:1.0504
Epoch 9, step:5000/9410 53.13%, Loss:0.0653, Perplexity:1.0463
Epoch 9, step:5100/9410 54.20%, Loss:0.0617, Perplexity:1.0437
Epoch 9, step:5200/9410 55.26%, Loss:0.0658, Perplexity:1.0466
Epoch 9, step:5300/9410 56.32%, Loss:0.0702, Perplexity:1.0499
Epoch 9, step:5400/9410 57.39%, Loss:0.0707, Perplexity:1.0502
Epoch 9, step:5500/9410 58.45%, Loss:0.0686, Perplexity:1.0487
Epoch 9, step:5600/9410 59.51%, Loss:0.0724, Perplexity:1.0515
Epoch 9, step:5700/9410 60.57%, Loss:0.0711, Perplexity:1.0505
Epoch 9, step:5800/9410 61.64%, Loss:0.0810, Perplexity:1.0577
Epoch 9, step:5900/9410 62.70%, Loss:0.0789, Perplexity:1.0562
Epoch 9, step:6000/9410 63.76%, Loss:0.0782, Perplexity:1.0557
Epoch 9, step:6100/9410 64.82%, Loss:0.0778, Perplexity:1.0554
Epoch 9, step:6200/9410 65.89%, Loss:0.0735, Perplexity:1.0523
Epoch 9, step:6300/9410 66.95%, Loss:0.0781, Perplexity:1.0556
Epoch 9, step:6400/9410 68.01%, Loss:0.0806, Perplexity:1.0574
Epoch 9, step:6500/9410 69.08%, Loss:0.0917, Perplexity:1.0656
Epoch 9, step:6600/9410 70.14%, Loss:0.0883, Perplexity:1.0631
Epoch 9, step:6700/9410 71.20%, Loss:0.0942, Perplexity:1.0675
Epoch 9, step:6800/9410 72.26%, Loss:0.0894, Perplexity:1.0639
Epoch 9, step:6900/9410 73.33%, Loss:0.0908, Perplexity:1.0649
Epoch 9, step:7000/9410 74.39%, Loss:0.0918, Perplexity:1.0657
Epoch 9, step:7100/9410 75.45%, Loss:0.0856, Perplexity:1.0611
Epoch 9, step:7200/9410 76.51%, Loss:0.0851, Perplexity:1.0608
Epoch 9, step:7300/9410 77.58%, Loss:0.0882, Perplexity:1.0630
Epoch 9, step:7400/9410 78.64%, Loss:0.0832, Perplexity:1.0594
Epoch 9, step:7500/9410 79.70%, Loss:0.0889, Perplexity:1.0636
Epoch 9, step:7600/9410 80.77%, Loss:0.1042, Perplexity:1.0749
Epoch 9, step:7700/9410 81.83%, Loss:0.1012, Perplexity:1.0726
Epoch 9, step:7800/9410 82.89%, Loss:0.0978, Perplexity:1.0701
Epoch 9, step:7900/9410 83.95%, Loss:0.1031, Perplexity:1.0741
Epoch 9, step:8000/9410 85.02%, Loss:0.0989, Perplexity:1.0709
Epoch 9, step:8100/9410 86.08%, Loss:0.0988, Perplexity:1.0709
Epoch 9, step:8200/9410 87.14%, Loss:0.0964, Perplexity:1.0691
Epoch 9, step:8300/9410 88.20%, Loss:0.1046, Perplexity:1.0752
Epoch 9, step:8400/9410 89.27%, Loss:0.0965, Perplexity:1.0692
Epoch 9, step:8500/9410 90.33%, Loss:0.1147, Perplexity:1.0827
Epoch 9, step:8600/9410 91.39%, Loss:0.1161, Perplexity:1.0838
Epoch 9, step:8700/9410 92.45%, Loss:0.1125, Perplexity:1.0811
Epoch 9, step:8800/9410 93.52%, Loss:0.1120, Perplexity:1.0807
Epoch 9, step:8900/9410 94.58%, Loss:0.1136, Perplexity:1.0819
Epoch 9, step:9000/9410 95.64%, Loss:0.1108, Perplexity:1.0798
Epoch 9, step:9100/9410 96.71%, Loss:0.1079, Perplexity:1.0777
Epoch 9, step:9200/9410 97.77%, Loss:0.1109, Perplexity:1.0799
Epoch 9, step:9300/9410 98.83%, Loss:0.1397, Perplexity:1.1017
Epoch 9, step:9400/9410 99.89%, Loss:0.1808, Perplexity:1.1335
Epoch 9, validation average loss:0.2890, Perplexity:1.2218
Saving checkpoint to ./ckpts/ckpt-9-0.2890.pt
Epoch: 10
Epoch 10, step:100/9410 1.06%, Loss:0.0666, Perplexity:1.0472
Epoch 10, step:200/9410 2.13%, Loss:0.0596, Perplexity:1.0421
Epoch 10, step:300/9410 3.19%, Loss:0.0604, Perplexity:1.0427
Epoch 10, step:400/9410 4.25%, Loss:0.0644, Perplexity:1.0457
Epoch 10, step:500/9410 5.31%, Loss:0.0702, Perplexity:1.0498
Epoch 10, step:600/9410 6.38%, Loss:0.0710, Perplexity:1.0505
Epoch 10, step:700/9410 7.44%, Loss:0.0702, Perplexity:1.0498
Epoch 10, step:800/9410 8.50%, Loss:0.0729, Perplexity:1.0518
Epoch 10, step:900/9410 9.56%, Loss:0.0721, Perplexity:1.0512
Epoch 10, step:1000/9410 10.63%, Loss:0.0873, Perplexity:1.0624
Epoch 10, step:1100/9410 11.69%, Loss:0.0748, Perplexity:1.0532
Epoch 10, step:1200/9410 12.75%, Loss:0.0844, Perplexity:1.0603
Epoch 10, step:1300/9410 13.82%, Loss:0.0869, Perplexity:1.0621
Epoch 10, step:1400/9410 14.88%, Loss:0.0828, Perplexity:1.0591
Epoch 10, step:1500/9410 15.94%, Loss:0.0952, Perplexity:1.0682
Epoch 10, step:1600/9410 17.00%, Loss:0.0878, Perplexity:1.0628
Epoch 10, step:1700/9410 18.07%, Loss:0.0910, Perplexity:1.0651
Epoch 10, step:1800/9410 19.13%, Loss:0.0894, Perplexity:1.0639
Epoch 10, step:1900/9410 20.19%, Loss:0.0996, Perplexity:1.0715
Epoch 10, step:2000/9410 21.25%, Loss:0.1053, Perplexity:1.0757
Epoch 10, step:2100/9410 22.32%, Loss:0.1003, Perplexity:1.0720
Epoch 10, step:2200/9410 23.38%, Loss:0.0913, Perplexity:1.0654
Epoch 10, step:2300/9410 24.44%, Loss:0.1097, Perplexity:1.0790
Epoch 10, step:2400/9410 25.50%, Loss:0.1209, Perplexity:1.0874
Epoch 10, step:2500/9410 26.57%, Loss:0.1164, Perplexity:1.0840
Epoch 10, step:2600/9410 27.63%, Loss:0.1092, Perplexity:1.0786
Epoch 10, step:2700/9410 28.69%, Loss:0.1064, Perplexity:1.0765
Epoch 10, step:2800/9410 29.76%, Loss:0.1049, Perplexity:1.0754
Epoch 10, step:2900/9410 30.82%, Loss:0.1185, Perplexity:1.0856
Epoch 10, step:3000/9410 31.88%, Loss:0.1317, Perplexity:1.0956
Epoch 10, step:3100/9410 32.94%, Loss:0.1351, Perplexity:1.0982
Epoch 10, step:3200/9410 34.01%, Loss:0.1380, Perplexity:1.1004
Epoch 10, step:3300/9410 35.07%, Loss:0.1539, Perplexity:1.1126
Epoch 10, step:3400/9410 36.13%, Loss:0.1623, Perplexity:1.1191
Epoch 10, step:3500/9410 37.19%, Loss:0.1355, Perplexity:1.0985
Epoch 10, step:3600/9410 38.26%, Loss:0.0533, Perplexity:1.0377
Epoch 10, step:3700/9410 39.32%, Loss:0.0492, Perplexity:1.0347
Epoch 10, step:3800/9410 40.38%, Loss:0.0530, Perplexity:1.0375
Epoch 10, step:3900/9410 41.45%, Loss:0.0571, Perplexity:1.0404
Epoch 10, step:4000/9410 42.51%, Loss:0.0590, Perplexity:1.0417
Epoch 10, step:4100/9410 43.57%, Loss:0.0534, Perplexity:1.0377
Epoch 10, step:4200/9410 44.63%, Loss:0.0562, Perplexity:1.0397
Epoch 10, step:4300/9410 45.70%, Loss:0.0565, Perplexity:1.0399
Epoch 10, step:4400/9410 46.76%, Loss:0.0548, Perplexity:1.0387
Epoch 10, step:4500/9410 47.82%, Loss:0.0601, Perplexity:1.0426
Epoch 10, step:4600/9410 48.88%, Loss:0.0583, Perplexity:1.0413
Epoch 10, step:4700/9410 49.95%, Loss:0.0668, Perplexity:1.0474
Epoch 10, step:4800/9410 51.01%, Loss:0.0586, Perplexity:1.0414
Epoch 10, step:4900/9410 52.07%, Loss:0.0647, Perplexity:1.0459
Epoch 10, step:5000/9410 53.13%, Loss:0.0592, Perplexity:1.0419
Epoch 10, step:5100/9410 54.20%, Loss:0.0602, Perplexity:1.0426
Epoch 10, step:5200/9410 55.26%, Loss:0.0617, Perplexity:1.0437
Epoch 10, step:5300/9410 56.32%, Loss:0.0691, Perplexity:1.0491
Epoch 10, step:5400/9410 57.39%, Loss:0.0693, Perplexity:1.0492
Epoch 10, step:5500/9410 58.45%, Loss:0.0663, Perplexity:1.0471
Epoch 10, step:5600/9410 59.51%, Loss:0.0658, Perplexity:1.0467
Epoch 10, step:5700/9410 60.57%, Loss:0.0666, Perplexity:1.0472
Epoch 10, step:5800/9410 61.64%, Loss:0.0738, Perplexity:1.0525
Epoch 10, step:5900/9410 62.70%, Loss:0.0741, Perplexity:1.0527
Epoch 10, step:6000/9410 63.76%, Loss:0.0736, Perplexity:1.0524
Epoch 10, step:6100/9410 64.82%, Loss:0.0748, Perplexity:1.0532
Epoch 10, step:6200/9410 65.89%, Loss:0.0672, Perplexity:1.0476
Epoch 10, step:6300/9410 66.95%, Loss:0.0704, Perplexity:1.0500
Epoch 10, step:6400/9410 68.01%, Loss:0.0763, Perplexity:1.0543
Epoch 10, step:6500/9410 69.08%, Loss:0.0865, Perplexity:1.0618
Epoch 10, step:6600/9410 70.14%, Loss:0.0813, Perplexity:1.0580
Epoch 10, step:6700/9410 71.20%, Loss:0.0856, Perplexity:1.0611
Epoch 10, step:6800/9410 72.26%, Loss:0.0852, Perplexity:1.0608
Epoch 10, step:6900/9410 73.33%, Loss:0.0850, Perplexity:1.0607
Epoch 10, step:7000/9410 74.39%, Loss:0.0834, Perplexity:1.0595
Epoch 10, step:7100/9410 75.45%, Loss:0.0821, Perplexity:1.0586
Epoch 10, step:7200/9410 76.51%, Loss:0.0792, Perplexity:1.0564
Epoch 10, step:7300/9410 77.58%, Loss:0.0807, Perplexity:1.0575
Epoch 10, step:7400/9410 78.64%, Loss:0.0771, Perplexity:1.0549
Epoch 10, step:7500/9410 79.70%, Loss:0.0831, Perplexity:1.0593
Epoch 10, step:7600/9410 80.77%, Loss:0.0985, Perplexity:1.0706
Epoch 10, step:7700/9410 81.83%, Loss:0.0983, Perplexity:1.0705
Epoch 10, step:7800/9410 82.89%, Loss:0.0936, Perplexity:1.0670
Epoch 10, step:7900/9410 83.95%, Loss:0.0972, Perplexity:1.0697
Epoch 10, step:8000/9410 85.02%, Loss:0.0918, Perplexity:1.0657
Epoch 10, step:8100/9410 86.08%, Loss:0.0927, Perplexity:1.0664
Epoch 10, step:8200/9410 87.14%, Loss:0.0869, Perplexity:1.0621
Epoch 10, step:8300/9410 88.20%, Loss:0.0972, Perplexity:1.0697
Epoch 10, step:8400/9410 89.27%, Loss:0.0934, Perplexity:1.0669
Epoch 10, step:8500/9410 90.33%, Loss:0.1114, Perplexity:1.0802
Epoch 10, step:8600/9410 91.39%, Loss:0.1120, Perplexity:1.0808
Epoch 10, step:8700/9410 92.45%, Loss:0.1090, Perplexity:1.0785
Epoch 10, step:8800/9410 93.52%, Loss:0.1039, Perplexity:1.0747
Epoch 10, step:8900/9410 94.58%, Loss:0.1059, Perplexity:1.0762
Epoch 10, step:9000/9410 95.64%, Loss:0.1055, Perplexity:1.0759
Epoch 10, step:9100/9410 96.71%, Loss:0.1027, Perplexity:1.0738
Epoch 10, step:9200/9410 97.77%, Loss:0.1068, Perplexity:1.0768
Epoch 10, step:9300/9410 98.83%, Loss:0.1317, Perplexity:1.0956
Epoch 10, step:9400/9410 99.89%, Loss:0.1655, Perplexity:1.1215
Epoch 10, validation average loss:0.2881, Perplexity:1.2211
Saving checkpoint to ./ckpts/ckpt-10-0.2881.pt
Epoch: 11
Epoch 11, step:100/9410 1.06%, Loss:0.0611, Perplexity:1.0432
Epoch 11, step:200/9410 2.13%, Loss:0.0546, Perplexity:1.0386
Epoch 11, step:300/9410 3.19%, Loss:0.0548, Perplexity:1.0387
Epoch 11, step:400/9410 4.25%, Loss:0.0571, Perplexity:1.0404
Epoch 11, step:500/9410 5.31%, Loss:0.0643, Perplexity:1.0456
Epoch 11, step:600/9410 6.38%, Loss:0.0697, Perplexity:1.0495
Epoch 11, step:700/9410 7.44%, Loss:0.0687, Perplexity:1.0488
Epoch 11, step:800/9410 8.50%, Loss:0.0692, Perplexity:1.0492
Epoch 11, step:900/9410 9.56%, Loss:0.0686, Perplexity:1.0487
Epoch 11, step:1000/9410 10.63%, Loss:0.0814, Perplexity:1.0581
Epoch 11, step:1100/9410 11.69%, Loss:0.0694, Perplexity:1.0493
Epoch 11, step:1200/9410 12.75%, Loss:0.0804, Perplexity:1.0573
Epoch 11, step:1300/9410 13.82%, Loss:0.0769, Perplexity:1.0547
Epoch 11, step:1400/9410 14.88%, Loss:0.0735, Perplexity:1.0523
Epoch 11, step:1500/9410 15.94%, Loss:0.0897, Perplexity:1.0642
Epoch 11, step:1600/9410 17.00%, Loss:0.0843, Perplexity:1.0602
Epoch 11, step:1700/9410 18.07%, Loss:0.0827, Perplexity:1.0590
Epoch 11, step:1800/9410 19.13%, Loss:0.0853, Perplexity:1.0609
Epoch 11, step:1900/9410 20.19%, Loss:0.0857, Perplexity:1.0612
Epoch 11, step:2000/9410 21.25%, Loss:0.0917, Perplexity:1.0656
Epoch 11, step:2100/9410 22.32%, Loss:0.0972, Perplexity:1.0697
Epoch 11, step:2200/9410 23.38%, Loss:0.0845, Perplexity:1.0603
Epoch 11, step:2300/9410 24.44%, Loss:0.1010, Perplexity:1.0726
Epoch 11, step:2400/9410 25.50%, Loss:0.1114, Perplexity:1.0802
Epoch 11, step:2500/9410 26.57%, Loss:0.1089, Perplexity:1.0784
Epoch 11, step:2600/9410 27.63%, Loss:0.1022, Perplexity:1.0734
Epoch 11, step:2700/9410 28.69%, Loss:0.1021, Perplexity:1.0733
Epoch 11, step:2800/9410 29.76%, Loss:0.1012, Perplexity:1.0727
Epoch 11, step:2900/9410 30.82%, Loss:0.1098, Perplexity:1.0791
Epoch 11, step:3000/9410 31.88%, Loss:0.1258, Perplexity:1.0911
Epoch 11, step:3100/9410 32.94%, Loss:0.1269, Perplexity:1.0919
Epoch 11, step:3200/9410 34.01%, Loss:0.1276, Perplexity:1.0925
Epoch 11, step:3300/9410 35.07%, Loss:0.1456, Perplexity:1.1062
Epoch 11, step:3400/9410 36.13%, Loss:0.1548, Perplexity:1.1133
Epoch 11, step:3500/9410 37.19%, Loss:0.1313, Perplexity:1.0953
Epoch 11, step:3600/9410 38.26%, Loss:0.0472, Perplexity:1.0332
Epoch 11, step:3700/9410 39.32%, Loss:0.0450, Perplexity:1.0317
Epoch 11, step:3800/9410 40.38%, Loss:0.0514, Perplexity:1.0363
Epoch 11, step:3900/9410 41.45%, Loss:0.0540, Perplexity:1.0382
Epoch 11, step:4000/9410 42.51%, Loss:0.0561, Perplexity:1.0397
Epoch 11, step:4100/9410 43.57%, Loss:0.0500, Perplexity:1.0353
Epoch 11, step:4200/9410 44.63%, Loss:0.0491, Perplexity:1.0346
Epoch 11, step:4300/9410 45.70%, Loss:0.0506, Perplexity:1.0357
Epoch 11, step:4400/9410 46.76%, Loss:0.0526, Perplexity:1.0372
Epoch 11, step:4500/9410 47.82%, Loss:0.0546, Perplexity:1.0386
Epoch 11, step:4600/9410 48.88%, Loss:0.0546, Perplexity:1.0386
Epoch 11, step:4700/9410 49.95%, Loss:0.0609, Perplexity:1.0431
Epoch 11, step:4800/9410 51.01%, Loss:0.0578, Perplexity:1.0409
Epoch 11, step:4900/9410 52.07%, Loss:0.0590, Perplexity:1.0418
Epoch 11, step:5000/9410 53.13%, Loss:0.0560, Perplexity:1.0396
Epoch 11, step:5100/9410 54.20%, Loss:0.0546, Perplexity:1.0386
Epoch 11, step:5200/9410 55.26%, Loss:0.0590, Perplexity:1.0417
Epoch 11, step:5300/9410 56.32%, Loss:0.0636, Perplexity:1.0451
Epoch 11, step:5400/9410 57.39%, Loss:0.0612, Perplexity:1.0434
Epoch 11, step:5500/9410 58.45%, Loss:0.0594, Perplexity:1.0420
Epoch 11, step:5600/9410 59.51%, Loss:0.0618, Perplexity:1.0438
Epoch 11, step:5700/9410 60.57%, Loss:0.0623, Perplexity:1.0441
Epoch 11, step:5800/9410 61.64%, Loss:0.0689, Perplexity:1.0489
Epoch 11, step:5900/9410 62.70%, Loss:0.0696, Perplexity:1.0494
Epoch 11, step:6000/9410 63.76%, Loss:0.0677, Perplexity:1.0480
Epoch 11, step:6100/9410 64.82%, Loss:0.0707, Perplexity:1.0502
Epoch 11, step:6200/9410 65.89%, Loss:0.0637, Perplexity:1.0451
Epoch 11, step:6300/9410 66.95%, Loss:0.0655, Perplexity:1.0464
Epoch 11, step:6400/9410 68.01%, Loss:0.0732, Perplexity:1.0521
Epoch 11, step:6500/9410 69.08%, Loss:0.0820, Perplexity:1.0585
Epoch 11, step:6600/9410 70.14%, Loss:0.0800, Perplexity:1.0570
Epoch 11, step:6700/9410 71.20%, Loss:0.0839, Perplexity:1.0598
Epoch 11, step:6800/9410 72.26%, Loss:0.0807, Perplexity:1.0575
Epoch 11, step:6900/9410 73.33%, Loss:0.0775, Perplexity:1.0552
Epoch 11, step:7000/9410 74.39%, Loss:0.0791, Perplexity:1.0564
Epoch 11, step:7100/9410 75.45%, Loss:0.0761, Perplexity:1.0541
Epoch 11, step:7200/9410 76.51%, Loss:0.0764, Perplexity:1.0544
Epoch 11, step:7300/9410 77.58%, Loss:0.0748, Perplexity:1.0532
Epoch 11, step:7400/9410 78.64%, Loss:0.0732, Perplexity:1.0521
Epoch 11, step:7500/9410 79.70%, Loss:0.0803, Perplexity:1.0572
Epoch 11, step:7600/9410 80.77%, Loss:0.0967, Perplexity:1.0694
Epoch 11, step:7700/9410 81.83%, Loss:0.0906, Perplexity:1.0648
Epoch 11, step:7800/9410 82.89%, Loss:0.0871, Perplexity:1.0623
Epoch 11, step:7900/9410 83.95%, Loss:0.0915, Perplexity:1.0655
Epoch 11, step:8000/9410 85.02%, Loss:0.0872, Perplexity:1.0623
Epoch 11, step:8100/9410 86.08%, Loss:0.0874, Perplexity:1.0624
Epoch 11, step:8200/9410 87.14%, Loss:0.0851, Perplexity:1.0608
Epoch 11, step:8300/9410 88.20%, Loss:0.0937, Perplexity:1.0671
Epoch 11, step:8400/9410 89.27%, Loss:0.0866, Perplexity:1.0618
Epoch 11, step:8500/9410 90.33%, Loss:0.1039, Perplexity:1.0747
Epoch 11, step:8600/9410 91.39%, Loss:0.1029, Perplexity:1.0740
Epoch 11, step:8700/9410 92.45%, Loss:0.1013, Perplexity:1.0727
Epoch 11, step:8800/9410 93.52%, Loss:0.0975, Perplexity:1.0699
Epoch 11, step:8900/9410 94.58%, Loss:0.1001, Perplexity:1.0719
Epoch 11, step:9000/9410 95.64%, Loss:0.1001, Perplexity:1.0718
Epoch 11, step:9100/9410 96.71%, Loss:0.0982, Perplexity:1.0705
Epoch 11, step:9200/9410 97.77%, Loss:0.0995, Perplexity:1.0714
Epoch 11, step:9300/9410 98.83%, Loss:0.1285, Perplexity:1.0932
Epoch 11, step:9400/9410 99.89%, Loss:0.1600, Perplexity:1.1173
Epoch 11, validation average loss:0.2816, Perplexity:1.2155
Epoch 14: reducing learning rate of group 0 to 1.5000e-04.
Saving checkpoint to ./ckpts/ckpt-11-0.2816.pt
Epoch: 12
Epoch 12, step:100/9410 1.06%, Loss:0.0463, Perplexity:1.0326
Epoch 12, step:200/9410 2.13%, Loss:0.0359, Perplexity:1.0252
Epoch 12, step:300/9410 3.19%, Loss:0.0378, Perplexity:1.0266
Epoch 12, step:400/9410 4.25%, Loss:0.0414, Perplexity:1.0291
Epoch 12, step:500/9410 5.31%, Loss:0.0422, Perplexity:1.0297
Epoch 12, step:600/9410 6.38%, Loss:0.0445, Perplexity:1.0313
Epoch 12, step:700/9410 7.44%, Loss:0.0455, Perplexity:1.0321
Epoch 12, step:800/9410 8.50%, Loss:0.0457, Perplexity:1.0322
Epoch 12, step:900/9410 9.56%, Loss:0.0429, Perplexity:1.0302
Epoch 12, step:1000/9410 10.63%, Loss:0.0567, Perplexity:1.0401
Epoch 12, step:1100/9410 11.69%, Loss:0.0450, Perplexity:1.0317
Epoch 12, step:1200/9410 12.75%, Loss:0.0560, Perplexity:1.0395
Epoch 12, step:1300/9410 13.82%, Loss:0.0538, Perplexity:1.0380
Epoch 12, step:1400/9410 14.88%, Loss:0.0512, Perplexity:1.0362
Epoch 12, step:1500/9410 15.94%, Loss:0.0624, Perplexity:1.0442
Epoch 12, step:1600/9410 17.00%, Loss:0.0595, Perplexity:1.0421
Epoch 12, step:1700/9410 18.07%, Loss:0.0595, Perplexity:1.0421
Epoch 12, step:1800/9410 19.13%, Loss:0.0606, Perplexity:1.0429
Epoch 12, step:1900/9410 20.19%, Loss:0.0650, Perplexity:1.0461
Epoch 12, step:2000/9410 21.25%, Loss:0.0672, Perplexity:1.0477
Epoch 12, step:2100/9410 22.32%, Loss:0.0689, Perplexity:1.0489
Epoch 12, step:2200/9410 23.38%, Loss:0.0618, Perplexity:1.0438
Epoch 12, step:2300/9410 24.44%, Loss:0.0734, Perplexity:1.0522
Epoch 12, step:2400/9410 25.50%, Loss:0.0868, Perplexity:1.0620
Epoch 12, step:2500/9410 26.57%, Loss:0.0807, Perplexity:1.0576
Epoch 12, step:2600/9410 27.63%, Loss:0.0756, Perplexity:1.0538
Epoch 12, step:2700/9410 28.69%, Loss:0.0773, Perplexity:1.0551
Epoch 12, step:2800/9410 29.76%, Loss:0.0736, Perplexity:1.0523
Epoch 12, step:2900/9410 30.82%, Loss:0.0859, Perplexity:1.0613
Epoch 12, step:3000/9410 31.88%, Loss:0.0996, Perplexity:1.0714
Epoch 12, step:3100/9410 32.94%, Loss:0.0969, Perplexity:1.0695
Epoch 12, step:3200/9410 34.01%, Loss:0.0998, Perplexity:1.0717
Epoch 12, step:3300/9410 35.07%, Loss:0.1194, Perplexity:1.0863
Epoch 12, step:3400/9410 36.13%, Loss:0.1284, Perplexity:1.0931
Epoch 12, step:3500/9410 37.19%, Loss:0.1056, Perplexity:1.0760
Epoch 12, step:3600/9410 38.26%, Loss:0.0375, Perplexity:1.0263
Epoch 12, step:3700/9410 39.32%, Loss:0.0341, Perplexity:1.0240
Epoch 12, step:3800/9410 40.38%, Loss:0.0359, Perplexity:1.0252
Epoch 12, step:3900/9410 41.45%, Loss:0.0374, Perplexity:1.0262
Epoch 12, step:4000/9410 42.51%, Loss:0.0394, Perplexity:1.0277
Epoch 12, step:4100/9410 43.57%, Loss:0.0355, Perplexity:1.0249
Epoch 12, step:4200/9410 44.63%, Loss:0.0366, Perplexity:1.0257
Epoch 12, step:4300/9410 45.70%, Loss:0.0365, Perplexity:1.0256
Epoch 12, step:4400/9410 46.76%, Loss:0.0363, Perplexity:1.0255
Epoch 12, step:4500/9410 47.82%, Loss:0.0385, Perplexity:1.0271
Epoch 12, step:4600/9410 48.88%, Loss:0.0391, Perplexity:1.0274
Epoch 12, step:4700/9410 49.95%, Loss:0.0434, Perplexity:1.0306
Epoch 12, step:4800/9410 51.01%, Loss:0.0376, Perplexity:1.0264
Epoch 12, step:4900/9410 52.07%, Loss:0.0433, Perplexity:1.0304
Epoch 12, step:5000/9410 53.13%, Loss:0.0368, Perplexity:1.0259
Epoch 12, step:5100/9410 54.20%, Loss:0.0372, Perplexity:1.0261
Epoch 12, step:5200/9410 55.26%, Loss:0.0405, Perplexity:1.0285
Epoch 12, step:5300/9410 56.32%, Loss:0.0436, Perplexity:1.0307
Epoch 12, step:5400/9410 57.39%, Loss:0.0412, Perplexity:1.0289
Epoch 12, step:5500/9410 58.45%, Loss:0.0390, Perplexity:1.0274
Epoch 12, step:5600/9410 59.51%, Loss:0.0412, Perplexity:1.0290
Epoch 12, step:5700/9410 60.57%, Loss:0.0420, Perplexity:1.0296
Epoch 12, step:5800/9410 61.64%, Loss:0.0466, Perplexity:1.0328
Epoch 12, step:5900/9410 62.70%, Loss:0.0453, Perplexity:1.0319
Epoch 12, step:6000/9410 63.76%, Loss:0.0471, Perplexity:1.0332
Epoch 12, step:6100/9410 64.82%, Loss:0.0478, Perplexity:1.0337
Epoch 12, step:6200/9410 65.89%, Loss:0.0425, Perplexity:1.0299
Epoch 12, step:6300/9410 66.95%, Loss:0.0442, Perplexity:1.0311
Epoch 12, step:6400/9410 68.01%, Loss:0.0491, Perplexity:1.0346
Epoch 12, step:6500/9410 69.08%, Loss:0.0539, Perplexity:1.0381
Epoch 12, step:6600/9410 70.14%, Loss:0.0535, Perplexity:1.0378
Epoch 12, step:6700/9410 71.20%, Loss:0.0554, Perplexity:1.0392
Epoch 12, step:6800/9410 72.26%, Loss:0.0541, Perplexity:1.0382
Epoch 12, step:6900/9410 73.33%, Loss:0.0538, Perplexity:1.0380
Epoch 12, step:7000/9410 74.39%, Loss:0.0551, Perplexity:1.0390
Epoch 12, step:7100/9410 75.45%, Loss:0.0553, Perplexity:1.0391
Epoch 12, step:7200/9410 76.51%, Loss:0.0513, Perplexity:1.0362
Epoch 12, step:7300/9410 77.58%, Loss:0.0505, Perplexity:1.0357
Epoch 12, step:7400/9410 78.64%, Loss:0.0497, Perplexity:1.0350
Epoch 12, step:7500/9410 79.70%, Loss:0.0515, Perplexity:1.0364
Epoch 12, step:7600/9410 80.77%, Loss:0.0673, Perplexity:1.0477
Epoch 12, step:7700/9410 81.83%, Loss:0.0628, Perplexity:1.0445
Epoch 12, step:7800/9410 82.89%, Loss:0.0605, Perplexity:1.0429
Epoch 12, step:7900/9410 83.95%, Loss:0.0656, Perplexity:1.0465
Epoch 12, step:8000/9410 85.02%, Loss:0.0613, Perplexity:1.0434
Epoch 12, step:8100/9410 86.08%, Loss:0.0594, Perplexity:1.0420
Epoch 12, step:8200/9410 87.14%, Loss:0.0586, Perplexity:1.0414
Epoch 12, step:8300/9410 88.20%, Loss:0.0652, Perplexity:1.0463
Epoch 12, step:8400/9410 89.27%, Loss:0.0588, Perplexity:1.0416
Epoch 12, step:8500/9410 90.33%, Loss:0.0747, Perplexity:1.0531
Epoch 12, step:8600/9410 91.39%, Loss:0.0741, Perplexity:1.0527
Epoch 12, step:8700/9410 92.45%, Loss:0.0726, Perplexity:1.0516
Epoch 12, step:8800/9410 93.52%, Loss:0.0703, Perplexity:1.0499
Epoch 12, step:8900/9410 94.58%, Loss:0.0715, Perplexity:1.0508
Epoch 12, step:9000/9410 95.64%, Loss:0.0715, Perplexity:1.0508
Epoch 12, step:9100/9410 96.71%, Loss:0.0682, Perplexity:1.0484
Epoch 12, step:9200/9410 97.77%, Loss:0.0715, Perplexity:1.0508
Epoch 12, step:9300/9410 98.83%, Loss:0.0950, Perplexity:1.0680
Epoch 12, step:9400/9410 99.89%, Loss:0.1199, Perplexity:1.0866
Epoch 12, validation average loss:0.2684, Perplexity:1.2045
Saving checkpoint to ./ckpts/best_ckpt.pt
Saving checkpoint to ./ckpts/ckpt-12-0.2684.pt
Epoch: 13
Epoch 13, step:100/9410 1.06%, Loss:0.0319, Perplexity:1.0224
Epoch 13, step:200/9410 2.13%, Loss:0.0258, Perplexity:1.0180
Epoch 13, step:300/9410 3.19%, Loss:0.0255, Perplexity:1.0178
Epoch 13, step:400/9410 4.25%, Loss:0.0289, Perplexity:1.0202
Epoch 13, step:500/9410 5.31%, Loss:0.0302, Perplexity:1.0212
Epoch 13, step:600/9410 6.38%, Loss:0.0308, Perplexity:1.0216
Epoch 13, step:700/9410 7.44%, Loss:0.0353, Perplexity:1.0248
Epoch 13, step:800/9410 8.50%, Loss:0.0324, Perplexity:1.0227
Epoch 13, step:900/9410 9.56%, Loss:0.0300, Perplexity:1.0210
Epoch 13, step:1000/9410 10.63%, Loss:0.0400, Perplexity:1.0281
Epoch 13, step:1100/9410 11.69%, Loss:0.0323, Perplexity:1.0226
Epoch 13, step:1200/9410 12.75%, Loss:0.0409, Perplexity:1.0288
Epoch 13, step:1300/9410 13.82%, Loss:0.0390, Perplexity:1.0274
Epoch 13, step:1400/9410 14.88%, Loss:0.0371, Perplexity:1.0260
Epoch 13, step:1500/9410 15.94%, Loss:0.0478, Perplexity:1.0337
Epoch 13, step:1600/9410 17.00%, Loss:0.0434, Perplexity:1.0306
Epoch 13, step:1700/9410 18.07%, Loss:0.0441, Perplexity:1.0310
Epoch 13, step:1800/9410 19.13%, Loss:0.0478, Perplexity:1.0337
Epoch 13, step:1900/9410 20.19%, Loss:0.0493, Perplexity:1.0347
Epoch 13, step:2000/9410 21.25%, Loss:0.0502, Perplexity:1.0354
Epoch 13, step:2100/9410 22.32%, Loss:0.0515, Perplexity:1.0363
Epoch 13, step:2200/9410 23.38%, Loss:0.0467, Perplexity:1.0329
Epoch 13, step:2300/9410 24.44%, Loss:0.0591, Perplexity:1.0418
Epoch 13, step:2400/9410 25.50%, Loss:0.0659, Perplexity:1.0467
Epoch 13, step:2500/9410 26.57%, Loss:0.0609, Perplexity:1.0431
Epoch 13, step:2600/9410 27.63%, Loss:0.0595, Perplexity:1.0421
Epoch 13, step:2700/9410 28.69%, Loss:0.0587, Perplexity:1.0415
Epoch 13, step:2800/9410 29.76%, Loss:0.0568, Perplexity:1.0402
Epoch 13, step:2900/9410 30.82%, Loss:0.0659, Perplexity:1.0468
Epoch 13, step:3000/9410 31.88%, Loss:0.0777, Perplexity:1.0553
Epoch 13, step:3100/9410 32.94%, Loss:0.0804, Perplexity:1.0573
Epoch 13, step:3200/9410 34.01%, Loss:0.0798, Perplexity:1.0569
Epoch 13, step:3300/9410 35.07%, Loss:0.0969, Perplexity:1.0695
Epoch 13, step:3400/9410 36.13%, Loss:0.1069, Perplexity:1.0769
Epoch 13, step:3500/9410 37.19%, Loss:0.0875, Perplexity:1.0626
Epoch 13, step:3600/9410 38.26%, Loss:0.0291, Perplexity:1.0204
Epoch 13, step:3700/9410 39.32%, Loss:0.0264, Perplexity:1.0185
Epoch 13, step:3800/9410 40.38%, Loss:0.0274, Perplexity:1.0192
Epoch 13, step:3900/9410 41.45%, Loss:0.0288, Perplexity:1.0202
Epoch 13, step:4000/9410 42.51%, Loss:0.0292, Perplexity:1.0204
Epoch 13, step:4100/9410 43.57%, Loss:0.0300, Perplexity:1.0210
Epoch 13, step:4200/9410 44.63%, Loss:0.0296, Perplexity:1.0207
Epoch 13, step:4300/9410 45.70%, Loss:0.0289, Perplexity:1.0202
Epoch 13, step:4400/9410 46.76%, Loss:0.0270, Perplexity:1.0189
Epoch 13, step:4500/9410 47.82%, Loss:0.0293, Perplexity:1.0205
Epoch 13, step:4600/9410 48.88%, Loss:0.0290, Perplexity:1.0203
Epoch 13, step:4700/9410 49.95%, Loss:0.0335, Perplexity:1.0235
Epoch 13, step:4800/9410 51.01%, Loss:0.0299, Perplexity:1.0210
Epoch 13, step:4900/9410 52.07%, Loss:0.0333, Perplexity:1.0234
Epoch 13, step:5000/9410 53.13%, Loss:0.0290, Perplexity:1.0203
Epoch 13, step:5100/9410 54.20%, Loss:0.0300, Perplexity:1.0210
Epoch 13, step:5200/9410 55.26%, Loss:0.0309, Perplexity:1.0216
Epoch 13, step:5300/9410 56.32%, Loss:0.0341, Perplexity:1.0239
Epoch 13, step:5400/9410 57.39%, Loss:0.0325, Perplexity:1.0228
Epoch 13, step:5500/9410 58.45%, Loss:0.0305, Perplexity:1.0214
Epoch 13, step:5600/9410 59.51%, Loss:0.0330, Perplexity:1.0232
Epoch 13, step:5700/9410 60.57%, Loss:0.0323, Perplexity:1.0226
Epoch 13, step:5800/9410 61.64%, Loss:0.0369, Perplexity:1.0259
Epoch 13, step:5900/9410 62.70%, Loss:0.0353, Perplexity:1.0248
Epoch 13, step:6000/9410 63.76%, Loss:0.0357, Perplexity:1.0250
Epoch 13, step:6100/9410 64.82%, Loss:0.0365, Perplexity:1.0256
Epoch 13, step:6200/9410 65.89%, Loss:0.0328, Perplexity:1.0230
Epoch 13, step:6300/9410 66.95%, Loss:0.0348, Perplexity:1.0244
Epoch 13, step:6400/9410 68.01%, Loss:0.0383, Perplexity:1.0269
Epoch 13, step:6500/9410 69.08%, Loss:0.0438, Perplexity:1.0308
Epoch 13, step:6600/9410 70.14%, Loss:0.0432, Perplexity:1.0304
Epoch 13, step:6700/9410 71.20%, Loss:0.0440, Perplexity:1.0310
Epoch 13, step:6800/9410 72.26%, Loss:0.0436, Perplexity:1.0307
Epoch 13, step:6900/9410 73.33%, Loss:0.0431, Perplexity:1.0303
Epoch 13, step:7000/9410 74.39%, Loss:0.0444, Perplexity:1.0312
Epoch 13, step:7100/9410 75.45%, Loss:0.0429, Perplexity:1.0302
Epoch 13, step:7200/9410 76.51%, Loss:0.0404, Perplexity:1.0284
Epoch 13, step:7300/9410 77.58%, Loss:0.0410, Perplexity:1.0289
Epoch 13, step:7400/9410 78.64%, Loss:0.0397, Perplexity:1.0279
Epoch 13, step:7500/9410 79.70%, Loss:0.0426, Perplexity:1.0300
Epoch 13, step:7600/9410 80.77%, Loss:0.0542, Perplexity:1.0383
Epoch 13, step:7700/9410 81.83%, Loss:0.0508, Perplexity:1.0358
Epoch 13, step:7800/9410 82.89%, Loss:0.0501, Perplexity:1.0353
Epoch 13, step:7900/9410 83.95%, Loss:0.0538, Perplexity:1.0380
Epoch 13, step:8000/9410 85.02%, Loss:0.0494, Perplexity:1.0348
Epoch 13, step:8100/9410 86.08%, Loss:0.0484, Perplexity:1.0341
Epoch 13, step:8200/9410 87.14%, Loss:0.0480, Perplexity:1.0338
Epoch 13, step:8300/9410 88.20%, Loss:0.0524, Perplexity:1.0370
Epoch 13, step:8400/9410 89.27%, Loss:0.0505, Perplexity:1.0356
Epoch 13, step:8500/9410 90.33%, Loss:0.0633, Perplexity:1.0449
Epoch 13, step:8600/9410 91.39%, Loss:0.0611, Perplexity:1.0433
Epoch 13, step:8700/9410 92.45%, Loss:0.0605, Perplexity:1.0429
Epoch 13, step:8800/9410 93.52%, Loss:0.0588, Perplexity:1.0416
Epoch 13, step:8900/9410 94.58%, Loss:0.0596, Perplexity:1.0422
Epoch 13, step:9000/9410 95.64%, Loss:0.0605, Perplexity:1.0428
Epoch 13, step:9100/9410 96.71%, Loss:0.0572, Perplexity:1.0404
Epoch 13, step:9200/9410 97.77%, Loss:0.0598, Perplexity:1.0423
Epoch 13, step:9300/9410 98.83%, Loss:0.0757, Perplexity:1.0539
Epoch 13, step:9400/9410 99.89%, Loss:0.0984, Perplexity:1.0706
Epoch 13, validation average loss:0.2867, Perplexity:1.2198
Saving checkpoint to ./ckpts/ckpt-13-0.2867.pt
Epoch: 14
Epoch 14, step:100/9410 1.06%, Loss:0.0247, Perplexity:1.0173
Epoch 14, step:200/9410 2.13%, Loss:0.0201, Perplexity:1.0140
Epoch 14, step:300/9410 3.19%, Loss:0.0211, Perplexity:1.0147
Epoch 14, step:400/9410 4.25%, Loss:0.0227, Perplexity:1.0158
Epoch 14, step:500/9410 5.31%, Loss:0.0245, Perplexity:1.0171
Epoch 14, step:600/9410 6.38%, Loss:0.0267, Perplexity:1.0187
Epoch 14, step:700/9410 7.44%, Loss:0.0269, Perplexity:1.0188
Epoch 14, step:800/9410 8.50%, Loss:0.0267, Perplexity:1.0187
Epoch 14, step:900/9410 9.56%, Loss:0.0272, Perplexity:1.0190
Epoch 14, step:1000/9410 10.63%, Loss:0.0341, Perplexity:1.0239
Epoch 14, step:1100/9410 11.69%, Loss:0.0286, Perplexity:1.0200
Epoch 14, step:1200/9410 12.75%, Loss:0.0373, Perplexity:1.0262
Epoch 14, step:1300/9410 13.82%, Loss:0.0334, Perplexity:1.0234
Epoch 14, step:1400/9410 14.88%, Loss:0.0316, Perplexity:1.0221
Epoch 14, step:1500/9410 15.94%, Loss:0.0412, Perplexity:1.0290
Epoch 14, step:1600/9410 17.00%, Loss:0.0383, Perplexity:1.0269
Epoch 14, step:1700/9410 18.07%, Loss:0.0391, Perplexity:1.0275
Epoch 14, step:1800/9410 19.13%, Loss:0.0409, Perplexity:1.0287
Epoch 14, step:1900/9410 20.19%, Loss:0.0449, Perplexity:1.0316
Epoch 14, step:2000/9410 21.25%, Loss:0.0430, Perplexity:1.0302
Epoch 14, step:2100/9410 22.32%, Loss:0.0433, Perplexity:1.0305
Epoch 14, step:2200/9410 23.38%, Loss:0.0401, Perplexity:1.0282
Epoch 14, step:2300/9410 24.44%, Loss:0.0528, Perplexity:1.0373
Epoch 14, step:2400/9410 25.50%, Loss:0.0592, Perplexity:1.0419
Epoch 14, step:2500/9410 26.57%, Loss:0.0544, Perplexity:1.0385
Epoch 14, step:2600/9410 27.63%, Loss:0.0527, Perplexity:1.0372
Epoch 14, step:2700/9410 28.69%, Loss:0.0516, Perplexity:1.0364
Epoch 14, step:2800/9410 29.76%, Loss:0.0505, Perplexity:1.0356
Epoch 14, step:2900/9410 30.82%, Loss:0.0561, Perplexity:1.0397
Epoch 14, step:3000/9410 31.88%, Loss:0.0693, Perplexity:1.0492
Epoch 14, step:3100/9410 32.94%, Loss:0.0706, Perplexity:1.0502
Epoch 14, step:3200/9410 34.01%, Loss:0.0730, Perplexity:1.0519
Epoch 14, step:3300/9410 35.07%, Loss:0.0894, Perplexity:1.0639
Epoch 14, step:3400/9410 36.13%, Loss:0.0968, Perplexity:1.0694
Epoch 14, step:3500/9410 37.19%, Loss:0.0792, Perplexity:1.0564
Epoch 14, step:3600/9410 38.26%, Loss:0.0260, Perplexity:1.0182
Epoch 14, step:3700/9410 39.32%, Loss:0.0234, Perplexity:1.0164
Epoch 14, step:3800/9410 40.38%, Loss:0.0251, Perplexity:1.0176
Epoch 14, step:3900/9410 41.45%, Loss:0.0252, Perplexity:1.0176
Epoch 14, step:4000/9410 42.51%, Loss:0.0260, Perplexity:1.0182
Epoch 14, step:4100/9410 43.57%, Loss:0.0252, Perplexity:1.0176
Epoch 14, step:4200/9410 44.63%, Loss:0.0250, Perplexity:1.0175
Epoch 14, step:4300/9410 45.70%, Loss:0.0253, Perplexity:1.0177
Epoch 14, step:4400/9410 46.76%, Loss:0.0235, Perplexity:1.0164
Epoch 14, step:4500/9410 47.82%, Loss:0.0246, Perplexity:1.0172
Epoch 14, step:4600/9410 48.88%, Loss:0.0248, Perplexity:1.0173
Epoch 14, step:4700/9410 49.95%, Loss:0.0283, Perplexity:1.0198
Epoch 14, step:4800/9410 51.01%, Loss:0.0261, Perplexity:1.0183
Epoch 14, step:4900/9410 52.07%, Loss:0.0278, Perplexity:1.0194
Epoch 14, step:5000/9410 53.13%, Loss:0.0252, Perplexity:1.0176
Epoch 14, step:5100/9410 54.20%, Loss:0.0256, Perplexity:1.0179
Epoch 14, step:5200/9410 55.26%, Loss:0.0265, Perplexity:1.0186
Epoch 14, step:5300/9410 56.32%, Loss:0.0309, Perplexity:1.0217
Epoch 14, step:5400/9410 57.39%, Loss:0.0285, Perplexity:1.0199
Epoch 14, step:5500/9410 58.45%, Loss:0.0268, Perplexity:1.0188
Epoch 14, step:5600/9410 59.51%, Loss:0.0283, Perplexity:1.0198
Epoch 14, step:5700/9410 60.57%, Loss:0.0270, Perplexity:1.0189
Epoch 14, step:5800/9410 61.64%, Loss:0.0322, Perplexity:1.0226
Epoch 14, step:5900/9410 62.70%, Loss:0.0300, Perplexity:1.0210
Epoch 14, step:6000/9410 63.76%, Loss:0.0328, Perplexity:1.0230
Epoch 14, step:6100/9410 64.82%, Loss:0.0340, Perplexity:1.0239
Epoch 14, step:6200/9410 65.89%, Loss:0.0300, Perplexity:1.0210
Epoch 14, step:6300/9410 66.95%, Loss:0.0306, Perplexity:1.0214
Epoch 14, step:6400/9410 68.01%, Loss:0.0344, Perplexity:1.0242
Epoch 14, step:6500/9410 69.08%, Loss:0.0399, Perplexity:1.0281
Epoch 14, step:6600/9410 70.14%, Loss:0.0386, Perplexity:1.0271
Epoch 14, step:6700/9410 71.20%, Loss:0.0396, Perplexity:1.0278
Epoch 14, step:6800/9410 72.26%, Loss:0.0387, Perplexity:1.0272
Epoch 14, step:6900/9410 73.33%, Loss:0.0391, Perplexity:1.0275
Epoch 14, step:7000/9410 74.39%, Loss:0.0396, Perplexity:1.0278
Epoch 14, step:7100/9410 75.45%, Loss:0.0372, Perplexity:1.0261
Epoch 14, step:7200/9410 76.51%, Loss:0.0382, Perplexity:1.0268
Epoch 14, step:7300/9410 77.58%, Loss:0.0367, Perplexity:1.0258
Epoch 14, step:7400/9410 78.64%, Loss:0.0359, Perplexity:1.0252
Epoch 14, step:7500/9410 79.70%, Loss:0.0392, Perplexity:1.0275
Epoch 14, step:7600/9410 80.77%, Loss:0.0493, Perplexity:1.0347
Epoch 14, step:7700/9410 81.83%, Loss:0.0466, Perplexity:1.0328
Epoch 14, step:7800/9410 82.89%, Loss:0.0467, Perplexity:1.0329
Epoch 14, step:7900/9410 83.95%, Loss:0.0482, Perplexity:1.0340
Epoch 14, step:8000/9410 85.02%, Loss:0.0446, Perplexity:1.0314
Epoch 14, step:8100/9410 86.08%, Loss:0.0440, Perplexity:1.0310
Epoch 14, step:8200/9410 87.14%, Loss:0.0436, Perplexity:1.0306
Epoch 14, step:8300/9410 88.20%, Loss:0.0486, Perplexity:1.0343
Epoch 14, step:8400/9410 89.27%, Loss:0.0459, Perplexity:1.0323
Epoch 14, step:8500/9410 90.33%, Loss:0.0563, Perplexity:1.0398
Epoch 14, step:8600/9410 91.39%, Loss:0.0572, Perplexity:1.0405
Epoch 14, step:8700/9410 92.45%, Loss:0.0556, Perplexity:1.0393
Epoch 14, step:8800/9410 93.52%, Loss:0.0530, Perplexity:1.0374
Epoch 14, step:8900/9410 94.58%, Loss:0.0547, Perplexity:1.0387
Epoch 14, step:9000/9410 95.64%, Loss:0.0548, Perplexity:1.0387
Epoch 14, step:9100/9410 96.71%, Loss:0.0530, Perplexity:1.0374
Epoch 14, step:9200/9410 97.77%, Loss:0.0547, Perplexity:1.0386
Epoch 14, step:9300/9410 98.83%, Loss:0.0654, Perplexity:1.0464
Epoch 14, step:9400/9410 99.89%, Loss:0.0890, Perplexity:1.0637
Epoch 14, validation average loss:0.2920, Perplexity:1.2243
Saving checkpoint to ./ckpts/ckpt-14-0.2920.pt
Epoch: 15
Epoch 15, step:100/9410 1.06%, Loss:0.0219, Perplexity:1.0153
Epoch 15, step:200/9410 2.13%, Loss:0.0162, Perplexity:1.0113
Epoch 15, step:300/9410 3.19%, Loss:0.0182, Perplexity:1.0127
Epoch 15, step:400/9410 4.25%, Loss:0.0202, Perplexity:1.0141
Epoch 15, step:500/9410 5.31%, Loss:0.0201, Perplexity:1.0140
Epoch 15, step:600/9410 6.38%, Loss:0.0230, Perplexity:1.0161
Epoch 15, step:700/9410 7.44%, Loss:0.0248, Perplexity:1.0173
Epoch 15, step:800/9410 8.50%, Loss:0.0233, Perplexity:1.0163
Epoch 15, step:900/9410 9.56%, Loss:0.0228, Perplexity:1.0159
Epoch 15, step:1000/9410 10.63%, Loss:0.0328, Perplexity:1.0230
Epoch 15, step:1100/9410 11.69%, Loss:0.0238, Perplexity:1.0166
Epoch 15, step:1200/9410 12.75%, Loss:0.0328, Perplexity:1.0230
Epoch 15, step:1300/9410 13.82%, Loss:0.0287, Perplexity:1.0201
Epoch 15, step:1400/9410 14.88%, Loss:0.0286, Perplexity:1.0200
Epoch 15, step:1500/9410 15.94%, Loss:0.0390, Perplexity:1.0274
Epoch 15, step:1600/9410 17.00%, Loss:0.0357, Perplexity:1.0251
Epoch 15, step:1700/9410 18.07%, Loss:0.0351, Perplexity:1.0246
Epoch 15, step:1800/9410 19.13%, Loss:0.0363, Perplexity:1.0255
Epoch 15, step:1900/9410 20.19%, Loss:0.0403, Perplexity:1.0283
Epoch 15, step:2000/9410 21.25%, Loss:0.0384, Perplexity:1.0270
Epoch 15, step:2100/9410 22.32%, Loss:0.0398, Perplexity:1.0280
Epoch 15, step:2200/9410 23.38%, Loss:0.0368, Perplexity:1.0259
Epoch 15, step:2300/9410 24.44%, Loss:0.0473, Perplexity:1.0333
Epoch 15, step:2400/9410 25.50%, Loss:0.0525, Perplexity:1.0371
Epoch 15, step:2500/9410 26.57%, Loss:0.0489, Perplexity:1.0344
Epoch 15, step:2600/9410 27.63%, Loss:0.0457, Perplexity:1.0322
Epoch 15, step:2700/9410 28.69%, Loss:0.0449, Perplexity:1.0316