[WIP] Multi round nbest rescoring #5

pkufool · 2021-08-04T06:49:20Z

There are several decoding methods in icefall now, their pipelines are demonstrated as the picture below. What I want to do in this pull request locates in the red rectangle, it was proposed here(k2-fsa/snowfall#232) several weeks ago.

This is just at the very beginning, just copy the related previous work from k2 & snowfall.

danpovey · 2021-08-27T07:48:38Z

icefall/decode.py

+        est_scores = 1 - 1/2 * (
+            1 + torch.erf(
+                (best_score - path_mean) / torch.sqrt(2 * path_var)
+            )


For finding the best path, instead of trying to use this kind of integral, I would use:
(path_mean - best_score) / torch.sqrt(path_var)
which can be interpreted as the "z score". This has a monotonic relationship with this percentile/integral thingy, and is easier to compute and better behaved numerically.

pkufool · 2021-09-07T12:29:25Z

After fixing some errors, now I can get the same WER (as original attention decoder) using nbest attention rescoring, see the table below:

Decoding method	WER(%) test-clean	WER(%) test-other	Notes
rescore with attention decoder (num-paths=500, max-duration=1, lattice-score-scale=0.5)	2.57	5.94	the original attention rescoring method
rescore all nbest path with attention decoder(num-paths=500, max-duration=1, lattice-score-scale=0.5)	2.57	5.98	rescore all the unique nbest path, should be the same with the above row, just to check the function `rescore_nbest_with_attention_rescorer` is right
rescore topk * 2 nbest path with attention decoder(topk=10,num-paths=500, max-duration=1, lattice-score-scale=0.5)	2.59	5.97	rescore the top2k nbest path (sorted by lattice scores) with attention decoder
rescore topk + topk nbest path with attention decoder(topk=10, num-paths=500, max-duration=1, lattice-score-scale=0.5)	2.59	6.03	rescore topk nbest path (sorted by lattice scores) first, select the other topk path by a small estimation model, then rescore the selected topk nbest path with attention decoder

The future plans are:

Tune the small model that predicting the rescoring score of a path (to select better path).
Support batch processing (only support batch-size=1 now).
Check the code to speed up the decoding (now it spends more time than original attention-decoder method, 29mins vs 20mins or so).

csukuangfj · 2021-09-08T02:13:18Z

icefall/score_estimator.py

+        self,
+        path: Path,
+        model: str,
+    ) -> None:


Could you add documentations to the methods/functions in this file?

csukuangfj · 2021-09-08T02:14:21Z

icefall/score_estimator.py

+import glob
+import logging
+from pathlib import Path
+from typing import Tuple, List


Could you have a look at
https://icefall.readthedocs.io/en/latest/contributing/code-style.html
to follow the code style?

csukuangfj · 2021-09-08T02:15:36Z

icefall/score_estimator.py

+            self.files = files[0: int(len(files) * 0.8)]
+        elif model == 'dev':
+            self.files = files[int(len(files) * 0.8): int(len(files) * 0.9)]
+        elif mode == 'test':


mode or model?

csukuangfj · 2021-09-08T02:18:26Z

icefall/score_estimator.py

+        x = self.embedding(x)
+        x = self.sigmod(x)
+        x = self.output(x)
+        mean, var = x[:, 0], x[:, 1]


var -> log_var?

csukuangfj · 2021-09-08T02:19:05Z

icefall/score_estimator.py

+        "--hidden-dim",
+        type=int,
+        default=20,
+        help="Neural number of didden layer.",


Suggested change

help="Neural number of didden layer.",

help="Neural number of hidden layer.",

csukuangfj · 2021-09-08T02:19:59Z

icefall/score_estimator.py

+        hidden_dim = args.hidden_dim
+    )
+
+    model = model.to("cuda")


Shall we support also CPU?

csukuangfj · 2021-09-08T02:23:46Z

icefall/score_estimator.py

+        step = 0
+        model.eval()
+        for x, y in dev_dataloader:
+            mean, var = model(x.cuda())


I would recommend you to define a variable device, which can be either a CPU or a CUDA device, and use model(x.to(device)).

csukuangfj · 2021-09-08T02:24:41Z

icefall/score_estimator.py

+        dev_loss = 0.0
+        step = 0
+        model.eval()
+        for x, y in dev_dataloader:


Please put the evaluation process in a context disabling gradient computation.

csukuangfj · 2021-09-08T02:25:58Z

icefall/score_estimator.py

+import torch
+import torch.nn as nn
+from torch.utils.data import DataLoader
+from torch.utils.tensorboard import SummaryWriter


Please remove unused imports.
If you have a look at https://icefall.readthedocs.io/en/latest/contributing/code-style.html
and start to use flake8, it will tell you this import is never used.

csukuangfj · 2021-09-08T02:30:17Z

icefall/score_estimator.py

+    parser = get_parser()
+    args = parser.parse_args()
+    torch.manual_seed(42)
+    torch.cuda.manual_seed(42)


Suggested change

torch.cuda.manual_seed(42)

torch.manual_seed(42) already did it.

pkufool added 3 commits August 4, 2021 14:27

Copy the files related to multi round nbest rescoring from k2 & snowfall

cabe8b6

Merge branch 'master' into nbest

286dce7

Add attention rescore pipeline

0669aa8

pkufool marked this pull request as draft August 9, 2021 08:25

Add multi round nbest rescoer

27c46b6

danpovey reviewed Aug 27, 2021

View reviewed changes

danpovey mentioned this pull request Sep 2, 2021

RuntimeError: Specified device cuda:0 does not match device of data cuda:-2 #33

Closed

pkufool added 2 commits September 7, 2021 11:50

Merge with origin master

1ac52e5

Fix the score of the nbest attention rescorer

355e324

pkufool marked this pull request as ready for review September 7, 2021 12:29

csukuangfj reviewed Sep 8, 2021

View reviewed changes

csukuangfj requested changes Sep 8, 2021

View reviewed changes

csukuangfj reviewed Sep 8, 2021

View reviewed changes

pkufool mentioned this pull request Sep 13, 2021

Refactor decode.py to make it more readable and more modular. #44

Merged

Lzhang-hub mentioned this pull request Oct 11, 2021

CUDA out of memory in decoding #70

Open

danpovey mentioned this pull request Nov 27, 2021

Decoding error 'Fsa' object doesn't support assignment. #133

Open

wwxm0523 mentioned this pull request Jan 30, 2022

LF-MMI GPU OOM #196

Open

ahazned mentioned this pull request Apr 13, 2022

Illegal memory error when training with multi-GPU #247

Open

ngoel17 mentioned this pull request Sep 30, 2024

Illegal memory access during zipformer training #1764

Closed

pkufool closed this Nov 21, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[WIP] Multi round nbest rescoring #5

[WIP] Multi round nbest rescoring #5

pkufool commented Aug 4, 2021

danpovey Aug 27, 2021

pkufool commented Sep 7, 2021 •

edited

Loading

csukuangfj Sep 8, 2021

csukuangfj Sep 8, 2021

csukuangfj Sep 8, 2021

csukuangfj Sep 8, 2021

csukuangfj Sep 8, 2021

csukuangfj Sep 8, 2021

csukuangfj Sep 8, 2021

csukuangfj Sep 8, 2021

csukuangfj Sep 8, 2021

csukuangfj Sep 8, 2021

	help="Neural number of didden layer.",
	help="Neural number of hidden layer.",

[WIP] Multi round nbest rescoring #5

[WIP] Multi round nbest rescoring #5

Conversation

pkufool commented Aug 4, 2021

Choose a reason for hiding this comment

pkufool commented Sep 7, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pkufool commented Sep 7, 2021 •

edited

Loading