Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use log probs for paraformer #120

Merged
merged 2 commits into from
Apr 9, 2023
+15 −11
Merged

Use log probs for paraformer #120

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
4451e47
Use log probs for paraformer
pengzhendong Apr 9, 2023
ef785cf
Fix
pengzhendong Apr 9, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions sherpa-onnx/csrc/math.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -103,5 +103,18 @@ std::vector<int32_t> TopkIndex(const T *vec, int32_t size, int32_t topk) {
return index;
}

template <class T>
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please follow

sherpa-onnx/sherpa-onnx/csrc/offline-ctc-greedy-search-decoder.cc

Lines 35 to 39 in 80060c2

auto y = static_cast<int64_t>(std::distance(
static_cast<const float *>(p_log_probs),
std::max_element(
static_cast<const float *>(p_log_probs),
static_cast<const float *>(p_log_probs) + vocab_size)));

to use std::max_element to replace ArgMax().

You don't need to reimplement the wheel.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

p_log_probs is const float *, why do we use static_cast again?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

p_log_probs is const float *, why do we use static_cast again?

yes, you are right. please remove the cast.

int32_t ArgMax(const T *vec, int32_t size) {
int32_t max_index = 0;
T max_value = vec[0];
for (int32_t i = 1; i < size; i++) {
if (vec[i] > max_value) {
max_value = vec[i];
max_index = i;
}
}
return max_index;
}

} // namespace sherpa_onnx
#endif // SHERPA_ONNX_CSRC_MATH_H_
3 changes: 1 addition & 2 deletions sherpa-onnx/csrc/offline-paraformer-decoder.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,7 @@ class OfflineParaformerDecoder {
/** Run beam search given the output from the paraformer model.
*
* @param log_probs A 3-D tensor of shape (N, T, vocab_size)
* @param token_num A 2-D tensor of shape (N, T). Its dtype is int64_t.
* log_probs[i].argmax(axis=-1) equals to token_num[i]
* @param token_num A 1-D tensor of shape (N). token_num equals to T.
*
* @return Return a vector of size `N` containing the decoded results.
*/
Expand Down
20 changes: 12 additions & 8 deletions sherpa-onnx/csrc/offline-paraformer-greedy-search-decoder.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,26 +6,30 @@

#include <vector>

#include "sherpa-onnx/csrc/math.h"

namespace sherpa_onnx {

std::vector<OfflineParaformerDecoderResult>
OfflineParaformerGreedySearchDecoder::Decode(Ort::Value /*log_probs*/,
Ort::Value token_num) {
std::vector<int64_t> shape = token_num.GetTensorTypeAndShapeInfo().GetShape();
OfflineParaformerGreedySearchDecoder::Decode(Ort::Value log_probs,
Ort::Value /*token_num*/) {
std::vector<int64_t> shape = log_probs.GetTensorTypeAndShapeInfo().GetShape();
int32_t batch_size = shape[0];
int32_t num_tokens = shape[1];
int32_t vocab_size = shape[2];

std::vector<OfflineParaformerDecoderResult> results(batch_size);

const int64_t *p = token_num.GetTensorData<int64_t>();
const float *p = log_probs.GetTensorData<float>();
for (int32_t i = 0; i != batch_size; ++i) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
const float *p = log_probs.GetTensorData<float>();
for (int32_t i = 0; i != batch_size; ++i) {
for (int32_t i = 0; i != batch_size; ++i) {
const float *p = log_probs.GetTensorData<float>() + i * num_tokens * vocab_size;

for (int32_t k = 0; k != num_tokens; ++k) {
if (p[k] == eos_id_) break;
int32_t max_idx = ArgMax(p, vocab_size);
if (max_idx == eos_id_) break;

results[i].tokens.push_back(p[k]);
}
results[i].tokens.push_back(max_idx);

p += num_tokens;
p += vocab_size;
}
}

return results;
Expand Down
2 changes: 1 addition & 1 deletion sherpa-onnx/csrc/offline-paraformer-greedy-search-decoder.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ class OfflineParaformerGreedySearchDecoder : public OfflineParaformerDecoder {
: eos_id_(eos_id) {}

std::vector<OfflineParaformerDecoderResult> Decode(
Ort::Value /*log_probs*/, Ort::Value token_num) override;
Ort::Value log_probs, Ort::Value /*token_num*/) override;

private:
int32_t eos_id_;
Expand Down