How do you understand semantic and acoustic?

[wuzhiyue111]

I saw "rich semantics" in WavTokenizer. What does this rich semantics mean? ASR probe or Emotion probe or something else？

[novateurjsp]

I saw "rich semantics" in WavTokenizer. What does this rich semantics mean? ASR probe or Emotion probe or something else？

we use arch benchmark to evaluation.

[wuzhiyue111]

I saw "rich semantics" in WavTokenizer. What does this rich semantics mean? ASR probe or Emotion probe or something else？

we use arch benchmark to evaluation.

The WavTokenizer's reconstruction is worse than that of the Encodec and DAC. If its reconstruction is poor, it means it's lost information at its bottleneck. When I use the WavTokenizer to reconstruct music, it's even out of tune. I don't think that even if the WavTokenizer's representation metrics are higher than those of the DAC and Encodec, it doesn't necessarily mean it contains more semantic information than the DAC and Encodec. Semantic is just a pattern that is easily discovered by downstream probes, but if it cannot be reconstructed, it is very difficult to explain it as "Semantic Rich". I don't know how the author views this issue. I think in an end-to-end paradigm, it should at least be compared with mimi 1 or speechtokenizer 2.