Skip to content
This repository has been archived by the owner on Apr 23, 2024. It is now read-only.

compatibility of youtokentome.BPE outputs with the official implementation #93

Open
l-k-11235 opened this issue Jan 11, 2022 · 0 comments

Comments

@l-k-11235
Copy link

Hello !
Is it possible to make youtokentome.BPE outputs compatible with official implementation of bpe_dropout ? (https://github.com/rsennrich/subword-nmt#how-implementation-differs-from-sennrich-et-al-2016)
The matter is that when building the merge operations table we have the space symbol "▁" and not the end-of-word token “”.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant