Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Verify context/matches for vocab #3

Open
james-s-w-clark opened this issue Apr 19, 2020 · 1 comment
Open

Verify context/matches for vocab #3

james-s-w-clark opened this issue Apr 19, 2020 · 1 comment

Comments

@james-s-w-clark
Copy link
Owner

May have vocab 你好，and content has 你好吗.

Can check dictionary to see if vocab is a subword, in which case e.g. 你好 of 你好吗 should not be highlighted.
@james-s-w-clark
Copy link
Owner Author

Also requires tokenization for no-space languages, will leave for now. Will focus on easier languages to expand usability first.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@james-s-w-clark