Skip to content

Latest commit

 

History

History

docs

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
README.md
README.md
 
 
benchmark.md
benchmark.md
 
 
compile.md
compile.md
 
 
map.md
map.md
 
 
small-dic.md
small-dic.md
 
 
train.md
train.md
 
 

README.md

vibrato/docs

Here provides several documents for advanced usages.

  • compile.md describes how to compile system dictionaries from language resources.
  • map.md describes how to achieve faster tokenization through cache-efficient id mapping.
  • train.md describes how to train dictionaries from your own corpus.
  • small-dic.md describes how to generate smaller system dictionaries.
  • benchmark describes how to benchmark the tokenization speed of Vibrato.