Feature request: expose tokenizer in public API

[Harry-Chen]

This library has done a great job in VHDL handling with Rust.
However I found it impossible to run only lexical analysis with current public APIs provided by vhdl_lang.

Specifically, I would like to obtain a TokenStream from source files to run some plagiarism detection algorithms on. I expect this library to work in a similar way as verilog_lang does, as shown in https://github.com/jiegec/verilog-lang/blob/master/src/examples/lex.rs. Running only tokenizer not only saves time of parsing, but keeps the location information of each token (which our algorithm needs for visualization) as well.

Thanks!

[kraigher]

You are welcome to make a PR.