Version 0.2.1

• version 0.2.1
  • Update some free function.

Version 0.2.0

• version 0.2.0
  • Fix EncodeWithIds, WordPieceTokenize API return error result.
  • Add Github Workflows.
  • Update test code.

Version 0.1.0

• version 0.1.0
  • Finish API initTokenizer, encode, encodeWithIds, wordPieceTokenize in CGO.
  • Finish API NewTokenizer, Close, Encode, EncodeWithIds, WordPieceTokenize in Golang.