Skip to content

v0.3.1

Choose a tag to compare

@mattdangerw mattdangerw released this 11 Nov 22:05

Summary

  • Add keras_nlp.tokenizers.BytePairTokenizer with tf.data friendly support for the tokenization used by GPT-2, RoBERTa and other models.
  • Remove the hard dependency on tensorflow and tensorflow-text when pip installing on MacOS, to accommodate M1 chips. See this section of our contributor guide for more information on MacOS development.

What's Changed

Full Changelog: v0.3.0...v0.3.1