chore: Updated comment in tests/test_tokenization.py

Co-authored-by: Felix Stollenwerk <[email protected]>
Modalities · Jun 14, 2024 · 5303be6 · 5303be6
1 parent 1cffee8
commit 5303be6
Showing 1 changed file with 1 addition and 1 deletion.
diff --git a/tests/test_tokenization.py b/tests/test_tokenization.py
@@ -172,7 +172,7 @@ def _assert_tokenization(tokenizer: TokenizerWrapper):
             15,
             0,
         ),
-        # If truncation=False and len(text)<model max length, we want a sequence to be truncated to the max_length, irrespective of the padding flag.
+        # If truncation=False and len(text)<model max length, we want a sequence to be unmodified, irrespective of the padding flag.
         (
             "AAAAAAAA" * 15,
             PreTrainedHFTokenizerConfig(