Skip to content

Commit

Permalink
fix: use renamed tokenizer file name
Browse files Browse the repository at this point in the history
  • Loading branch information
lllAlexanderlll committed Feb 19, 2024
1 parent a0513e3 commit bc03021
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion tests/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ def indexed_dummy_data_path(dummy_data_path) -> DataPathCollection:

@pytest.fixture
def gpt2_tokenizer() -> GPT2TokenizerFast:
default_gpt2_tokenizer_path = Path(__file__).parents[1] / Path("data", "tokenizer", "tokenizer.json")
default_gpt2_tokenizer_path = Path(__file__).parents[1] / Path("data", "tokenizer", "tokenizer_gpt2.json")
assert default_gpt2_tokenizer_path.is_file()
return GPT2TokenizerFast(tokenizer_file=str(default_gpt2_tokenizer_path))

Expand Down

0 comments on commit bc03021

Please sign in to comment.