Runtume error on a finetuned model

[realtimeriddle]

I'm getting an error when I run pplm with a gpt2 model I finetuned with the language modeling example from the huggingface transformers repo.

run_pplm.py
-B /path/to/BOW.txt
--pretrained_model=/user/FindtunedModelOut
--cond_text="potato"
--num_samples=20
--length=150
--stepsize=0.03
--num_iterations=3
--window_length=5
--gamma=1.5
--gm_scale=0.95
--kl_scale=0.01
--colorama
--verbosity=regular
--sample

and I get the error:

Traceback (most recent call last):
File "/pythonProjects/transformerTest/venv/PPLM/run_pplm.py", line 936, in
run_pplm_example(**vars(args))
File "/pythonProjects/transformerTest/venv/PPLM/run_pplm.py", line 768, in run_pplm_example
unpert_gen_tok_text, pert_gen_tok_texts, _, _ = full_text_generation(
File "/pythonProjects/transformerTest/venv/PPLM/run_pplm.py", line 472, in full_text_generation
pert_gen_tok_text, discrim_loss, loss_in_time = generate_text_pplm(
File "/pythonProjects/transformerTest/venv/PPLM/run_pplm.py", line 584, in generate_text_pplm
pert_past, _, grad_norms, loss_this_iter = perturb_past(
File "/pythonProjects/transformerTest/venv/PPLM/run_pplm.py", line 213, in perturb_past
bow_logits = torch.mm(probs, torch.t(one_hot_bow))
RuntimeError: mat1 dim 1 must match mat2 dim 0

I'm not sure if I screwed up the finetuning or pplm, but the model does generate text with the run_generation example, and if I just change the model to gpt2 pplm runs on the bag of words. Anyone know how to fix this error, or what I am doing wrong?
Thanks.

Edit: The problem seems to have to do with the special tokens I added.

[dathath]

Can you elaborate what "gpt2 pplm runs on the bag of words" means? I don't know if I fully follow what is causing the problem. I haven't seen this error in this context before.

[iedr]

Hi, I have the same problem; I think the issue lies with added tokens in the finetuning process... Any ideas how to solve this?
E.g. I added X number of tokens in the finetuning process. These tokens are added to added_tokens.json, but this addition is not reflected when I run the pplm example. The tokenizer still has the 50257 number of tokens. This error happened in the perturb_past function.

[realtimeriddle]

Can you elaborate what "gpt2 pplm runs on the bag of words" means? I don't know if I fully follow what is causing the problem. I haven't seen this error in this context before.

I meant that pplm runs without error with the pretrained gpt2 model, without my fine-tuneing, using a bag of words. I was trying to relay that pplm works with other models and that the model, not the bag of words, was causing the problem. If I remove the two lines of code that I use to add my special tokens to huggingface's language modeling example, pplm does run.

special_tokens_dict = {'bos_token': '<SOT>', 'eos_token': '<EOT>', 'pad_token': '<PAD>'}
num_added_toks = tokenizer.add_special_tokens(special_tokens_dict)