-
Notifications
You must be signed in to change notification settings - Fork 78
[Bug] An error is reported due to too many prompts. #37
Copy link
Copy link
Open
Description
In the main.py https://github.com/Wenyueh/MinivLLM/blob/main/main.py
prompts = [
"introduce yourself",
"list all prime numbers within 100",
"give me your opinion on the impact of artificial intelligence on society",
] * 30
When the quantity of prompts is multiplied by 30, the following error message will appear.
267 number of processed tokens 306.48394818862164 tokens/sec during prefilling
273 number of processed tokens 3175.8109528388927 tokens/sec during prefilling
276 number of processed tokens 2882.28122676026 tokens/sec during prefilling
267 number of processed tokens 2672.4301140895254 tokens/sec during prefilling
[rank0]: Traceback (most recent call last):
[rank0]: File "/home/lee/project/MinivLLM/main.py", line 78, in <module>
[rank0]: main()
[rank0]: File "/home/lee/project/MinivLLM/main.py", line 66, in main
……
……
……
However, using the following prompts, minivllm will run normally
prompts = [
"introduce yourself" * 15,
"list all prime numbers within 100" * 15,
"give me your opinion on the impact of artificial intelligence on society" * 15,
] * 30
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels