Skip to content

[Bug] An error is reported due to too many prompts. #37

@LeeWant

Description

@LeeWant

In the main.py https://github.com/Wenyueh/MinivLLM/blob/main/main.py

    prompts = [
        "introduce yourself",
        "list all prime numbers within 100",
        "give me your opinion on the impact of artificial intelligence on society",
    ] * 30

When the quantity of prompts is multiplied by 30, the following error message will appear.

267 number of processed tokens 306.48394818862164 tokens/sec during prefilling
273 number of processed tokens 3175.8109528388927 tokens/sec during prefilling
276 number of processed tokens 2882.28122676026 tokens/sec during prefilling
267 number of processed tokens 2672.4301140895254 tokens/sec during prefilling
[rank0]: Traceback (most recent call last):
[rank0]:   File "/home/lee/project/MinivLLM/main.py", line 78, in <module>
[rank0]:     main()
[rank0]:   File "/home/lee/project/MinivLLM/main.py", line 66, in main
……
……
……

However, using the following prompts, minivllm will run normally

    prompts = [
        "introduce yourself" * 15,
        "list all prime numbers within 100" * 15,
        "give me your opinion on the impact of artificial intelligence on society" * 15,
    ] * 30

Metadata

Metadata

Assignees

Labels

No labels
No labels

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions