I am working with LLaVA code and i have a question about sequence length when using pipeline parallelism.
In my understanding, tensor shape for recv, send is fixed using args.seq_length in pipeline_parallel/schedules.py.
And if padding tokens make up most of the input, it becomes very inefficient in terms of memory or execution speed.
Is there any way to use variable input length when using pipeline parallel?
Thanks.
I am working with LLaVA code and i have a question about sequence length when using pipeline parallelism.
In my understanding, tensor shape for recv, send is fixed using args.seq_length in pipeline_parallel/schedules.py.
And if padding tokens make up most of the input, it becomes very inefficient in terms of memory or execution speed.
Is there any way to use variable input length when using pipeline parallel?
Thanks.