We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
python build.py --hf_model_dir ./tmp/Qwen/7B --quant_ckpt_path ./tmp/Qwen/7B/int4-gptq/gptq_model-4bit-128g.safetensors --dtype float16 --remove_input_padding --use_gpt_attention_plugin float16 --enable_context_fmha --use_gemm_plugin float16 --use_weight_only --weight_only_precision int4_gptq --per_group --world_size 1 --tp_size 1 --output_dir ./tmp/Qwen/7B/trt_engines/int4-gptq/1-gpu 报错: File "/app/tensorrt_llm/examples/qwen/build.py", line 770, in build(0, args) File "/app/tensorrt_llm/examples/qwen/build.py", line 735, in build engine = build_rank_engine( File "/app/tensorrt_llm/examples/qwen/build.py", line 498, in build_rank_engine tensorrt_llm_qwen = QWenForCausalLM_TRT( File "/app/tensorrt_llm/examples/qwen/model.py", line 1027, in init super().init( File "/app/tensorrt_llm/examples/qwen/model.py", line 885, in init self.layers = ModuleList([ File "/app/tensorrt_llm/examples/qwen/model.py", line 886, in QWenBlock( File "/app/tensorrt_llm/examples/qwen/model.py", line 692, in init self.attention = QWenAttention( File "/app/tensorrt_llm/examples/qwen/model.py", line 317, in init self.dense = RowLinear(hidden_size, TypeError: RowLinear.init() got an unexpected keyword argument 'instance_id'
The text was updated successfully, but these errors were encountered:
qwen1需要用0.7.0的tensorrt-llm
Sorry, something went wrong.
No branches or pull requests
python build.py --hf_model_dir ./tmp/Qwen/7B
--quant_ckpt_path ./tmp/Qwen/7B/int4-gptq/gptq_model-4bit-128g.safetensors
--dtype float16
--remove_input_padding
--use_gpt_attention_plugin float16
--enable_context_fmha
--use_gemm_plugin float16
--use_weight_only
--weight_only_precision int4_gptq
--per_group
--world_size 1
--tp_size 1
--output_dir ./tmp/Qwen/7B/trt_engines/int4-gptq/1-gpu
报错: File "/app/tensorrt_llm/examples/qwen/build.py", line 770, in
build(0, args)
File "/app/tensorrt_llm/examples/qwen/build.py", line 735, in build
engine = build_rank_engine(
File "/app/tensorrt_llm/examples/qwen/build.py", line 498, in build_rank_engine
tensorrt_llm_qwen = QWenForCausalLM_TRT(
File "/app/tensorrt_llm/examples/qwen/model.py", line 1027, in init
super().init(
File "/app/tensorrt_llm/examples/qwen/model.py", line 885, in init
self.layers = ModuleList([
File "/app/tensorrt_llm/examples/qwen/model.py", line 886, in
QWenBlock(
File "/app/tensorrt_llm/examples/qwen/model.py", line 692, in init
self.attention = QWenAttention(
File "/app/tensorrt_llm/examples/qwen/model.py", line 317, in init
self.dense = RowLinear(hidden_size,
TypeError: RowLinear.init() got an unexpected keyword argument 'instance_id'
The text was updated successfully, but these errors were encountered: