Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build报错 #121

Open
zjjznw123 opened this issue May 24, 2024 · 1 comment
Open

build报错 #121

zjjznw123 opened this issue May 24, 2024 · 1 comment

Comments

@zjjznw123
Copy link

python build.py --hf_model_dir ./tmp/Qwen/7B
--quant_ckpt_path ./tmp/Qwen/7B/int4-gptq/gptq_model-4bit-128g.safetensors
--dtype float16
--remove_input_padding
--use_gpt_attention_plugin float16
--enable_context_fmha
--use_gemm_plugin float16
--use_weight_only
--weight_only_precision int4_gptq
--per_group
--world_size 1
--tp_size 1
--output_dir ./tmp/Qwen/7B/trt_engines/int4-gptq/1-gpu
报错: File "/app/tensorrt_llm/examples/qwen/build.py", line 770, in
build(0, args)
File "/app/tensorrt_llm/examples/qwen/build.py", line 735, in build
engine = build_rank_engine(
File "/app/tensorrt_llm/examples/qwen/build.py", line 498, in build_rank_engine
tensorrt_llm_qwen = QWenForCausalLM_TRT(
File "/app/tensorrt_llm/examples/qwen/model.py", line 1027, in init
super().init(
File "/app/tensorrt_llm/examples/qwen/model.py", line 885, in init
self.layers = ModuleList([
File "/app/tensorrt_llm/examples/qwen/model.py", line 886, in
QWenBlock(
File "/app/tensorrt_llm/examples/qwen/model.py", line 692, in init
self.attention = QWenAttention(
File "/app/tensorrt_llm/examples/qwen/model.py", line 317, in init
self.dense = RowLinear(hidden_size,
TypeError: RowLinear.init() got an unexpected keyword argument 'instance_id'

@Tlntin
Copy link
Owner

Tlntin commented May 24, 2024

qwen1需要用0.7.0的tensorrt-llm

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants