strides() called on an undefined Tensor

[sankexin]

Quantizing model.layers.0:   0%|                                                                                                                                                                                                   | 0/28 [00:00<?, ?it/s]Traceback (most recent call last):
  File "/usr/local/bin/auto-round", line 8, in <module>
    sys.exit(run())
  File "/home/quantize/auto-round_bak/auto_round/__main__.py", line 53, in run
    tune(args)
  File "/home/quantize/auto-round_bak/auto_round/script/llm.py", line 604, in tune
    model, folders = autoround.quantize_and_save(export_dir, format=args.format)
  File "/home/quantize/auto-round_bak/auto_round/autoround.py", line 825, in quantize_and_save
    model, _ = self.quantize()
  File "/home/quantize/auto-round_bak/auto_round/autoround.py", line 1574, in quantize
    self._quantize_blocks(
  File "/home/quantize/auto-round_bak/auto_round/autoround.py", line 2763, in _quantize_blocks
    q_input, input_ids = quantize_block(
  File "/home/quantize/auto-round_bak/auto_round/autoround.py", line 2606, in _quantize_block
    self._scale_loss_and_backward(scaler, loss)
  File "/home/quantize/auto-round_bak/auto_round/autoround.py", line 2978, in _scale_loss_and_backward
    scale_loss.backward()
  File "/usr/local/lib/python3.10/dist-packages/torch/_tensor.py", line 521, in backward
    torch.autograd.backward(
  File "/usr/local/lib/python3.10/dist-packages/torch/autograd/__init__.py", line 289, in backward
    _engine_run_backward(
  File "/usr/local/lib/python3.10/dist-packages/torch/autograd/graph.py", line 769, in _engine_run_backward
    return Variable._execution_engine.run_backward(  # Calls into the C++ engine to run the backward pass
RuntimeError: strides() called on an undefined Tensor

auto_round-0.6.1.dev0 has this problem, but auto_round-0.6.0 is normal.

[wenhuach21]

Thanks for reporting this issue. We would appreciate if you could provide more information to help reproduce this issue, e.g. the cmd you are using