AutoModelForCausalLM.from_pretrained() fails: "TypeError: 'type' object is not subscriptable"

[neuron-whisperer]

I'm trying to run moondream on a Raspberry Pi 5 (8gb) running Raspberry Pi OS Bookworm Lite (fully up-to-date) and Python 3.8.19, which is the latest version of Python that is compatible with pygame 1.x (the latest version of pygame that runs on Raspberry Pi Lite).

I followed the install instructions and example script upt to the AUtoModelForCausalLM.form_pretrained statement, which generated this error:

>>> model = AutoModelForCausalLM.from_pretrained(
...     model_id, trust_remote_code=True, revision=revision
... )
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/pi/.pyenv/versions/3.8.19/lib/python3.8/site-packages/transformers/models/auto/auto_factory.py", line 551, in from_pretrained
    model_class = get_class_from_dynamic_module(
  File "/home/pi/.pyenv/versions/3.8.19/lib/python3.8/site-packages/transformers/dynamic_module_utils.py", line 514, in get_class_from_dynamic_module
    return get_class_in_module(class_name, final_module)
  File "/home/pi/.pyenv/versions/3.8.19/lib/python3.8/site-packages/transformers/dynamic_module_utils.py", line 212, in get_class_in_module
    module_spec.loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 843, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/pi/.cache/huggingface/modules/transformers_modules/vikhyatk/moondream2/92d3d73b6fd61ab84d9fe093a9c7fd8c04bf2c0d/moondream.py", line 9, in <module>
    from .vision_encoder import VisionEncoder
  File "/home/pi/.cache/huggingface/modules/transformers_modules/vikhyatk/moondream2/92d3d73b6fd61ab84d9fe093a9c7fd8c04bf2c0d/vision_encoder.py", line 209, in <module>
    class VisionEncoder(nn.Module):
  File "/home/pi/.cache/huggingface/modules/transformers_modules/vikhyatk/moondream2/92d3d73b6fd61ab84d9fe093a9c7fd8c04bf2c0d/vision_encoder.py", line 251, in VisionEncoder
    self, images: Union[PIL.Image.Image, list[PIL.Image.Image], torch.Tensor]
TypeError: 'type' object is not subscriptable

[vikhyat]

Could you try using the Python client library? https://pypi.org/project/moondream/

I have it set to require Python 3.10+ for now since I haven't been able to test older versions, but if you pass --ignore-requires-python it should ignore that check.

[Aritra02091998]

How to solve it, in the transformer implementation ? I need to finetune the model