SamImageProcessor do_convert_rgb not working correctly

[MSt-10]

System Info

• transformers version: 4.49.0.dev0
• Platform: Linux-5.8.0-63-generic-x86_64-with-glibc2.31
• Python version: 3.11.10
• Huggingface_hub version: 0.26.3
• Safetensors version: 0.4.5
• Accelerate version: 1.1.1
• Accelerate config: not found
• PyTorch version (GPU?): 2.4.0 (True)
• Tensorflow version (GPU?): not installed (NA)
• Flax version (CPU?/GPU?/TPU?): not installed (NA)
• Jax version: not installed
• JaxLib version: not installed
• Using distributed or parallel set-up in script?:
• Using GPU in script?:
• GPU type: NVIDIA RTX 6000 Ada Generation

Who can help?

@qubvel

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

When passing a gray scale PIL image (mode='L') to the SamProcessor, the SamImageProcessor is called to preprocess the image. This includes potentially converting the image to rgb if possible. This is done in the method SamImageProcessor._preprocess_image:

        image = to_numpy_array(image)

        # PIL RGBA images are converted to RGB
        if do_convert_rgb:
            image = convert_to_rgb(image)

        # All transformations expect numpy arrays.
        image = to_numpy_array(image)

Which does the following:

def convert_to_rgb(image: ImageInput) -> ImageInput:
    """
    Converts an image to RGB format. Only converts if the image is of type PIL.Image.Image, otherwise returns the image
    as is.
    Args:
        image (Image):
            The image to convert.
    """
    requires_backends(convert_to_rgb, ["vision"])

    if not isinstance(image, PIL.Image.Image):
        return image

    if image.mode == "RGB":
        return image

    image = image.convert("RGB")
    return image

So when passing a grayscale PIL image to the processor I would expect it to be correctly converted to RGB when setting do_convert_rgb.

The cause of the error is the following line:

transformers/src/transformers/models/sam/image_processing_sam.py

Line 310 in 92c5ca9

image = to_numpy_array(image)

The passed image gets converted to a numpy array before the convert_to_rgb method is called.
Hence, the check if the image is a PIL image fails.
Since there is also a conversion to numpy after the convert_to_rgb method call, I suspect this is just a simple mistake.

Expected behavior

When passing a grayscale PIL image to the processor I would expect it to be correctly converted to RGB.