RuntimeError: An image must be set with .set_image(...) before mask prediction.

[gaohy63601-droid]

Dear author, I recently tried to use groundingdino_sam in a segmentation task. However, when I ran the code, it shows this error, do you know why this problem happens?

Traceback (most recent call last):
File "foreground_utils.py", line 50, in extract_foreground_pil
masks, _, _ = sam2_predictor.predict(
File "sam2/sam2_image_predictor.py", line 281, in predict
raise RuntimeError(
RuntimeError: An image must be set with .set_image(...) before mask prediction.

These are my code:

image_source, image = load_image(pil_image)
sam2_predictor.set_image(image_source)

boxes, _, _ = predict(
model=grounding_model,
image=image,
caption=text_prompt,
box_threshold=BOX_THRESHOLD,
text_threshold=TEXT_THRESHOLD,
device=device
)

h, w, _ = image_source.shape
boxes = boxes * torch.Tensor([w, h, w, h])
input_boxes = box_convert(boxes=boxes, in_fmt="cxcywh", out_fmt="xyxy").numpy()

with torch.autocast(device_type=device.split(":")[0], dtype=torch.bfloat16):
masks, _, _ = sam2_predictor.predict(
point_coords=None,
point_labels=None,
box=input_boxes,
multimask_output=False,
)

[rentainhe]

Maybe you can refer to this for the correct SAM API usage: https://github.com/facebookresearch/sam2/blob/main/notebooks/image_predictor_example.ipynb