vqa_models misjudge the spatial relationship in most of the cases

[yhy-2000]

Hello, thank you for your repo.

When I test some cases with tifa, I find that the model totally confused right and left, below is an example:

what I generate is as follows:

The result by mPLUG is {'id': 'paintskill_29', 'caption': 'a photo of bird and boat; boat is right to bird', 'question': 'is the boat right to or left to the bird?', 'choices': ['right to', 'left to', 'in front of', 'behind'], 'answer': 'right to', 'element_type': 'spatial', 'element': 'right to', 'free_form_vqa': 'left', 'multiple_choice_vqa': 'left to', 'scores': 0, 'pred_image_path': '/share/project/yhy/project/frag/image_editing_pipeline/baseline/LayoutLLM_T2I_main/auto_RAIG_output/tifa/1111.png'}

Similarly, for the image:

The result is
{'id': 'paintskill_14', 'caption': 'a photo of bike and chair; chair is below bike', 'question': 'is the chair below or above the bike?', 'choices': ['below', 'above', 'next to', 'behind'], 'answer': 'below', 'element_type': 'spatial', 'element': 'below', 'free_form_vqa': 'above', 'multiple_choice_vqa': 'above', 'scores': 0, 'pred_image_path': '/share/project/yhy/project/frag/image_editing_pipeline/baseline/LayoutLLM_T2I_main/auto_RAIG_output/tifa/1072.png'}

Is there something wrong?
Can you help me verify this result to test whether it is the bug of my code? (Actually I barely changed the repository code)

Thank you in advance.