awesome-perception-with-neural-networks

Paper list for papers that relate artifitial neural networks with human visual perception. Work-in-progress.

Feel free to suggest relevant papers in the following format.

Table of Contents

• Analizing replications of visual illusions with neural networks
• Generation of visual illusions
• Talk and Tutorial
• TO READ

Analizing replications of visual illusions with neural networks

1. What are lightness illusions and why do we see them?
   Corney, D., & Lotto, R. B PLoS computational biology 2007 paper
   Note: first paper; lightness/brightness illusions;

2. Convolutional Neural Networks Can Be Deceived by Visual Illusions
   Gomez-Villa, A., Martin, A., Vazquez-Corral, J., & Bertalmio, M. CVPR 2019 paper
   Note: lightness/brightness & color illusions;

3. Color illusions also deceive CNNs for low-level vision tasks: Analysis and implications
   Gomez-Villa, A., Martín, A., Vazquez-Corral, J., Bertalmío, M., & Malo, J. Vision Research 2020 paper
   Note: lightness/brightness & color illusions;

4. Grounding Visual Illusions in Language: Do Vision-Language Models Perceive Illusions Like Humans?
   Zhang, Y., Pan, J., Yuchen, Z., Pan, R. & Chai, J. EMNLP 2023 paper
   Note: lightness/brightness,color, geometric, and perspective illusions;

5. Is clip fooled by optical illusions? Ngo, J., Sankaranarayanan, S., & Isola, P. Tiny paper at ICLR 2023 paper
   Note: lightness/brightness,color, geometric, and perspective illusions;

6. Decoding Illusion Perception: A Comparative Analysis of Deep Neural Networks in the Müller-Lyer Illusion Zhang, H., Yoshida, S., & Li, Z. SMC 2023 paper
   Note:

7. Exploring Deep Neural Networks in Simulating Human Vision through Five Optical Illusions Zhang, H., & Yoshida, S. Applied Sciences 2024 paper
   Note:

8. Evaluating Model Perception of Color Illusions in Photorealistic Scenes Mao, L., Tang, Z., & Suhr, A. arXiv preprint 2024 paper
   Note:

9. The Art of Deception: Color Visual Illusions and Diffusion Models Gomez-Villa, A., Wang, K., Parraga, A. C., Twardowski, B., Malo, J., Vazquez-Corral, J., & van de Weijer, J. arXiv preprint 2024 paper
   Note:

10. Deep learning models for perception of brightness related illusions Mukherjee, A., Paul, A., & Ghosh, K. Applied Intelligence 2024 paper
    Note: Shape/contour illusions

11. Challenging deep learning models with image distortion based on the abutting grating illusion Fan, J., & Zeng, Y. Patterns 2023 paper
    Note: Shape/contour illusions

12. A machine learning model perceiving brightness optical illusions: Quantitative evaluation with psychophysical data Kubota, Y., Hiyama, A., & Inami, M. Augmented Humans International Conference 2021 paper
    Note:

13. In the shadow, Neural Networks tricked by visual illusions Zeng, Z. Master of science thesis 2024 paper
    Note: 3D scene reconstruction model can be deceived by shading illusions.

14. Studying Geometric Optical Illusions through the Lens of a Convolutional Neural Network LaBerge, N. CMC Senior Theses 2019 paper
    Note:

15. Neural Networks Trained on Natural Scenes Exhibit Gestalt Closure Kim, B., Reif, E., Wattenberg, M. et al. Comput Brain Behav 2021 paper
    Note:

16. Shared visual illusions between humans and artificial neural networks Benjamin, A., Qiu, C., Zhang, L. Q., Kording, K., & Stocker, A. Conference on Cognitive Computational Neuroscience 2019 paper
    Note:

17. ImageNet-trained deep neural networks exhibit illusion-like response to the Scintillating grid Sun, E. D., & Dekel, R. Journal of Vision 2021 paper
    Note:

18. Exploring perceptual illusions in deep neural networks Ward, E. J. Journal of Vision 2019 paper
    Note:

Generation of visual illusions

1. On the synthesis of visual illusions using deep generative models
   Gomez-Villa, A., Martín, A., Vazquez-Corral, J., Bertalmío, M., & Malo, J. JOV 2022. paper
   Note: lightness/brightness & color illusions;

2. Color Visual Illusions: A Statistics-based Computational Model
   Hirsch, E., & Tal, A. (n.d.) NeurIPS 2020. paper
   Note: lightness/brightness & color illusions;

3. Motion illusion-like patterns extracted from photo and art images using predictive deep neural networks
   Kobayashi, T., Kitaoka, A., Kosaka, M., Tanaka, K., & Watanabe, E. Scientific Reports 2020. paper
   Note: Motion illusions;

4. Evolutionary Generation of Visual Motion Illusions
   Sinapayen, L., & Watanabe, E. arXiv 2021 paper
   Note: Motion illusions;

5. Visual anagrams: Generating multi-view optical illusions with diffusion models
   Geng, D., Park, I., & Owens, A. CVPR 2024 paper
   Note: Motion illusions;

Datasets

1. Optical illusions images dataset
   Williams, R. M., & Yampolskiy, R. V. INSAM Journal of Contemporary Music, Art and Technology 2019 paper
   Note:

2. IllusionVQA: A Challenging Optical Illusion Dataset for Vision Language Models
   Shahgir, H. S., Sayeed, K. S., Bhattacharjee, A., Ahmad, W. U., Dong, Y., & Shahriyar, R. COLM 2024 paper
   Note:

3. BRI3L: A Brightness Illusion Image Dataset for Identification and Localization of Regions of Illusory Perception.
   Roy, A., Roy, A., Mitra, S., & Ghosh, K. ICIP 2024 paper
   Note:

4. IllusionBench: A Large-scale and Comprehensive Benchmark for Visual Illusion Understanding in Vision-Language Models
   Zhang, Y., Zhang, Z., Wei, X., Liu, X., Zhai, G., & Min, X. arXiv preprint 2025 paper
   Note:

Deep neural networks are similar to human visual system

1. What do deep neural networks tell us about biological vision?
   Heinke, D., Leonardis, A., & Leek, E. C. Vision Research 2022 paper
   Note:

2. Gloss perception: Searching for a deep neural network that behaves like humans
   Prokott, K. E., Tamura, H., & Fleming, R. W. Journal of Vision 2021 paper
   Note:

3. How convolutional neural network architecture biases learned opponency and color tuning
   Harris, E., Mihai, D., & Hare, J. Neural Computation 2021 paper
   Note:

4. Deep Learning Networks and Visual Perception
   Lindsay, G. W., & Serre, T. Oxford Research Encyclopedia of Psychology 2021 paper
   Note:

5. Qualitative similarities and differences in visual object representations between brains and deep networks
   Jacob, G., Pramod, R. T., Katti, H., & Arun, S. P. Nature communications 2021 paper
   Note:

6. Deep neural networks for modeling visual perceptual learning
   Wenliang, L. K., & Seitz, A. R. Journal of Neuroscience 2018 paper
   Note:

Deep neural networks are different from human visual system

1. A comparative biology approach to DNN modeling of vision: A focus on differences, not similarities
   Lonnqvist, B., Bornet, A., Doerig, A., & Herzog, M. H. Journal of vision 2021 paper
   Note:

2. Deep problems with neural network models of human vision
   Bowers, J. S., Malhotra, G., Dujmović, M., Montero, M. L., Tsvetkov, C., Biscione, V., ... & Blything, R. Behavioral and Brain Sciences 2023 paper
   Note:

3. Generalisation in humans and deep neural networks
   Geirhos, R., Temme, C. R., Rauber, J., Schütt, H. H., Bethge, M., & Wichmann, F. A. NeurIPS 2018 paper
   Note:

4. Clarifying status of DNNs as models of human vision
   Bowers, J. S., Malhotra, G., Dujmović, M., Montero, M. L., Tsvetkov, C., Biscione, V., ... & Blything, R. Behavioral and Brain Sciences 2023 paper
   Note:

5. Inconsistent illusory motion in predictive coding deep neural networks.
   Kirubeswaran, O. R., & Storrs, K. R. Vision Research 2023 paper Note:

6. Five points to check when comparing visual perception in humans and machines
   Funke, C. M., Borowski, J., Stosio, K., Brendel, W., Wallis, T. S., & Bethge, M. Journal of Vision 2021 paper Note:

7. Are deep neural networks adequate behavioral models of human visual perception?
   Wichmann, F. A., & Geirhos, R. Annual Review of Vision Science 2023 paper Note:

8. Deep learning models fail to capture the configural nature of human shape perception
   Baker, N., & Elder, J. H. Iscience 2022 paper Note:

Artificial Psychophysics

1. Psychophysics may be the game-changer for deep neural networks (DNNs) to imitate the human vision.
   Chandran, K. S., Paul, A. M., Paul, A., & Ghosh, K. Behavioral and Brain Sciences 2023 paper Note:

2. Informing Machine Perception With Psychophysics
   Dulay, J., Poltoratski, S., Hartmann, T. S., Anthony, S. E., & Scheirer, W. J. Proceedings of the IEEE 2024 paper Note:

3. MindSet: Vision. A toolbox for testing DNNs on key psychological experiments
   Biscione, V., Yin, D., Malhotra, G., Dujmovic, M., Montero, M. L., Puebla, G., ... & Bowers, J. S. arXiv preprint 2024 paper Note:

4. Artificial perception meets psychophysics, revealing a fundamental law of illusory motion
   Kobayashi, T., & Watanabe, E. arXiv preprint 2021 paper Note:

Aligning deep neural networks with human perception

1. Harmonizing the object recognition strategies of deep neural networks with humans
   Fel, T., Rodriguez Rodriguez, I. F., Linsley, D., & Serre, T. NeurIPS 2022 paper Note:

2. Recurrent neural circuits for contour detection
   Linsley, Drew, Junkyung Kim, Alekh Ashok, and Thomas Serre ICLR 2020 paper Note:

3. Ecological Data and Objectives for Human Alignment
   Nagaraj, Akash, Alekh Karkada Ashok, Drew Linsley, Francis E. Lewis, Peisen Zhou, and Thomas Serre ccneuro 2024 paper Note:

4. CYBORG: Blending human saliency into the loss improves deep learning-based synthetic face detection
   Boyd, A., Tinsley, P., Bowyer, K. W., & Czajka, A. WACV 2023 paper Note:

5. Exploring Primitive Visual Measurement Understanding and the Role of Output Format in Learning in Vision-Language Models
   Ankit Yadav, Lingqiao Liu, Yuankai Qi, Arxiv preprint 2025 paper Note:

6. Can We Talk Models Into Seeing the World Differently?
   Paul Gavrikov and Jovita Lukasik and Steffen Jung and Robert Geirhos and Muhammad Jehanzeb Mirza and Margret Keuper and Janis Keuper, ICLR 2025 paper Note: