In this section the code is for creating saliency heatmaps and comparing them with the human gaze heatmaps created by using the Reader class. To execute the files, you might need additional code from the parent folder and the asterix folder of this repository and data as e.g. the model`s weights or the saliency database we created and change some paths in the files.
- integrated_gradients.py contains all needed functions to compute the integrated gradients of a model (Source).
- perturbation_for_sarfa.py contains all the functions for perturbing an image and calculating the SARFA-saliency. For the calculation process it uses a function from the file sarfa_saliency.py which was made publically availble by the original authors on the Github accompanying their Paper [1].
- heatmap_comparison.py contains the functions for the comparison of the gaze data (e.g. gaze heatmap) and the saliency heatmaps.
Main Files:
- create_saliency_database_main.py calculates the SARFA-saliency heatmaps for the given area and stepsize (as variables) and saves them as png. (The created dataset we used for the comparison can be found here).
- all_saliency_comparison_main.py compares the saliency maps we put in the dataset using one of the methodes presented in the report. Which one is chosen by a variable.
- average_pixel_comaprison_main.py compares the average pixels being one when converting the human gaze heatmap and saliency heatmaps to binary maps and the value for the fixation maps (already binary) and also their average pixel value originally.
- comparison_finding_max_min_main.py this main finds the max and min value of a comparison method and returns the value and index. If the method is correlation is also visualizes both heatmaps that are compared as well as their correlation heatmap.
saliency_results :
In this folder there are textfiles containing the output of all_saliency_comparison_main.py for different methods as well as the output of average_pixel_comaprison_main.py and comparison_finding_max_min_main.py using correlation. All averaged values are also in the all_results.csv file and in the file list_all_result_values.csv are the calculated values for each single data pair.
Paper_Image_Creation_Code :
In this folder are several main files that create visualizations for our report. They need other files to work, but are stuck in this folder, because they where only for creating these visualizations.
[1] Puri, N., Verma, S., Gupta, P., Kayastha, D., Deshmukh, S., Krishnamurthy, B., & Singh, S. (2020). Explain your move: Understanding agent actions using specific and relevant feature attribution. International Conference on Learning Representations (ICLR).