Summary of RGB-T Salient Object Detection, Semantic segmentation and Crowd Counting

Provide a summary of RGB-T-Salient-Object-Detection, Semantic segmentation and Crowd Counting
(Paper, Code, Dataset, Evaluation and more).

---

🏃 keep updating. 🏃
🚩2024.5.26 Update.

---

Content:

1. RGB-T Salient Object Detection
2. RGB-T Semantic segmentation
3. RGB-T Crowd Counting
4. Dataset
5. Evaluation
6. Other Summary
7. Acknowledgement

---

RGB-T Salient Object Detection

2017

No.	Pub.	Title	Links
01	ISCID	Learning Multiscale Deep Features and SVM Regressors for Adaptive RGB-T Saliency Detection	Paper/Code

2018

No.	Pub.	Title	Links
01	IGTA	RGB-T Saliency Detection Benchmark: Dataset, Baselines, Analysis and a Novel Approach	Paper/Code

2019

No.	Pub.	Title	Links
01	MIPR	M3S-NIR: Multi-Modal Multi-Scale Noise-Insensitive Ranking for RGB-T Saliency Detection	Paper/Code
02	TMM	RGB-T Image Saliency Detection via Collaborative Graph Learning	Paper/Code
03	TCSVT	RGBT Salient Object Detection: Benchmark and A Novel Cooperative Ranking Approach	Paper/Code

2020

No.	Pub.	Title	Links
01	TIP	RGB-T Salient Object Detection via Fusing Multi-Level CNN Features	Paper/Code
02	TCSVT	Revisiting Feature Fusion for RGB-T Salient Object Detection	Paper/Code

2021

No.	Pub.	Title	Links
01	TCSVT	ECFFNet: Effective and Consistent Feature Fusion Network for RGB-T Salient Object Detection	Paper/Results(pin:tx48)
02	TCSVT	Unified Information Fusion Network for Multi-Modal RGB-D and RGB-T Salient Object Detection	Paper/Code
03	TCSVT	CGFNet: Cross-Guided Fusion Network for RGB-T Salient Object Detection	Paper/Code
04	TCSVT	Efficient Context-Guided Stacked Refinement Network for RGB-T Salient Object Detection	Paper/Code
05	SPL	TSFNet: Two-Stage Fusion Network for RGB-T Salient Object Detection	Paper/Code
06	TETCI	APNet: Adversarial Learning Assistance and Perceived Importance Fusion Network for All-Day RGB-T Salient Object Detection	Paper/Code
07	TIP	Multi-Interactive Dual-Decoder for RGB-Thermal Salient Object Detection	Paper/Code
08	TCSVT	SwinNet: Swin Transformer drives edge-aware RGB-D and RGB-T salient object detection	Paper/Code
09	TCSVT	Multi-graph Fusion and Learning for RGBT Image Saliency Detection	Paper/Code
10	CYBER	Salient Target Detection in RGB-T Image based on Multi-level Semantic Information	Paper/Code

2022

No.	Pub.	Title	Links
01	Applied Intelligence	RGB-T salient object detection via CNN feature and result saliency map fusion	Paper/Code
02	Neurocomputing	Multi-modal Interactive Attention and Dual Progressive Decoding Network for RGB-D/T Salient Object Detection	Paper/Code
03	TCSVT	CGMDRNet: Cross-Guided Modality Difference Reduction Network for RGB-T Salient Object Detection	Paper/Code
04	arxiv	Glass Segmentation with RGB-Thermal Image Pairs	Paper/Code
05	TIP	Weakly Alignment-free RGBT Salient Object Detection with Deep Correlation Network	Paper/Code
06	TIM	Real-time One-stream Semantic-guided Refinement Network for RGB-Thermal Salient Object Detection	Paper/Code
07	TCSVT	Cross-Collaborative Fusion-Encoder Network for Robust RGB-Thermal Salient Object Detection	Paper/Code
08	EAAI	Unidirectional RGB-T salient object detection with intertwined driving of encoding and fusion	Paper/Code
09	MVA	EAF-Net: an enhancement and aggregation–feedback network for RGB-T salient object detection	Paper/Code
11	arxiv	Mirror Complementary Transformer Network for RGB-thermal Salient Object Detection	Paper/Code
12	CVIU	Enabling modality interactions for RGB-T salient object detection	Paper/Code
13	Applied Intelligence	Modal complementary fusion network for RGB-T salient object detection	Paper/Code
14	TMM	Does Thermal really always matter for RGB-T salient object detection	Paper/Code
15	Arxiv	Interactive Context-Aware Network for RGB-T Salient Object Detection	Paper/Code
16	DSP	MFENet: Multitype fusion and enhancement network for detecting salient objects in RGB-T images	Paper/Code
17	PR	Cross-modal co-feedback cellular automata for RGB-T saliency detection	Paper/Code
18	KBS	Asymmetric cross-modal activation network for RGB-T salient object detection	Paper/Code

2023

No.	Pub.	Title	Links
01	TCSVT	Cross-Modality Double Bidirectional Interaction and Fusion Network for RGB-T Salient Object Detection	Paper/Code
02	TIP	LSNet: Lightweight Spatial Boosting Network for Detecting Salient Objects in RGB-Thermal Images	Paper/Code
03	ICME	Scribble-Supervised RGB-T Salient Object Detection	Paper/Code
04	RAL	Explicit Attention-Enhanced Fusion for RGB-Thermal Perception Tasks	Paper/Code
05	EAAI	Thermal images-aware guided early fusion network for cross-illumination RGB-T salient object detection	Paper/Code
06	TMM	MFFNet: Multi-modal Feature Fusion Network for V-D-T Salient Object Detection	Paper/Code
07	Neurocomputing	Feature aggregation with transformer for RGB-T salient object detection	Paper/Code
08	Neurocomputing	MENet: Lightweight multimodality enhancement network for detecting salient objects in RGB-thermal images	Paper/Code
09	KBS	Three-stream interaction decoder network for RGB-thermal salient object detection	Paper/Code
11	TIP	Position-Aware Relation Learning for RGB-Thermal Salient Object Detection	Paper/Code
11	TCSVT	Multiple Graph Affinity Interactive Network and a Variable Illumination Dataset for RGBT Image Salient Object Detection	Paper/Code
12	TIP	CAVER: Cross-Modal View-Mixed Transformer for Bi-Modal Salient Object Detection	Paper/Code
13	PR	Cross-modal co-feedback cellular automata for RGB-T saliency detection	Paper/Code
14	TIP	WaveNet: Wavelet Network With Knowledge Distillation for RGB-T Salient Object Detection	Paper/Code
15	ICIP	Feature Enhancement and Fusion for RGB-T Salient Object Detection	Paper/Code
16	arxiv	All in One: RGB, RGB-D, and RGB-T Salient Object Detection	Paper/Code
17	ACM MM	Saliency Prototype for RGB-D and RGB-T Salient Object Detection	Paper/Code
18	PR	Frequency-aware feature aggregation network with dual-task consistency for RGB-T salient object detection	Paper/Code
19	arxiv	Unified-modal Salient Object Detection via Adaptive Prompt Learning	Paper/Code

2024

No.	Pub.	Title	Links
1	NN	Salient object detection in low-light RGB-T scene via spatial-frequency cues mining	Paper/Code
2	NN	MSEDNet: Multi-scale fusion and edge-supervised network for RGB-T salient object detection	Paper/Code
3	TIP	Quality-Aware Selective Fusion Network for V-D-T Salient Object Detection	Paper/Code
4	TCSVT	Learning Adaptive Fusion Bank for Multi-modal Salient Object Detection	Paper/Code
5	NN	UTDNet: A unified triplet decoder network for multimodal salient object detection	Paper/Code
6	PR	TMNet: Triple-modal interaction encoder and multi-scale fusion decoder network for V-D-T salient object detection	Paper/Code

---

RGB-T Semantic segmentation

2017

No.	Pub.	Title	Links
01	IROS	MFNet: Towards Real-Time Semantic Segmentation for Autonomous Vehicles with Multi-Spectral Scenes	Paper/Code

2019

No.	Pub.	Title	Links
01	RAL	RTFNet: RGB-Thermal Fusion Network for Semantic Segmentation of Urban Scenes	Paper/Code

2020

No.	Pub.	Title	Links
01	ICRA	PST900: RGB-Thermal Calibration, Dataset and Segmentation Network	Paper/Code
02	TASE	FuseSeg: Semantic Segmentation of Urban Scenes Based on RGB and Thermal Data Fusion	Paper/Code
02	CINE	Using thermal intensities to build conditional random fields for object segmentation at night	Paper/Code

2021

No.	Pub.	Title	Links
🚩01	TIP	GMNet: Graded-Feature Multilabel-Learning Network for RGB-Thermal Urban Scene Semantic Segmentation	Paper/Code
🚩02	CVPR	ABMDRNet: Adaptive-weighted Bi-directional Modality Difference Reduction Network for RGB-T Semantic Segmentation	Paper/Code
03	IROS	FEANet: Feature-Enhanced Attention Network for RGB-Thermal Real-time Semantic Segmentation	Paper/Code
04	Measurement	Robust semantic segmentation based on RGB-thermal in variable lighting scenes	Paper/Code
05	TMM	MFFENet: Multiscale Feature Fusion and Enhancement Network for RGBThermal Urban Road Scene Parsing	Paper/Code
06	Applied Intelligence	MMNet: Multi-modal multi-stage network for RGB-T image semantic segmentation	Paper/Code
07	Neurocomputing	CCAFFMNet: Dual-spectral semantic segmentation network with channel-coordinate attention feature fusion module	Paper/Code
08	IROS	HeatNet: Bridging the Day-Night Domain Gap in Semantic Segmentation with Thermal Images	Paper/Code

2022

No.	Pub.	Title	Links
🚩01	AAAI	Edge-aware guidance fusion network for RGB–thermal scene parsing	Paper/Code
02	TIV	MTANet: Multitask-Aware Network with Hierarchical Multimodal Fusion for RGB-T Urban Scene Understanding	Paper/Code
🚩 03	TITS	CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation with Transformers	Paper/Code
03	ACPR	ARTSeg: Employing Attention for Thermal Images Semantic Segmentation	Paper/Code
04	Neurocomputing	GCNet: Grid-Like Context-Aware Network for RGB-Thermal Semantic Segmentation	Paper/Code
05	TCSVT	RGB-T Semantic Segmentation with Location, Activation, and Sharpening	Paper/Code
06	SPL	GEBNet: Graph-Enhancement Branch Network for RGB-T Scene Parsing	Paper/Code
07	TCSVT	A Feature Divide-and-Conquer Network for RGB-T Semantic Segmentation	Paper/Code

2023

No.	Pub.	Title	Links
01	TITS	Embedded Control Gate Fusion and Attention Residual Learning for RGB–Thermal Urban Scene Parsing	Paper/Code
02	RAL	Explicit Attention-Enhanced Fusion for RGB-Thermal Perception Tasks	Paper/Code
03	PR	Complementarity-aware cross-modal feature fusion network for RGB-T semantic segmentation	Paper/Code
04	TCSVT	MMSMCNet: Modal Memory Sharing and Morphological Complementary Networks for RGB-T Urban Scene Semantic Segmentation	Paper/Code
05	TIV	CACFNet: Cross-Modal Attention Cascaded Fusion Network for RGB-T Urban Scene Parsing	Paper/Code
06	TSMC	DBCNet: Dynamic Bilateral Cross-Fusion Network for RGB-T Urban Scene Understanding in Intelligent Vehicles	Paper/Code
07	TCSVT	SGFNet: Semantic-Guided Fusion Network for RGB-Thermal Semantic Segmentation	Paper/Code
08	arxiv	Efficient Multimodal Semantic Segmentation via Dual-Prompt Learning	Paper/Code
09	TITS	A RGB-Thermal Image Segmentation Method Based on Parameter Sharing and Attention Fusion for Safe Autonomous Driving	Paper/Code
10	GRSL	UTFNet: Uncertainty-Guided Trustworthy Fusion Network for RGB-Thermal Semantic Segmentation	Paper/Code
11	TIM	SFAF-MA: Spatial Feature Aggregation and Fusion With Modality Adaptation for RGB-Thermal Semantic Segmentation	Paper/Code
12	TIV	On Exploring Shape and Semantic Enhancements for RGB-X Semantic Segmentation	Paper/Code

2024

No.	Pub.	Title	Links
01	TMM	Context-Aware Interaction Network for RGB-T Semantic Segmentation	Paper/Code
02	PR	Region-adaptive and context-complementary cross modulation for RGB-T semantic segmentation	Paper/Code
03	Neurocomputing	Residual spatial fusion network for RGB-thermal semantic segmentation	Paper/Code
04	Neurocomputing	DHFNet: Decoupled Hierarchical Fusion Network for RGB-T dense prediction tasks	Paper/Code
05	TIV	Multi-branch Differential Bidirectional Fusion Network for RGB-T Semantic Segmentation	Paper/Code
06	AAAI	Prompting Multi-Modal Image Segmentation with Semantic Grouping	Paper/Code

---

RGB-T Crowd Counting

2021

No.	Pub.	Title	Links
🚩01	CVPR	Cross-Modal Collaborative Representation Learning and a Large-Scale RGBT Benchmark for Crowd Counting	Paper/Code
02	IC-NIDC	I-MMCCN: Improved MMCCN for RGB-T Crowd Counting of Drone Images	Paper/Code

2022

No.	Pub.	Title	Links
🚩01	TITS	DEFNet: Dual-Branch Enhanced Feature Fusion Network for RGB-T Crowd Counting	Paper/Code
02	ISCAS	TAFNet: A Three-Stream Adaptive Fusion Network for RGB-T Crowd Counting	Paper/Code
03	ACCV	Spatio-channel Attention Blocks for Cross-modal Crowd Counting	Paper/Code

2023

No.	Pub.	Title	Links
01	RAL	Explicit Attention-Enhanced Fusion for RGB-Thermal Perception Tasks	Paper/Code

---

Dataset

RGBT SOD Saliency Dataset(VT821,VT1000,VT5000)
You can found in VT800,VT1000,VT5000.
RGBT Semantic segmentation Dataset(MFNet,PST900,SemanticRT)
You can found in MFNet and PST900 and SemanticRT.
RGBT Crowd Counting Dataset(RGBT-CC)
You can found in RBGT-CC

---

Evaluation

RGBT SOD Saliency Evaluation
Python version: here(CPU) and here(GPU).
Matlab version: here(include weighted F) and here.
RGBT Semantic segmentation Evaluation
Recommend the evaluation toolbox of RTFNet or GMNet.
RGBT Crowd Counting
Recommend the evaluation toolbox of DEFNet or BL+IADM

---

Other Summary

RGBD SOD Summary1： https://github.com/jiwei0921/SOD-CNNs-based-code-summary-.
RGBD SOD Summary2： https://github.com/taozh2017/RGBD-SODsurvey.
RGBT SOD Summary： https://github.com/lz118/RGBT-Salient-Object-Detection.

---

Acknowledgement

The collection of this summary is thanks to Zhun Li , jinfu Liu and Yi Pan.
The summary template comes from ji wei.

---

🏳️‍🌈 Thanks to the above authors for their excellent work！