AnytimeFormer: Fusing irregular and asynchronous SAR-optical time series to reconstruct reflectance at any given time

Authors: Kai Tang, Xuehong Chen, Tianyu Liu, Anqi Li, Yao Tang, Peng Yang, Jin Chen*

---

📰 News

• [2026-01-13] Based on user feedback, we identified a minor bug in the process of generating the HDF dataset for Anytime mode, which could lead to the best model checkpoint not being correctly saved during training. We have fixed this by correcting the indicating_mask used in the computation of the validation metric. Detailed revision is provided here. It is recommended to use the latest 3_generate_hdf_for_Germany_anytime.ipynb for Anytime HDF generation.
• [2025-11-07] AnytimeFormer has been accepted by Remote Sensing of Environment (RSE) ! [Paper]

📜Outline

• Hightlights
• Installation
• Dataset Preparation
• Training
• Testing
• Future
• Citation

---

🚀Hightlights

1. Proposes AnytimeFormer for gap-free surface reflectance time series reconstruction.
2. AnytimeFormer adapts to irregular optical and asynchronous SAR time series.
3. AnytimeFormer supports reflectance reconstruction at any given time.
4. An average 38% decrease in RMSE compared to the five benchmark methods.
5. Reduces training time by ten times compared to STORI, the second-best model.

---

🏗 Architecture

📦Installation

Step 1. Clone/download the repository

git clone https://github.com/tangkai-RS/AnytimeFormer.git
cd AnytimeFormer

Step 2. Create environment and install dependencies

conda create -n anytime python=3.8
conda activate anytime
pip install torch==1.12.1+cu116 torchvision==0.13.1+cu116 torchaudio==0.12.1 --extra-index-url https://download.pytorch.org/whl/cu116
pip install -r requirements.txt
conda install -c conda-forge gdal=3.4.3

📚Dataset Preparation

Dataset

• Germany-site can be found and downloaded here: Huggingface
• Other sites' shpfiles can be found and downloaded here: Google Drive

After downloading the Germany-site dataset and placing it in the folder ./datasets/, please follow the directory structure below:

dataset  
|-- dataset_for_model  
|   |-- Germany             
|   |   |-- tif               
|   |   |   |-- 40%            
|   |   |   |-- 60%           
|   |   |   |-- 80%          
|   |   |-- hdf

Note: Due to the large volume of timeseries data, only preprocessed data for the Germany site is provided here for quick testing. Dataset for other sites can be downloaded and preprocessed using shapefile and Google Earth Engine (GEE). For GEE and preprocessing scripts, please refer to folder Data Preparation Guide.

🔥Training

For Linux

Mode 1: Sequence to Sequence

Perform AnytimeFormer training with normal mode (sequence to sequence):

bash scripts/train-Germany.sh

or

WORK_DIR="work_dir/Germany/base"
ORIGINAL_DATASET_PATH="dataset/dataset_for_model/Germany/hdf/random_missing.hdf"
REF_TIF_PATH="dataset/dataset_for_model/Germany/tif/40%/S2_L2A_20190208.tif"
CONFIG_FILE="configs/Germany/base/AnytimeFormer-Germany-40%-r8-128.yaml"

python main.py \
    --config_path $CONFIG_FILE \
    --work_dir $WORK_DIR \
    --original_dataset_path $ORIGINAL_DATASET_PATH \
    --ref_tif_path $REF_TIF_PATH \
    --mode "train_test" \
    --debug_mode False \
    --cal_performance_metric True

Mode 2: ✨Anytime

Perform AnytimeFormer training with anytime mode (support any given time outputs):

bash scripts/train-Germany-anytime.sh

or

WORK_DIR="work_dir/Germany/anytime"
ORIGINAL_DATASET_PATH="dataset/dataset_for_model/Germany/hdf/anytime.hdf"
REF_TIF_PATH="dataset/dataset_for_model/Germany/tif/40%/S2_L2A_20190208.tif" 
CONFIG_FILE="configs/Germany/anytime/AnytimeFormer-Germany-40%-r8-128-wTV.yaml"

python main.py \
    --config_path $CONFIG_FILE \
    --work_dir $WORK_DIR \
    --original_dataset_path $ORIGINAL_DATASET_PATH \
    --ref_tif_path $REF_TIF_PATH \
    --mode "train_test_anytime" \
    --debug_mode False

For Windows

On Windows systems, you can directly modify the arguments in main.py and run it without using shell scripts.
For more details about parameter settings, please refer to:

• Lines 414–507 in main.py, where detailed comments for each argument are provided
• configs — configuration file

Note that any arguments specified directly in main.py will override the corresponding parameters in the configuration files (configs). If you have any questions about parameter setting, please submit issues.

Results

After training, you can find the results in the following locations:

work_dir
|-- Germany
|   |-- anytime
|   |-- base
|   |   |-- 2025-10-30_T20-22-31_AnytimeFormer-Germany-40%-r8-128
|   |   |   |-- inference
|   |   |   |-- models
|   |   |   |-- log.log

• Model Weights
  Located in the models/ folder: best_model.ckp
• Output Images
  In the inference/ folder, you can find two types of images:
• Imputed Images
  • Only the missing regions in the original time-series images are replaced with the model's predictions.
• Reconstructed Images
  • The entire time-series images are generated by the model's predictions.
• Performance Metric
  • If --cal_performance_metric is set to True, the accuracy metrics can also be found in log.log.

❄️Testing

For Linux

Mode 1: Sequence to Sequence

Perform AnytimeFormer training with normal mode (sequence to sequence):

bash scripts/inference-Germany.sh

or

WORK_DIR="work_dir/Germany/base"
ORIGINAL_DATASET_PATH="dataset/dataset_for_model/Germany/hdf/random_missing.hdf"
REF_TIF_PATH="dataset/dataset_for_model/Germany/tif/40%/S2_L2A_20190208.tif"
CONFIG_FILE="configs/Germany/base/AnytimeFormer-Germany-40%-r8-128.yaml"
SAVED_MODEL_PATH="work_dir/Germany/best_model_40%.ckpt"

python main.py \
    --config_path $CONFIG_FILE \
    --work_dir $WORK_DIR \
    --original_dataset_path $ORIGINAL_DATASET_PATH \
    --ref_tif_path $REF_TIF_PATH \
    --mode "test" \
    --saved_model_path $SAVED_MODEL_PATH

Mode 2: ✨Anytime

Perform AnytimeFormer training with anytime mode (support any given time outputs):

bash scripts/inference-Germany-anytime.sh

or

WORK_DIR="work_dir/Germany/anytime"
ORIGINAL_DATASET_PATH="dataset/dataset_for_model/Germany/hdf/anytime.hdf"
REF_TIF_PATH="dataset/dataset_for_model/Germany/tif/40%/S2_L2A_20190208.tif" 
CONFIG_FILE="configs/Germany/anytime/AnytimeFormer-Germany-40%-r8-128-wTV.yaml"
SAVED_MODEL_PATH="work_dir/Germany/best_model_anytime.ckpt"

python main.py \
    --config_path $CONFIG_FILE \
    --work_dir $WORK_DIR \
    --original_dataset_path $ORIGINAL_DATASET_PATH \
    --ref_tif_path $REF_TIF_PATH \
    --mode "test_anytime" \
    --saved_model_path $SAVED_MODEL_PATH \
    --debug_mode False

• Pay attention to anytime_ouput of the config.yaml

For Windows

On Windows systems, you can directly modify the arguments in main.py and run it without using shell scripts.

For more details about parameter settings, relative to training, you only need to make the following modifications:

• Set --mode to "test" (seq2seq model) or "test_anytime" (anytime mode)
• Set --saved_model_path to the path of the best-trained model
• Pay attention to anytime_ouput of the config.yaml

✨If you have any questions about parameter setting, please submit issues.

🔭 Future

We will gradually release pipelines for some benchmark methods (e.g., U-TILISE and RESTORE-DiT) to help more people follow and join the time-series reconstruction field.

📄 Citation

@article{tang2026anytime,
title = {AnytimeFormer: Fusing irregular and asynchronous SAR-optical time series to reconstruct reflectance at any given time},
author  = {Tang, Kai and Chen, Xuehong and Liu, Tianyu and Li, Anqi and Tang, Yao and Yang, Peng and Chen, Jin},
journal = {Remote Sensing of Environment},
volume = {333},
pages = {115120},
year = {2026},
issn = {0034-4257},
}

✨If you have any questions or collaboration suggestions, please contact us (tangkai@mail.bnu.edu.cn).