Potential collaboration/contribution: Multi-source input extension for Soil Mapping

[Morteza-Khazaei]

Hi TESSERA Team,

We are exploring TESSERA's capabilities for creating embedding spaces for Digital Soil Mapping. To adapt the model for the STEP-AWBH framework, we are planning to modify the architecture to ingest additional environmental factors (Topography, Parent Material, Atmosphere) alongside the current Optical and SAR inputs.

We see great potential in TESSERA's current "ready-to-use" embedding approach. Does the team have any active plans to expand the supported modalities? If not, we would be happy to discuss how a multi-source variant might fit into the TESSERA ecosystem or if you have architectural guidelines for adding new encoder branches without disrupting the pre-trained weights.

[avsm]

Thanks for bringing this up, @Morteza-Khazaei! Just before we drop into the multi-source variant discussion, have you tried a quick experiment with adding the extra environment factors as a downstream tuning task over the existing embeddings? We're seeing some success doing that without having to go through a training step, but it depends on the nature and resolution of the extra factors.

[sk818]

Our strong preference is to keep the embeddings pixel based, high-resolution (10m), and using only S1/S2, wit additional modes and patches brought in for downstream tasks. This prevents blurring and also makes the embeddings useful for a wider variety of downstream tasks.

[Morteza-Khazaei]

Thank you for your response. If I understand correctly, you are suggesting using the embedding dataset that you are currently sharing as input for another learning task by adding other modalities (e.g., precipitation and temperature) to fine-tune the existing 128-band embedding from TESSERA. I would be interested to hear about your reasons for doing so and your experiences from the practical experiments you have conducted in the past.

Besides, our group comprises soil scientists, plant scientists and remote sensing experts, each of whom sees the embedding potential from a different perspective. As Prof. Keshav emphasised, the spatial resolution of the final embedded space is always the top priority in many applications.

The following are the reasons why we believe a new variant of TESSERA is needed for such downstream tasks.

The STEP-AWBH concept is a space-time modeling framework used in digital soil mapping (DSM) to predict soil properties by incorporating both relatively static and dynamic environmental and human-induced factors.

The STEP-AWBH model categorizes soil-forming factors into components that are largely static (changing over millennia) and those that are significantly dynamic (changing over short or long periods). The first group, STEP, are considered relatively stable, while the second group, AWBH, are distinctly dynamic.

Soil properties: Core properties like soil texture and mineralogy are relatively static or change over very long time scales.

Topographic properties: Features such as elevation, slope, and landscape position are essentially permanent fixtures of the environment.

Ecological properties: Broad ecoregions or physiographic regions are stable over long periods.

Parent material and geologic properties: The underlying geology or lithology is a fundamental, unchanging base for soil development.

Atmospheric properties: Variables like precipitation, temperature, and solar radiation change daily and seasonally and are subject to long-term climate change trends.

Water properties: This includes dynamic aspects like soil moisture, infiltration rates, and surface runoff, which fluctuate constantly with weather and land use.

Biotic properties: This covers living organisms, vegetation cover (land cover), and land use. These change rapidly due to human activity, natural disturbances (fires), and the life cycles of organisms.

Human-induced forcings: The most dynamic set of factors, including land management practices, contamination events, and socio-economic factors that impose change upon the soil system.

[sk818]

Thanks for your response. You ask about our experience in using only S1 and S2 for creating embedding and using other sensing modalities for downstream tasks.May I please refer you to our paper for this. For examples, in our canopy height model, we use LiDAR-based ground truth canopy height to train the embedding-to-canopy height model. Using other sensors along with the embeddings as inputs to an MLP or UNET is also an avenue that we advocate but haven't explored much in our work. Your argument about adding more inputs to the embeddings is perhaps very relevant in your domain, which I have no expertise in. I would suggest using an MAE to combine the Tessera embeddings with ground truth data to create new embeddings that you might find more relevant for your specific tasks.