Self-supervised pretraining of transformers with curriculum learning in satellite imagery

Motivation

Vision Transformers are notorious for their high data requirements. Discovering an effective approach to enhance the efficiency of the pretraining phase with vast amounts of data can lead to significant time and resource savings. In the domain of remote sensing, where access to large datasets like ImageNet is limited, successful scalability can be achieved by employing self-supervision in combination with intelligent training techniques.

Task

Pre-Training in self-supervised fashion with Sentinel-2 images. Semantic Segmentation in the finetuning phase.

Details

Self-supervised pretraining of transformers with curriculum learning in satellite imagery segmentation

Crisis-FACTs Reworked

Motivation

Consistent and reproducible evaluation enables research advancements. While CrisisFACTs presents a unique challenge in the current retrieval landscape, its evaluation needs to be reconsidered from the ground up to support new research efforts. The objective of the task is to select relevant texts for crisis operators and then create concise summaries. Currently, the evaluation relies on humans, which is impractical for testing any new model.

Task

The thesis aims to develop a new evaluation system and replicate previous solutions to establish a new working benchmark.

References

CrisisFACTS: Building and Evaluating Crisis Timelines

Cross-Modal Retrieval for Land Use Land Cover

Motivation Retrieving relevant satellite imagery from vast archives is crucial for applications like disaster response and environmental monitoring. While existing text-to-image retrieval systems have shown promise, they are mostly limited to RGB data, failing to exploit the rich information in other sensor types like Synthetic Aperture Radar (SAR) and multispectral imagery. The CLOSP architecture successfully created a unified semantic space for both optical and SAR data, but its analysis is limited to single, static images. Many critical remote sensing applications, such as change detection and trend analysis, rely on observing areas over time. The current framework has not been extended to a temporal dimension, which would be a logical and valuable next step to enable dynamic queries like "urban growth over the last decade". Additionally, ready-made embeddings like Google Satellite Embeddings provide a ready-to-use aggregated resource for global-scale analysis.

Task