Implementation of the SatVision-TOA model for All-Sky Remote Sensing Imagery.
The package includes the pretrained SatVision-TOA model along with fine-tuned versions for both 1) Image Reconstruction and 2) 3D Cloud Retrieval. While instructions for training the model are provided for reference, this deployment is primarily intended for downstream SatVision-TOA task usage and customization.
Concept and design details are provided in the paper on arXiv.
Contents:
We introduce SatVision-TOA, a novel foundation model pre-trained on 14-band MODIS L1B Top-Of-Atmosphere (TOA) radiance imageries, addressing the need for models that handle moderate- and coarse-resolution all- sky remote sensing data. The SatVision-TOA model is pre-trained using a Masked-Image-Modeling (MIM) framework and the SwinV2 architecture, and learns detailed contextual representations through self-supervised learning without the need for labels. It is a 3 billion parameter model that is trained on 100 million images. To validate the model we focused on two key applications:
- Image reconstruction performance, which evaluates the masked-image-modeling task on unseen data.
- 3D cloud retrieval, a downstream task targeting the prediction of vertical cloud structure.
This is the largest foundation model trained solely on satellite remote sensing imagery. Initial results indicate that SatVision-TOA achieves superior performance over baseline methods when applied to downstream tasks such as 3D cloud retrieval. Our work advances pre-trained vision modeling for multispectral remote sensing by learning from a variety of atmospheric and aerosol conditions to improve cloud and land surface monitoring.
SatVision-TOA is a foundation model, which means that it was first generally trained on a lot of data, and then can be adapted to specialized cloud and land surface monitoring tasks with relatively little data.
Cite us as follows:
@misc{satvision-base,
author = {Carroll, Mark and Li, Jian and Spradlin, Caleb and Caraballo-Vega, Jordan},
doi = {10.57967/hf/1017},
month = aug,
title = {{satvision-base}},
url = {https://huggingface.co/nasa-cisto-data-science-group/satvision-base},
repository-code = {https://github.com/nasa-nccs-hpda/pytorch-caney}
year = {2023}
}