# SpA-GAN_for_cloud_removal **Repository Path**: workhard123/SpA-GAN_for_cloud_removal ## Basic Information - **Project Name**: SpA-GAN_for_cloud_removal - **Description**: Cloud Removal for High-resolution Remote Sensing Imagery based on Generative Adversarial Networks. - **Primary Language**: Unknown - **License**: MIT - **Default Branch**: master - **Homepage**: None - **GVP Project**: No ## Statistics - **Stars**: 1 - **Forks**: 0 - **Created**: 2021-11-10 - **Last Updated**: 2024-10-30 ## Categories & Tags **Categories**: Uncategorized **Tags**: None ## README

SpA GAN for Cloud Removal

### new - 2020.9.29 The draft is released now at https://arxiv.org/abs/2009.13015. ## 1. INTRODUCTION This is the source code of [***Cloud Removal for Remote Sensing Imagery via Spatial Attention Generative Adversarial Network***](https://arxiv.org/abs/2009.13015). In this work, I proposes a novel cloud removal model called ***spatial attention generative adversarial networks*** or ***SpA GAN***, which use [spatial attention networks (SPANet)](https://github.com/stevewongv/SPANet) as generator. The architecture of *SpA GAN* is shown as fellow: - **Generator** *SpA GAN* uses *spatial attention networks* an generator. See `./models/gen/SPANet.py` for more details.

- **Discriminator** Discriminator is a fully CNN that **C** is convolution layer, **B** is batch normalization and **R** is Leaky ReLU. See `./models/dis/dis.py` for more details.

- **Loss** The total loss of *SpA GAN* is formulated as fellow:

the first part is the loss of GAN

the second part is standard $L_1$ loss where $\lambda_c$ is a hyper parameter to control the weight of each channel to the loss.

the third part is attention loss where $A$ is the attention map and $M$ is the mask of cloud that computed from $M=|I_{in}-I_{gt}|_1$.

## 2. DATASET ### 2.1. RICE_DATASET Click [official address](https://github.com/BUPTLdy/RICE_DATASET) or [Google Drive](https://drive.google.com/file/d/1Tsm9qEugNyDKLe4bu06e-2IqEhENu64D/view?usp=sharing) to download the open source RICE dataset. Build the file structure as the folder `data` shown. Here `cloudy_image` is the folder where the cloudy image is stored and the folder `ground_truth` stores the corresponding cloudless images. ``` ./ +-- data +-- RICE_DATASET +-- RICE1 | +-- cloudy_image | | +-- 0.png | | +-- ... | +-- ground_truth | +-- 0.png | +-- ... +-- RICE2 +-- cloudy_image | +-- 0.png | +-- ... +-- ground_truth +-- 0.png +-- ... ``` ### 2.2. Perlin Dataset Construct the dataset by adding Perlin noise as cloud into the image. ## 3. TRAIN Modify the `config.yml` to set your parameters and run: ```bash python train.py ``` ## 4. TEST ```bash python predict.py --config --test_dir --out_dir --pretrained --cuda ``` There're my pre-trained models on [RICE1](./pretrained_models/RICE1/)(`./pretrained_models/RICE1/gen_model_epoch_200.pth`) and [RICE2]((./pretrained_models/RICE1/))(`./pretrained_models/RICE2/gen_model_epoch_200.pth`). Some results are shown as bellow and the images from left to right are: cloudy image, attention map, SpA GAN's output, ground truth.

## 5. EXPERIMENTS In this section, I compares *SpA GAN* with *conditional GAN* and *cycle GAN* using peak signal to noise ratio (***PSNR***) and structural similarity index (***SSIM***) as metrics on datasets RICE1 and RICE2. ### 5.1 RICE1 **qualitative analysis** The result are shown as bellow and the images from left to right are: cloudy image, conditional GAN's output, cycle GAN's output , SpA GAN's output, ground truth.

**quantitative analysis** | | PSNR | SSIM | | :-----------: | :----: | :---: | | **cGAN** | 26.547 | 0.903 | | **cycle GAN** | 25.880 | 0.893 | | **SpA GAN** | 30.232 | 0.954 | ### 5.1 RICE2 **qualitative analysis** The result are shown as bellow and the images from left to right are: cloudy image, conditional GAN's output, cycle GAN's output , SpA GAN's output, ground truth.

**quantitative analysis** | | PSNR | SSIM | | :-----------: | :----: | :---: | | **cGAN** | 25.384 | 0.811 | | **cycle GAN** | 23.910 | 0.793 | | **SpA GAN** | 28.368 | 0.906 | ## 6. CONTACT Contact me if you have any questions about the code and its execution. E-mail: penn000@foxmail.com If you think this work is helpful for your research, give me a star :-D ### Citations ``` @article{Pan2020, title = {Cloud Removal for Remote Sensing Imagery via Spatial Attention Generative Adversarial Network}, author = {Heng Pan}, journal = {arXiv preprint arXiv:2009.13015}, year = {2020} } ```