This video shows the creation of the loss mask and the masked image to inject. The local mean color is used for the image mask to create better performance when injecting the image on lower scales. The created loss mask has the same shape as the image mask.
masked.mp4
The SinGAN can be trained on the original image with the masked loss and partial convolution to ignore the unwanted structures. Afterwards the downsampled version of the masked image can be injected into a generator to obtain a sample. This process can be seen as an editing or paint-to-image task.
output.mp4
# install
pip install -r requirements.txt
# masking
python3 masking_app.py
# train models for the five images on the poster
python3 train_script.py
# sample images by injecting the masked image in different scales
python3 paint2image_script.py
This project is highly inspired by - and uses code from https://github.com/tamarott/SinGAN, https://github.com/antoyang/SinGAN and https://github.com/NVIDIA/partialconv.