RGI: Sturdy GAN-inversion for Masks-free Picture Inpainting and Unsupervised Pixel-wise Anomaly Detection

Generative adversarial networks (GANs), educated on a large-scale picture dataset, could be a good approximator of the pure picture manifold. GAN-inversion, utilizing a pre-trained generator as a deep generative prior, is a promising device for picture restoration below corruptions. Nevertheless, the efficiency of GAN-inversion could be restricted by an absence of robustness to unknown gross corruptions, i.e., the restored picture may simply deviate from the bottom fact. On this paper, we suggest a Sturdy GAN-inversion (RGI) technique with a provable robustness assure to attain picture restoration below unknown textit{gross} corruptions, the place a small fraction of pixels are fully corrupted. Underneath gentle assumptions, we present that the restored picture and the recognized corrupted area masks converge asymptotically to the bottom fact. Furthermore, we lengthen RGI to Relaxed-RGI (R-RGI) for generator fine-tuning to mitigate the hole between the GAN realized manifold and the true picture manifold whereas avoiding trivial overfitting to the corrupted enter picture, which additional improves the picture restoration and corrupted area masks identification efficiency. The proposed RGI/R-RGI technique unifies two necessary purposes with state-of-the-art (SOTA) efficiency: (i) mask-free semantic inpainting, the place the corruptions are unknown lacking areas, the restored background can be utilized to revive the lacking content material; (ii) unsupervised pixel-wise anomaly detection, the place the corruptions are unknown anomalous areas, the retrieved masks can be utilized because the anomalous area’s segmentation masks.