CATLAB

Recent Publications

1. ICCV

   LatentCLR: A Contrastive Learning Approach for Unsupervised Discovery of Interpretable Directions

   Oguz Kaan Yuksel*, Enis Simsar*, Ezgi Gulperi Er, and Pinar Yanardag

   International Conference on Computer Vision (ICCV) , 2021

   Abstract PDF Code Project Page

   Recent research has shown great potential for finding interpretable directions in the latent spaces of pre-trained Generative Adversarial Networks (GANs). These directions provide controllable generation and support a wide range of semantic editing operations such as zoom or rotation. The discovery of such directions is often performed in a supervised or semi-supervised fashion and requires manual annotations, limiting their applications in practice. In comparison, unsupervised discovery enables finding subtle directions a priori hard to recognize. In this work, we propose a contrastive-learning-based approach for discovering semantic directions in the latent space of pretrained GANs in a self-supervised manner. Our approach finds semantically meaningful dimensions compatible with state-of-the-art methods.

2. WACV

   StyleMC: Multi-Channel Based Fast Text-Guided Image Generation and Manipulation

   Umut Kocasari, Alara Dirik, Mert Tiftikci, and Pinar Yanardag

   Winter Applications of Computer Vision (WACV) , 2021

   Abstract COMING SOON

   Pre-trained GANs have shown great potential for interpretable directions in the latent space. The discovery of such directions is often done in a supervised or self-supervised manner and requires manual annotations which limits their application in practice. On the other hand, unsupervised approaches provide a way to discover interpretable directions without any supervision, but no fine-grained attribute can be discovered. Recent work such as StyleCLIP aims to overcome this limitation by leveraging the power of CLIP, a joint representational model for text and images, for text-driven image manipulation. While promising, these methods take several hours of pre-processing or training time, and require multiple text prompts. In this work, we propose a fast and efficient method for text-guided image generation and manipulation by leveraging the power of StyleGAN2 and CLIP. Our method uses a CLIP-based loss and an identity loss to manipulate images via user-supplied text prompts without changing any of the irrelevant attributes. Unlike previous work, our method requires only 12 seconds of optimization per text prompt and can be used with any pre-trained StyleGAN2 model. We demonstrate the effectiveness of our method with extensive results and comparisons to state-of-the-art approaches.

3. AIM at ICCV

   Graph2Pix: A Graph-Based Image to Image Translation Framework

   Dilara Gokay*, Enis Simsar*, Efehan Atici, Atif Emre Yilmaz, Alper Ahmetoglu, and Pinar Yanardag

   Advances of Image Manipulation Workshop at ICCV (AIM at ICCV) , 2021

   Abstract PDF Code Project Page

   In this paper, we propose a graph-based image-to-image translation framework for generating images. We use rich data collected from the popular creativity platform Artbreeder, where users interpolate multiple GAN-generated images to create artworks. This unique approach of creating new images leads to a tree-like structure where one can track historical data about the creation of a particular image. Inspired by this structure, we propose a novel graph-to-image translation model called Graph2Pix, which takes a graph and corresponding images as input and generates a single image as output. Our experiments show that Graph2Pix is able to outperform several image-to-image translation frameworks on benchmark metrics, including LPIPS (with a 25% improvement) and human perception studies (n=60), where users preferred the images generated by our method 81.5% of the time.