Generative Adversarial Networks (GANs) hold the state-of-the-art when it comes to image generation.

However, while the rest of computer vision is slowly taken over by transformers or other attention-based architectures, all working GANs to date contain some form of convolutional layers. This paper changes that and builds TransGAN, the first GAN where both the generator and the discriminator are transformers. The discriminator is taken over from ViT (an image is worth 16×16 words), and the generator uses pixelshuffle to successfully up-sample the generated resolution. Three tricks make training work: Data augmentations using DiffAug, an auxiliary superresolution task, and a localized initialization of self-attention.

Their largest model reaches competitive performance with the best convolutional GANs on CIFAR10, STL-10, and CelebA.

Yannic Kilcher explains

Google’s UK-based lab and research company DeepMind has added Jraph to the growing number of open-sourced libraries around JAX, while surveying the machine learning framework’s development and ecosystem.

JAX is a Python library that Google researchers developed and introduced in 2018 for high-performance numerical computing.

JAX combines NumPy, automatic differentiation, and GPU/TPU support. In a new blog post, DeepMind researchers look at how JAX and its emergent ecosystem of open source libraries have served and accelerated an increasing number of machine learning projects.