This week in multimodal ai art (16/04 - 22/04)

Follow multimodalart on Twitter, come hang on our Discord and consider supporting on Patreon

A big week: OpenAI released the biggest CLIP on the original paper. Also good news in the StyleGAN world as well as a new Colab that stiches together two other Colabs: Latent Diffusion + Disco Diffusion v5.2

New CLIP and CLIP like models:

CLIP ViT-L/[email protected] (GitHub)

by OpenAI

OpenAI released the ViT-L/[email protected] CLIP model. This is the biggest model mentioned in the original CLIP paper released 1 year and 4 months ago. It is super VRAM heavy (doesn't run on free and in most of the Colab Pro GPUs). The community is still tuning the models to work well with the model. Despite being the biggest on the paper, it is still smaller than the (unreleased CLIP ViT-H/16 used to train Dalle 2)

Text-to-Image synthesizers:

- StyleGAN-Human

by SenseTime Research, S-Lab, Nanyang Technological University, Shanghai AI Laboratory
A new different set of StyleGANs trained on full body human generation. The speedrun to who releases a Colab with this hooked with CLIP has already started

- StyleGAN XL + CLIP [Katherine's modification] (Colab)

by Katherine Crowson
Katherine Crowson (@rivershavewings) has modified StyleGAN+CLIP notebook we reported here three weeks ago with a different approach to optimization. Her Colab also makes videos automatically.

- Centipede Diffusion (colab)

by Zalring
Centipede Diffusion is a combination of our LAION 400M Google Colab with Disco Diffusion 5.2. Essentially the LAION model is used as an initial image and Disco Diffusion is a glorified upscaler, finishing and polishing the execution. It inaugurates a trend of automating this process already done in the community of mixing models as initial images from one another. We plan on doing it on MindsEye too.

- MindsEye now can pilot Latent Diffusion

MindsEye, our GUI for text-to-image now has a new model for it to pilot: Latent Diffusion. Now from the same UI and without touching a line of code (not even to tweak parameters) you can control VQGAN+CLIP, CLIP Guided Diffusion and now Latent Diffusion. Check MindsEye out!

MindsEye GIF

Learning AI Art:

AIAIArt course (GitHub, Discord)

AIArt is a free and open source AI art course by John Whitaker. There are synchronous classes for the next few Saturdays 4 PM UTC on Twitch. All previous classes stay recorded and available on Google Colabs on the GitHub link, this Saturday there's no classes so it's a good time to start so you can catch up with the content!