This week in multimodal ai art (30/Apr - 06/May)

Follow on Twitter, come hang on Discord, consider supporting on Patreon. Try MindsEye, our image generation GUI

Text-to-image synthesizers update

CLIP-GEN models released* DALLE-Mega mid-training checkpoint* Centipede Diffusion Inpainting Upgrade* DALL-E Flow released*

New CLIP and CLIP-like models

OpenCLIP ViT-B/16+ trained on LAION-400M*

Text-to-3D synthesizers updates

CLIP-Forge released


AIAIART course*

* code released

Text-to-Image synthesizers:

- CLIP GEN small model released (Colab, GitHub, Paper)

by HFAiLab
Last week we reported that HFAiLab released the code for CLIP-GEN. This week they released pre-trained models. I've assembled a Colab for it.

- Dall-E Mega mid-training checkpoint (Hugging Face Spaces)

by Boris Dayma
We reported two weeks ago that DALL-E Mega (the DALL-E replication) had started training, last week a 15% training model was out. And now a ~50% in-training model was released

- Centipede Diffusion Inpainting Upgrade (colab)

by Zalring
We reported Centipede Diffusion two weeks ago here, it generates images with Latent Diffusion and then upscales with Disco Diffusion. It got an update with both Inpainting for both the Latent Diffusion and the Disco Diffusion bits, check it out!

- DALL-E Flow released (Colab,GitHub)

Like Centipede Diffusion it is a mix of models: It generates images with DALL-E Mega, refines and creates variations with Latent Diffusion, ranks the best with CLIP and upscales the results with SwinR. The backend runs on Jina's servers, which can be convenient for not needing Colab's VM, but also can be overloaded

New CLIP and CLIP like models:

OpenCLIP ViT-B/16+ (GitHub)

by mlfoundations
Four weeks ago we reported on the LAION 400M trained OpenCLIP ViT-B/32. Last week we reported ViT-B/16. Now they released a ViT-B/16+ beefed up version, that overcomes OpenAI's ViT-B/16 performance. I am not aware of any notebook that uses that yet, but I will soon add all OpenCLIP models as options on MindsEye

Text-to-3D synthesizers:

CLIP-Forge code and models released (GitHub, Paper)

by AutodeskAILab
Generate 3D shapes from text!

Learning AI Art:

AIAIArt course (GitHub, Discord)

AIArt is a free and open source AI art course by John Whitaker. There are synchronous classes for the next few Saturdays 4 PM UTC on Twitch. All previous classes stay recorded and available on Google Colabs on the GitHub link