This week in multimodal ai art (07/May - 14/May)

Follow on Twitter, come hang on Discord, consider supporting on Patreon. Try MindsEye, our image generation GUI

Multi-modal single models

GATO announced

Collage synthesizers updates

CLIP-CLOP released*

Text-to-image synthesizers updates

MindsEye Lite released* Centipede Diffusion v3 upgrade* Latent Princess Generator released*

New CLIP and CLIP-like models

Multi-Modal-Comparators released*

Education

AIAIART course*

* code released

Multi-modal single models:

- GATO announced (Paper, Blog post)

by DeepMind
DeepMind announced GATO - a generalist agent capable of performing tasks that go from Image Captioning to playing Atari. The innovation here is that this is using the same pre-trained model and weights for all those different tasks, rather than a few models stitched together. A huge step for multi-modal models and beyond! Very promising - but nothing open source released

Collage synthesizers updates:

- CLIP-CLOP collage assembler released (Colab, Paper, GitHub)

by DeepMind
A collage assembler using CLIP! One can load their image datasets, type a prompt and CLIP will guide rotating, resizing, moving it around the canvas to fulfill the prompt

Text-to-Image synthesizers:

- MindsEye Lite released (Hugging Face Spaces)

by multimodalart (yes, that's us!)
We are happy to have released MindsEye Lite this week! A tool to use multiple simplified versions of many multi-modal AI art models in one place. Can be good for quick experiments and to introduce new people to this technology. Try it out! For a full version try out MindsEye beta

- Centipede Diffusion v3 Inpainting Upgrade (colab)

by Zalring
We reported Centipede Diffusion three weeks ago here, it generates images with Latent Diffusion and then upscales with Disco Diffusion. It got an update to add a UI inpainting check it out!

- Latent Princess Generator (Colab)

by Dango233 and multimodalart
We have released Dango233 new model! Dango is a prolific developer in this space and it was an honor to work with him to put Latent Princess Generator in a Colab and release it. Despite the name, it is a general purpose model and one of the best available so far. We are still trying to figure out default settings. So if you get somtehing good, share with us on Twitter.

New CLIP and CLIP like models:

Multi-Modal-Comparators released (GitHub)

by David Marx (@DigThatData)
David Marx released a very useful library: Multi-Modal-Comparators. It is basically a wrapper for all CLIP and CLIP-like models available online. This can be plugged into any multimodal AI art model (VQGAN+CLIP, Guided Diffusion, etc.) so new CLIP models beyond OpenAI's can be leveraged. It was released together with the new Pytti-Tools and I have hooked it to the Latent Princess Generator as well.

Learning AI Art:

AIAIArt course (GitHub, Discord)

AIArt is a free and open source AI art course by John Whitaker. There are synchronous classes for the next few Saturdays 4 PM UTC on Twitch. All previous classes stay recorded and available on Google Colabs on the GitHub link