This week in multimodal ai art (07/Jun - 13/Jun)

Follow on Twitter, come hang on Discord, consider supporting on Patreon. Try MindsEye, our image generation GUI

4 new exciting text-to-3D notebooks, fine tuning CLIP on unlabeled images, Dall-E mini went viral, a new way to control a StyleGAN from CLIP and more!

Text-to-image updates

Dall-E Mini went viral* ContraCLIP released*

Text-to-3D updates

CLIPMesh-SMPLX released*

CLIP and CLIP-like models

MUST - CLIP fine-tuning on unlabeled images


AIAIART course*

* code released

Text-to-Image synthesizers:

- DALL-E Mini went viral (Hugging Face Spaces)

by Boris Dayma
DALL-E Mini, a text-to-image model that tries to replicate and amplify DALL-E 1 went viral on its Hugging Face Spaces for meme generation. It is very exciting to see a text-to-image model with full public unrestricted access, in a perspective of understanding its capacities and limitations through play and exploration as opposed to gatekeeping.

- ContraCLIP - Interpretable GAN generation driven by pairs of contrasting sentences, released (Paper, GitHub)

by Christos Tzelepis
ContraCLIP is a way to manipulate the latent space of an image generation GAN (such as StyleGAN) with two contrastive sentences (using CLIP!) to create a cool interpolation between each image.

Text-to-3D updates

- CLIP-Mesh SMPLX released (GitHub)

by Nasir Khalid
4 different Google Colabs were released by the author: text-to-character, text-to-facial-expression, text-to-pose, text-to-mesh (similar to the previously reported CLIPMatrix) - but faster and less memory intensive

New CLIP and CLIP-like models:

MUST - Masked Unsupervised Self-training for Zero-shot Image Classification - CLIP fine-tuning on unlabeled images (GitHub)

by Salesforce
MUST is a technique to fine-tune models like CLIP on unlabeled images, this means that you can make models like CLIP perform better on domain-specific tasks and even generally by fine-tuning it on unlabeled images.

Learning AI Art:

AIAIArt course (GitHub, Discord)

AIArt is a free and open source AI art course by John Whitaker. There are synchronous classes for the next few Saturdays 4 PM UTC on Twitch. All previous classes stay recorded and available on Google Colabs on the GitHub link