The modern world is developing rapidly, and new technologies are appearing. Information technology is becoming increasingly essential for humanity, and AI does not stand still either. AI is an algorithm that can learn itself. The field of AI includes mathematics, statistics, probability theory, physics, machine learning, computer vision, and so on. Today they recognize speech, write texts, create images, and analyze network traffic. One such popular AI is Stable Diffusion.
What is Stable Diffusion?
Stable Diffusion is a deep learning, text-to-image model. This open-source program was released in 2022 and has become a trend due to the quality of created images based on text descriptions. Check this detailed Stablle Diffusion guideline to start using it just now.
Stable Diffusion was developed by the startup Stability AI and the CompVis group at public research universities LMU Munich and Runway, and LAION. In October 2022, Lightspeed Venture Partners and Coatue Management invested $101 million in Stability AI.
Why is it so popular?
In 2022, many AIs for imaging were released, such as Dall-e 2, Imagen, Craiyon, and Stable Diffusion. Stable Diffusion stands out among the other AIs because it is open-source and easily scalable. Everyone can install and access it through a local network or API. It has become popular because of the “img2img” technique, which can translate any low-quality image into high-quality. Also, it gamifies and creates 3D images. Artists and 3D modelers can visualize ideas with it without spending days or weeks, which helps lower the threshold of entry into some areas of IT.
How does it work?
There are three main stages of Stable Diffusion.
Text Encoder (CLIPText)
The user enters a text description that goes into a Stable Diffusion component called a transformer. The transformer translates the text description into a digital form and of a vector of numbers that describes each word/token. This token embedding stores in an array.
Diffusion
Diffusion is a process that takes place within the information creation phase. How does diffusion work? It performs inside the Image encoder component. Firstly, token embeddings go into Image Information Creator with a random image information tensor. Then AI compresses the original image. It adds a hidden noise layer to the compressed image with UNet. The Noise Predictor (UNet) is the main component that predicts noises and imposes noises on images. Then the process repeats until the image consists entirely of noise. So, it creates dozens of noisy drawings and saves processed information in an array.
The mathematical formula of the diffusion model for this is:
And training algorithm is:
In other words, we take a random sample from complex data, sample a noise level, and sample some noise from a Gaussian distribution, and this noise corrupts the input. The neural network predicts noise based on corrupted images.
Image Decoder
The main trick of Stable Diffusion is that the diffusion process works in reverse order. All processed image information is stored in an array, and AI uses UNet to take noisy images from there. Then the image generation process starts with reverse diffusion. UNet gradually denoises images by removing latent noise layers and draws new pixels. The process repeats dozens of times until the AI gets quality output. The result can be drastically different from the original because it can be drawings, perhaps smoothed out or even in a different style.
Work examples
Stable Diffusion can take care of all your needs, you just need to write down exactly what you want. Jim Clyde Monge published his examples of images made with Stable Diffusion.
- Prompt: !dream “Old viking woman with braids in gray hair wearing fur and jewelry :: very detailed, symmetric, unreal engine, rim-light” -i -S 474323078
- Prompt: !dream “jean-claude van damme as tyrion lannister”
More examples:
Benefits of Stable Diffusion
- Fast image processing and detailed results with high quality
- You can create an image in any style by providing a detailed text description (painting, cell shading, cartooning, 3d, photos)
- Endless expansion possibilities and open source
- Works well with symmetrical faces and can blend multiple faces
- Outputs result faster than many AIs
The future of Stable Diffusion
Overall, stable diffusion is a promising technique that has the potential to improve the quality of generated images, and we will likely see many advances, for example, the integration of other forms of data such as video, audio, and text soon. Another improvement could be the use of more complex architectures and more powerful computing resources, allowing models to generate images with higher resolution. If you need assistance of professional software development team, get in touch.