DALL.E 2: To infinity and beyond sentence-to-image generations.

Since the dawn of the computers humans have theorized about a kind of human like intelligence developed form a computer who would dominate the human race, well that was the theory in practice artificial intelligence is a simulation of human intelligence by machines done with the help of vast amount of data which is used to get a desired outcome.

Mystical Artificial system that creates images from from any description; giving a form your wildest dreams.

Eg of a image created by 2

What is Ai ?

Since the dawn of the computers humans have theorized about a kind of human like intelligence developed form a computer who would dominate the human race, well that was the theory in practice artificial intelligence is a simulation of human intelligence by machines done with the help of vast amount of data which is used to get a desired outcome. AI are mostly used for things which would be vastly difficult or outright impossible for a regular human-being. CONFUSED well don’t worry we have got you covered the following are the ways by which the AI actually function.

The Learning Process

This is a very integral part in the functioning of an AI system. It involves the acquisition of data and the process of defining the rules which turn the data into actionable information. These rules are called algorithms and They provide the computing device with comprehensive instructions which tells them how to complete a specific task.

The Reasoning Process

This process in an AI focuses on choosing the right Algorithms to complete a desired task.

The Self-Correction Process

This is crucial aspect of an AI focuses on fine-tuning the algorithm so that it gives the most accurate results as possible.

Brief description on AI

Importance of AI

Automation: Now Repetitive tasks can be easily done with the help of AI in any condition without the problem of exhaustion or error.

Enhancement: AI can not only make product and services smarter but also effective and faster. It can work 24×7 without any exertion, fatigue or breaks.

Risk to life: It can work very Dangerous jobs , eliminating the risk to human life.

THE DALL.E 2

Art is one of the greatest claims of humans. We pride our selves on our great art and their artists. Who knew that the likes of Leonardo da Vinci, Vincent van Gogh, Pablo Picasso would be joined by an AI. Yes the DALL.E 2 is the next great feat of human ingenuity, a versatile model that can go beyond sentence-to-image generations.

Explained like you are 5

Ever wanted to see what a chainsaw wielding astronaut, riding a horse would look like well this is now possible. The DALL.E 2 is a new ai that allows you to create an image from the description that you give it. The AI not only creates one image but several variations of it so that you can chose the image that you envisioned in our head. Think it and the DALLE-2 can make it.

How Does the Picasso of AI work?

CLIP

the most important building block in the architecture of the DALL.E 2 is CLIP. CLIP stands for Contrastive Language-Image Pre-training, and it’s essential to DALL.E 2 because it functions as the main bridge between text and images.

CLIP is the representation of the idea that that any language can be a mode for the computer system to learn. The thing that makes CLIP different is that it not only trains the machine to identify the image but it identifies the caption of each image from a randomized list of captions. This allows the system to learn the intricacies of a language so that it can understand the difference between “a man riding a horse on fire” and “a man on fire riding a horse”.

Therefore with these capabilities clip can create a vector space whose dimensions represent both the features of the image and language. This shared vector space serves as a image text dictionary which allows the models to translate between one another.

But this is not enough to represent the entirety of DALL.E 2 , even with a way to translate all other languages one would need to learn all the pronunciations and grammar. CLIP only enables a model to take textual phrases and map them into a image but we need a way to generate these images in a way that is true to the users understanding of the text. Here enters the second building block of the DALL.E 2 system.

Diffusion Models

The crux of the diffusion model is made in the following way. We take an image and scramble it until we have a picture of complete noise. Then we train a model to reduce the noise one pixel at a time until we have the original image. This leaves us with a model which can create an image from randomness.

To infuse these two blocks we condition the diffusion models on the CLIP embeddings. This basically means that we can pass the vectors from our joint CLIP space to the diffusion model which then calculates what pixels on each step of the image generation process.

Importance of DALL.E 2

DALL.E was a 12-billion parameter model that worked using a dataset of text-image pairs. DALL.E-2 mostly does the same thing that DALL.E does, However, DALL.E 2 is far more versatile and capable of producing images of a higher resolution. DALL.E 2 functions on a 3.5-billion parameter model while using another 1.5-billion parameter model to enhance the resolution of its digitally-produced images. The model is also faster at processing images than DALL.E. 

Advantages of DALL.E 2 OVER DALL.E

More efficient:

The DALL.E-2 has far more parameters compared to DALL.E so it can produce an image of far higher resolution and it also has the diffusion model which allows it to create the image faster than DALL.E

image showing difference between DALL.E and DALL.E 2
Difference between DALL.E and DALL.E 2

More realistic:

The DALL.E 2 can create more realistic images. Images have more character with the inclusion of rounded and more complex backgrounds, realistic lighting and reflections.

DIFFERENCE BETWEEN DALL.E AND DALL.E 2

Editing:

One of the new features in the DALL.E 2 is the editing feature which allows the user to change a selected part of the image according to his/her preferences. This is known as inpainting and is a key feature which sets the DALL.E 2 apart from DALL.E.

DIFFERENCE BETWEEN DALL.E AND DALL.E 2

Multiple variations:

DALL.E 2 is also able to produce multiple variations of a single image. These variations could be an impressionistic version of the image or a close resemblance of it. The user can even give the model a second image and DALL.E 2 can combine the more vital features of both the images to form a final one. 

Text Diffs

Another cool ability of DALL.E 2 is : interpolation. Using a technique that is called txt diffs DALL.E 2 can convert one image into another. Down below is an example with the pictures of two dogs and Van Gogh’s The Starry Night.

Exciting feature of DALL.E 2

Here is a tesla being converted into another older car

Conversion of a new car into an old one using DALL.E 2

Limitations

Open AI has said it is conscious of the potential negative impact that DALL.E 2 could have in the wrong hands. In today’s world of deep fakes, the model could easily be used to produce misinformation or racist imagery, which is why OPEN AI has allowed DALL.2 to be used by developers on solely an invite-only basis. All the prompts that the model receives must adhere to a strict content policy. To completely rule out the possibility of DALL.E 2 producing any hateful or violent images, the dataset itself omitted the inclusion of any dangerous weapons. While OpenAI has said that it intends to turn it into an API eventually, it is prepared to proceed with caution in the case of DALL.E 2. 

Biases and stereotypes

DALL.E 2 tends to show people and places in a western setting by default. It also engages in gender stereotypes. When prompted these are what comes up for flight attendant and a builder.

Biases and stereotypes of DALL.E 2
Biases and stereotypes of DALL.E 2

This is what is called a  representational bias and it is quiet prevalent in models like DALL.E 2 and GPT-3. These stereotypes often categories people one or the other form on the basis of their identity(race, gender, nationality, etc.).

Harassment and bullying

Deep fakes can cause a lot of problems for individuals. DALL.E 2 could be used to harass any individual by tampering with their photos to defame them.

Explicit content

The idiom “an image is worth a thousand words” reflects this very issue. The open AI policy does not allow to create such explicit image such as a horse in a pool of blood but users have been known to involve them selves in such shenanigans. Now this may be tormenting to other users. This type of content is called spurious content.

A photo of a horse sleeping in a pool of red liquid.” Credit: Open AI

Disinformation

As stated above deep fakes can cause a lot of problems for the masses, one of them is Disinformation. This is one of the grater faults of the DALL.E 2 it can be used to cause mass panic by spreading rumors and misinformation which can be fatal at times to mass populous.

Amazing images created by DALL.E 2

Here are a few of the amazing creations of DALL.E 2 which show at what lengths the AI can go to get the pitch perfect image.

Image by DALL.E 2 of a dog and a boy
Something great form the mind of Merzmensch Kosmopol

INSTAGRAM OSiNT: let us hack

Share your love
Samudra Banerjee
Samudra Banerjee
Articles: 1

5 Comments

Comments are closed.

Home
Editorials
Articles
Search