This article originally appeared on Popular Science.

Open AI, developers of the AI text-to-image generator DALL-E 2, have just announced a new feature for the app called “outpainting”. It allows users to extend existing images and works of art with AI-generated content. It’s pretty exciting, and hugely expands the capabilities of the tool. 

Related: AI-generated image wins art contest, ‘actual’ artists irate

What are text-to-image generators? And how do they work?

DALL-E 2 is one of the most popular text-to-image generators available at the moment. With more than a million users, it’s no wonder that content created by it seems to be everywhere. (A lot of other text-to-image generators are either in a closed beta, like Stable Diffusion, are not available to the public, like Google’s Imagen, or are much more limited in scope, like Craiyon.) 

DALL-E 2 takes a text prompt, like “an astronaut riding a horse in the style of Andy Warhol,” and generates nine 1,024-pixel by 1,024-pixel images that illustrate it. It uses a process called “diffusion” where it starts with randomly generated noise and then edits it to match the salient features of the prompt as closely as possible. 

Until now, users were limited with the size and aspect ratio of what they could create with DALL-E 2. The AI program could only generate 1,024-pixel by 1,024-pixel squares—anything larger or a different shape was out of the question.

DALL-E 2 does offer a feature called “inpainting” to modify details in existing artworks, but to actually create a bigger canvas involved manually stitching different sections together using an app like Photoshop. (For different aspect ratios, you could crop your image, but that reduced the overall resolution.)

DALL-E image extender

Now with “outpainting,” the only limit users face—other than the content filters—are the number of credits they have. (Everyone gets 50 free generation credits during their first month and 15 to use every month after that. Blocks of 115 additional credits can be purchased for $15.) Generating an initial image takes one credit, as does every additional outpainted section. 

Outpainting works as an extension to DALL-E 2. Users select a 1,024-pixel by 1,024-pixel square area where they want to extend the image to and can specify any additional prompts to guide the AI. For example, to add more of a background to the astronaut on a horse, you could change the prompt to “an astronaut riding a horse on the moon with stars in the background in the style of Andy Warhol.” 

For each outpainted section, DALL-E 2 will offer up four possibilities for users to select. If none of them work for the image, you can get it to try again. 

Most impressively, outpainting “takes into account the image’s existing visual elements—including shadows, reflections, and textures.” This means that any details added “maintain the context” of the image and can really look like part of a coherent whole. 

Extending Girl with a Pearl Earring

In DALL-E 2’s announcement of outpainting, there’s a timelapse showing Girl with a Pearl Earring by Johannes Vermeer being extended to around 20 times its original size. Instead of a simple portrait, it shows a young woman standing in a cluttered house. It’s fascinating to see because so long as you don’t look too closely, it really does look like an extension of the original painting. The overall style and mood are spot on. It’s almost like an imaginary behind-the-scenes shot.

If you want to try outpainting, you will need to sign up to DALL-E 2. Open AI is currently operating a rolling waitlist. If you want to sign up, you can do so here. 

The post Peer ‘behind-the-scenes’ of famous paintings with the help of AI appeared first on Popular Photography.

Articles may contain affiliate links which enable us to share in the revenue of any purchases made.

In a paper published in Scientific Reports earlier this year, researchers at Radboud University in the Netherlands, led by PHD candidate Thirza Dado, combined non-invasive brain imaging and AI-learning models in an attempt to read peoples’ minds—or at least recreate the image they’re looking at. It’s a fascinating experiment, though it’s easy to overstate its success. Still, mind-reading AI might not be as far off as we think.

fMRI and AI imaging

Functional magnetic resonance imaging (fMRI) is a noninvasive technique used to detect brain activity by measuring changes in blood flow to different areas of the brain. It’s been used for the last few decades to identify which parts of the brain are responsible for which functions. 

In this study, Dado’s team went a step further. They used an AI model (specifically a Generative Adversarial Network, or GAN) to attempt to interpret the fMRI results and convert the readings back into an image. The results are pretty impressive.

A trained AI

In the study, Dado’s team showed participants undergoing an fMRI 36 generated faces repeated 14 times for the test set and 1,050 generated faces for the training set (over nine sessions). 

Using the fMRI data from the 1,050 unique faces, they trained the AI model to convert the brain imaging results into actual images. (It works like a more primitive version of DALL-E 2 or Stable Diffusion.)

The results of the study, then, are based on the AI model’s interpretation of the fMRI data from the 36 faces in the test set. You can see a sample of them above. The image in the first column is the target image, and the images in the second and third columns are the AI-generated results from the two subjects. 

Is this mind reading?

While it’s easy to cherry-pick a few examples where the image (re)created by the AI closely matches the target image, it’s hard to call this mind reading. The results of the study measured the accuracy of the AI in matching the gender, age, and pose, as well as whether the generated face was wearing eyeglasses, and whether the generated face was smiling, not whether or not the generated face was recognizable as the target. 

It’s also important to note that the AI was trained on fMRI data from the test subjects. If you or I were to hop into an fMRI machine, the results would likely be incredibly scattershot. We’re still a long way from being able to accurately read anyone’s mind—with or without a car-sized scientific instrument. Still, it’s fascinating to see how AI tools and machine learning can play a role in other areas—rather than just winning fine art competitions.

The post Minding-reading AI creates images from human brainwaves appeared first on Popular Photography.

Articles may contain affiliate links which enable us to share in the revenue of any purchases made.