yolo-diffusion

A simple demo for quick experimentation with Stable Diffusion 2.0 applied to specific objects in 2d images, detected and segmented via YOLOv8.

You can get started in collab now - or watch the walkthrough videos below first.

Overview

With this simple demo you can start with any image - like the one on the left below.

You can then select any object in the image - like the pizza. YOLO will detect and segment this object, and then you can replace it with a diffused suggestion via Stable v2.0.

Like "a pile of cash" - shown on the right.

You can take the same image and do the same thing with another object - like the cat - as shown below.

Here we used the prompt "'sinister looking cartoon squirrel wearing sunglasses, high resolution'".

Here's another example. In this case we replace the person with 'an ape, smiling, high resolution, holding something"

YOLOv8 has more than 70 pre-trained objects that can be quickly detected / segmented, then fed to Stable Diffusion. See the demo notebook for a complete list of available objects.