Mix and match styles with OctoAI: Create your own fine-tuning assets and use it together with popular community created ones, with OctoAI’s Asset Library

When you’re building with AI image generation, you’ll often find that no single checkpoint or LoRA captures the exact image style you’re looking for. The most interesting customization often comes from mixing & matching several assets to create unique images. OctoAI makes it easy to create and manage your own custom assets, then combine them during image generation. Let’s walk through a simple example of creating a Halloween-themed ghost then giving our ghost some Christmas spirit!

Image generation with custom assets

Customers can easily create or import fine-tuning assets to customize images. Here, I’ll upload a LoRA that gives images a paint splash style. I can easily upload using the OctoAI web application, CLI, or Python SDK:

Then I’ll start generating images using a consistent prompt, along with various custom assets. I’ll start with the Stable Diffusion XL base model, then try a 3D cartoon style checkpoint, and finally add a banshee LoRA:

The Halloween-themed ghost looks great using a custom checkpoint + LoRA, and now I’ll simply update my image prompt to describe the ghost as Santa Claus:

It didn’t quite turn out the way I wanted! The custom assets do a great job of creating the Halloween theme but they limit my flexibility to further update the subject. Fortunately, I can quickly create a fine-tune to better adapt the image details.

OctoAI fine-tuning

Customers can fine-tune Stable Diffusion v1.5 or Stable Diffusion XL on OctoAI. Using a few images from my previous generations, I’ll fine-tune to create a Stable Diffusion XL LoRA that will give me more control over the image details:

The fine-tuned LoRA is stored in the Asset Library along with my other custom assets, and immediately available for image generation. Now let’s try again to depict the ghost as Santa Claus. This time I’ll combine my fine-tuned LoRA along with the paint splash LoRA I uploaded earlier:

Fine-tuning gave me much more control over the images, and our Halloween-themed ghost finally has the Christmas spirit! Check out this video to see how easy it is to mix & match image styles.

Putting it all together