Sign up
Log in
Sign up
Log in
On-demand webinar
Learn from our technical deep dive into using function calling to develop AI agents.
Watch now

Mix and match styles with OctoAI: Create your own fine-tuning assets and use it together with popular community created ones, with OctoAI’s Asset Library

Blog Author - Jordan Janes

Nov 16, 2023

2 minutes

When you’re building with AI image generation, you’ll often find that no single checkpoint or LoRA captures the exact image style you’re looking for. The most interesting customization often comes from mixing & matching several assets to create unique images. OctoAI makes it easy to create and manage your own custom assets, then combine them during image generation. Let’s walk through a simple example of creating a Halloween-themed ghost then giving our ghost some Christmas spirit!

Image generation with custom assets

Customers can easily create or import fine-tuning assets to customize images. Here, I’ll upload a LoRA that gives images a paint splash style. I can easily upload using the OctoAI web application, CLI, or Python SDK:

Then I’ll start generating images using a consistent prompt, along with various custom assets. I’ll start with the Stable Diffusion XL base model, then try a 3D cartoon style checkpoint, and finally add a banshee LoRA:

The Halloween-themed ghost looks great using a custom checkpoint + LoRA, and now I’ll simply update my image prompt to describe the ghost as Santa Claus:

It didn’t quite turn out the way I wanted! The custom assets do a great job of creating the Halloween theme but they limit my flexibility to further update the subject. Fortunately, I can quickly create a fine-tune to better adapt the image details.

OctoAI fine-tuning

Customers can fine-tune Stable Diffusion v1.5 or Stable Diffusion XL on OctoAI. Using a few images from my previous generations, I’ll fine-tune to create a Stable Diffusion XL LoRA that will give me more control over the image details:

The fine-tuned LoRA is stored in the Asset Library along with my other custom assets, and immediately available for image generation. Now let’s try again to depict the ghost as Santa Claus. This time I’ll combine my fine-tuned LoRA along with the paint splash LoRA I uploaded earlier:

Fine-tuning gave me much more control over the images, and our Halloween-themed ghost finally has the Christmas spirit! Check out this video to see how easy it is to mix & match image styles.

Putting it all together

Sign up and start building today at no cost, with the free tier of the OctoAI Image Gen Solution. You’re also welcome to join us on Discord to engage with the team and our community.

Your choice of models on our SaaS or in your environment

Run any model or checkpoint on our efficient, reliable, and customizable API endpoints. Sign up and start building in minutes.

Sign Up Today