Customize SDXL with LoRAs
Learn from Thierry Moreau, Co-Founder at OctoML, how to customize your SDXL inference by using the OctoAI Asset Orchestrator to apply LoRAs. We are using a new app, Smartflakes, with a specific requirement to make product images – cereal boxes.
SDXL customization for your use cases
We built a new app, Smartflakes, to showcase one way you can customize SDXL, but there are a myriad of possibilities based on your use case. We used a community created LoRA found on Hugging Face. You can select from the many already built assets, or you could use OctoAI to fine-tune on your specific set of images for your needs.
Testing custom assets on OctoAI Image Gen Solution
We uploaded the custom cereal box LoRA to OctoAI’s Asset Library to play around with it before building it into our app. There are 10+ free assets included upon sign up you can test with if you are unsure of what custom asset you want to use with SDXL.
Testing your custom asset:
Access OctoAI’s Asset Library
Select “My Assets” to upload your asset
Fill out required fields to upload
Once uploaded select your asset, we used:
cereal_box_sdxl_v1
Select the asset and adjust the weight
Send LoRA to: Text to Image Tool
Launch the tool to test prompts, and other parameters
Now you can start testing prompts to see results. There are some known issues with SDXL generating text, so keep generating until you get a decent looking cereal box. We used the prompt: star wars wookie bits, free lightsaber inside
, to generate the below.
Add LoRAs to your API
In OctoAI you will see the code examples once you have added the desired LoRA to the demo, and it shows exactly where to add the LoRA parameter in Python, TypeScript, and cURL examples. See the cURL example below:
In the Smartflakes repo, in the route.ts file, you can see how we call the SDXL API endpoint with the LoRA and weight.
Share what you build
We are excited to see what you build – feel free to show us in our Discord, and see what other community members are creating.
Your choice of models on our SaaS or in your environment
Run any model or checkpoint on our efficient, reliable, and customizable API endpoints. Sign up and start building in minutes.