Our Client:

Dot Square Lab

Situation:

Dot Square Lab recognised the transformational potential of text-to-image generative - which uses a text-based prompt to generate high-quality and personalised images - in multiple sectors, from marketing, graphic design, construction and more.

The aim was to create a platform for fine-tuning Stable Diffusion models using DreamBooth and deploy in a production environment. The Stable Diffusion model is a state-of-the-art text-to-image generation method. However, adapting this model to be compatible with new ‘concepts’ (i.e., objects) requires a complex training pipeline and software setup.

The platform would automate new model training and deployment as they are fine-tuned, without requiring manual intervention. This would enable any user or business to train a model using personally selected images and generate custom images.

Task:

DSL set out to develop a platform for fine-tuning Stable Diffusion models with DreamBooth. The core requirements were scalability, easy and minimal manual intervention deployment to handle the growing demand for generative AI tools.

DSL built and implemented a continuous integration/continuous deployment (CI/CD) pipeline which allows for automated and no-maintenance approach to shipping new features once ready. Separate staging and production environments enable beta testing before a new release to the live platform.

The infrastructure was managed with Kubernetes, which provides replicas, rolling updates, load balancing, networking, persistent volumes, and auto-scaling out of the box. Additionally, resources were managed by GKE Autopilot, which automatically managed virtual machine (VM) scaling when more resources were required. Shared persistent storage enables caching and model storage across VMs so data didn’t have to be pulled each time a new VM was added. The system state was stored by the backend (FastAPI) and database (MongoDB), which allowed us to restart the ML pipeline without dropping the processing queue. Finally, logs for each job were stored in the database in case of an error, making it easy to track and fix failed jobs.

Result:

The platform allows for easy experimentation with Stable Diffusion models using DreamBooth, making it possible to fine-tune models on new concepts in a fast, simple and scalable way. This is a leap forward in terms of democratising access to text-to-image generative AI tools by enabling individuals, teams or businesses with little-to-no experience in machine learning development or deployment to train custom models for unique use cases.