Playing Around with Stable Diffusion

Posted on Saturday, 6 May 2023 Suggest an Edit

art

stable-diffusion

If you have been following for a while now, you may have noticed that I sometimes include images in my posts. Some of these have been generated by AI, specifically Midjourney. Using Midjourney for the first time completely blew me away.

I’m currently avidly playing around with this cutting-edge technology. Primarily, I use the Stable Diffusion checkpoints (which the AI art generation community have been sharing) for this stuff. I’ve got AUTO1111’s client setup locally and ~60GB (which is actually very little) of various checkpoint models and a bunch of other things like embeddings and LoRA models.

My GTX1660 6GB GPU has put up with more than I expected. These workloads can easily consume >8GB VRAM and my barebones, unoptimized workflow doesn’t help - if I can even call it that.

Honestly, I’m just kind of staggering about. Generate an amazing image here and there, play around with tools people have open sourced, explore the huge library of fine-tuned models people are training, learn about prompt engineering. There is just so much! Things in this space are also moving crazy fast - it is unsurprising if a guide created only a month ago no longer works.

Some notable things I’ve spent my time on:

Going through a practical deep learning course created by the folks at fast.ai
Breaking open the blackbox. Calculus. Intuition. 3Blue1Brown - The Essence of Calculus
Many attempts training an art-style LoRA. Miraculously got one with decent results in the end.
- Check it out! ttgl-eyecatch-LoRA

I plan to continue all this, but I think my GPU is reaching its limits. I’m unsure as to whether I should fork out some 💰 for a more powerful GPU or to simply get a google colab subscription.

I’ll probably just wait and see until after I finish the fast.ai course.