This Model Makes Wolfy-Nail's Character Wendy

this was part of a bounty I recently fulfilled because I had gotten a bit rusty and wanted to train another model. it makes the character, what more should I say.

How I trained it

Dataset Curation

I collected 9 good images from e6 using the following filter

-long_image -animated solo -dialogue -comic -webm -multiple_versions -flash_game wendy_(wolfy-nail)

I then generated and inpainted some examples using a prototype version of the LoRA (V2.3) and added them to the dataset for V3. my total dataset for V3 was 13 mediocre instance images, and 13 reg images of the wolf class scraped from e6 with a similar filter as above, just replace wendy_(wolfy-nail) with wolf and add order:favcount to get the most favorited images so we have some good quality posts to train with. unfortunately this selection process for reg images captured an old snowskau post and so if you prompt for snowskau the style is now biased towards that older snowskau style rather than the modern one.

Hardware and Training Parameters

I used kohya sd-scripts for training and each 1000 steps (batch size 4 so 4k samples) took approximately 45 minutes of GPU time on a RTX 4000 Ada SFF. I ran a large number of training runs mainly because I had never done a character LoRA before and wanted to figure out what particulars are important for it, trying different tagging, and hyper parameters. in the end I determined the best tagging is as verbose as you can possibly muster (I just used literally e6 tag and some quality tags), with caption tag dropout (NOT caption dropout) enabled. I also utilized zero terminal snr because data leakage on a small dataset is incredibly undesirable. I used the AdamW optimizer, a learning rate of 3e-4, network dimensions of 2, network alpha matching network dim and a cosine scheduler.

I hope this will work as a PSA to model trainers, ther only reason you would need a high rank for a single character LoRA is if you want it to overfit. this is especially true if you have a small dataset. LoRAs stand for "Low-Rank Adaptation" and it's designed to have the most efficent gains when it is low rank. if you have a small dataset, lower rank is desirable, as low as 1 or 2 is acceptable in a surprising number of situations.