RDBT | Anima
Details
Download Files (1)
Model description
RDBT [Anima]
Mid scale finetuned + guidance distilled.
I use it as a starting point to stack more style LoRAs.
See this page for update log. Random experiment, random quality. New version != better version. Feel free to leave feedback.
See this page for original LoRA (update more frequently, probably).
This model is based on
ym: AnimaYume (hf link) (civitai link). Has latest dataset.
b,p: Anima pretrained (hf link)
Sharing merges using this model is not allowed. If someone is selling this model as their own, I'm happy to list them here so everyone knows. Known model thieves: NukeA.I (behind paywall on tensorart). I wrote a story about it. Also contains a simple guide for trainers about "how to bake special trigger word into your model".
Usage:
Settings:
CFG scale: 1~4. This model has been guidance distilled. You can disable CFG (CFG 1) and run the model 2x faster. Cover images are without CFG for demonstration.
Steps: 16~24. If you need low steps (8~12). Try to add 0.2~0.5x turbo lora.
Prompt
Always specify style, or use a style LoRA. Otherwise, you will get random/mixed style. This model does not provide overfitted default style. This is a feature, not a bug.
Quality tags:
It's recommended to omit all the quality tags, or just keep the "masterpiece", if you're not confident. Omitting those redundant tokens allows LLM to pay more attention on other words.
Quality tags have been reinforced during distillation. Thus they don't have noticeable effects. Same as negative tags. If you use cfg, there is no need to dump "score_1, blurry, worst quality, jpeg artifacts, extra arms,... x100 words" in your negative prompt. Those things have been distilled out.
Training settings:
~10k images finetuning -> guidance distillation
All captions are NL from Google Gemini.
Optimizer: adamw, constant lr 0.00002.
LoRA rank/alpha 24.
Guidance distillation target CFG 4.
Block 0-2 and adaln linear layers are skipped.
FAQ: This "guidance distillation" and Anima Turbo.
Anima turbo:
Can generate high-quality images in 4 steps without CFG. 12x faster, compared with 30 steps with CFG (60 steps total).
Has the highest stability, and lowest diversity.
You can lower the LoRA strength to get the diversity back, but you will need CFG again.
RDBT guidance distillation:
Need 12~16 steps. 4x faster.
Try to get a trade-off between speed, stability and diversity.
More natural (?). Anima Turbo adds so much details and feels unnatural. Seems to be reinforcement learning.