Tetori(Orisries)

Tetori, the Holy Church Angel from the game Orisries, was trained with a total of nearly 300 mixed images.

The first two models were trained with 3,000 images and about 300 images respectively?

____________

The colors used in this game seem to cause details to collapse easily. This is a solution experiment I came up with after multiple trainings.

Compared to other results, this solution seems much better. I used cosine scheduling and adamw8bit's learning rate adjustment, but trained twice and used interrupts to save the state for continued training and adjustment.

First, use a large learning rate (25e-4) to learn for about three to four rounds, then double the number of rounds and adjust the learning rate to between 1e-45e-5, and the weight decay is 0.1 both times.

You can quickly search the overall large space first, and then fill in the small details after it is correct enough. Does this method look right?

There are also some ideas that suggest using a constant large learning rate for the first time and a small learning rate of cosine for the second time. Which is more efficient? Learned from LLM and other techniques.