Flow 🎨 Flux.1-D

Overview

"Flow" is a 2024 animated adventure film directed by Latvian filmmaker Gints Zilbalodis. The narrative follows a dark grey cat navigating a post-apocalyptic world submerged in water, accompanied by various animals seeking survival and companionship. Notably, the film is entirely wordless, relying solely on visual storytelling and authentic animal sounds to convey its narrative.

The production of "Flow" spanned five and a half years, utilizing the open-source animation software Blender. The film premiered at the Cannes Film Festival on May 22, 2024, in the Un Certain Regard section, marking a significant milestone as the first Latvian film featured in that category since 1998. It received critical acclaim and broke several Latvian box-office records, becoming the most-viewed film in Latvian theaters, surpassing both domestic and foreign films. In January 2025 "Flow" won the Golden Globe Award for Best Animated Feature Film, further cementing its status in the global film industry, and at the 97th Academy Awards on March 2, 2025, it also won the Oscar for Best Animated Feature, marking Latvia's first Academy Award.

Usage

This model is LyCORIS, which is supported in ComfyUI out of the box. It was trained with the trigger words "In style of Flow", but it probably can be used without them, though I haven't tested it (all the images I've published always include the trigger words).

I used the following parameters for inference:

Model: flux1-dev (fp8_e4m3fn)
Text Encoder: t5pxxl_fp16/clip_l
Sampler: euler
Scheduler: normal
Steps: 24
Flux Guidance: 4
LoRA Strength: 1

Compatibility with other checkpoints and models was not tested.

Due to the specific nature of the movie's visuals (no people, only animals) and fairly limited dataset, the model is limited in its rendering capabilities. It does not always reliably draws complex scenes, especially those without animals in frame.

It tends to sometimes (rarely) transform humans into cats, or (more frequently) gives them cat ears. I'm not sure what caused this - whether it was incorrect captioning, poor training parameters (this is the first LyCORIS model I made), or something else entirely. At first, I was pretty embarrassed by this issue and even considered retraining the model. However, I soon realised that whenever it happens, it's actually kind of cute and even memetic, especially because people look very silly. So, I decided to take this bug as a feature, as a small gift to the furry community 😆

While I'm not fully satisfied with the model due to aforementioned issues and because backgrounds sometimes tend to be too realistic, I think the model turned out pretty decent, it captures the original character design and style well, generalizing it effectively. Also, the fact it can completely change the composition of an image used with and without LyCORIS, even with the same seed, I usually consider a good sign. A well-trained style model shouldn't just be another palette filter, after all.

I suspect model might be slightly overtrained, so there is possibility I retrain it with more diverse dataset, but I'm not sure yet.

Training

This model was trained using ai-toolkit on an RTX 3090. A total of 67 images were used in the training process. The images were captioned with CogVLM2, with appliance of the following caption prompt: "Describe the scene in natural narrative language, describe only visible physical elements and the location, including key objects, structures, and environmental details without mentioning artistic style, mood, or emotions. Avoid subjective descriptions, atmosphere, or quality judgments. Use natural language structure and flow."

For training I used the default config with the following changes:

network.type: lokr
network.lokr_full_rank: true
network.lokr_factor: 8
network.linear: 16
network.linear_alpha: 16
datasets.caption_dropout_rate: 0.1
train.optimizer: lion
train.lr: 2e-4