Hunter - Bloodborne - LoRA Flux 1 Dev

Workflow

Dataset creation:

1. I've used gallery-dl (GitHub) to scrape images of of different websites

2. I've removed duplicates with czkawka (GitHub)

3. I've chose my favorite images and captioned them using ChatGPT (gpt4-o)

4. I've captioned the rest with joycaption using this taggui fork (GitHub)

5. I've used [Code 1] to make sure the captions were made correctly using an llm with ollama

6. I've trained the model for 6000 steps using [config 1] with ostris/ai-toolkit (GitHub)

Code 1:

import ollama
import os
directory = [dataset-path]
instruction = “”“You will receive a caption text that describes pictures featuring a the game character from Bloodborne called the “Hunter”. Your task is to seamlessly incorporate the characters name into the caption with minimal changes when the characters name is not mentioned. The goal is to subtly include the characters name without explicitly stating that the character is called the “Hunter” if this is not the case already. If the characters name is already mentioned in the caption just return the caption as it was. Do not return something else besides the caption”””
modelname = “Tiger-Gemma-9B:latest”
output_directory = [output-path]
def pairCaptions(txtfiles, first_directory, instruction, modelname, output_directory, second_directory):
progress = 0
for txtfile in txtfiles:
with open(os.path.join(first_directory, txtfile), “r”) as content: caption = content.read()
with open(os.path.join(second_directory, txtfile), “r”) as content2: caption2 = content2.read()
modifiedcaptions = ollama.generate(model=modelname, prompt=f”{instruction}{caption}. And here is the descriptory list: {caption2}”)
open(os.path.join(output_directory, txtfile), “w”).write(modifiedcaptions[“response”])
progress += 1
print(f”Modified {txtfile} ({progress}/{len(txtfiles)})”)
print(modifiedcaptions[“response”])
def CaptionsEdit(txtfiles, directory, instruction, modelname, output_directory):
progress = 0
for txtfile in txtfiles:
with open(os.path.join(directory, txtfile), “r”) as content: caption = content.read()
modifiedcaptions = ollama.generate(model=modelname, prompt=f”{instruction}This is the caption text: {caption}”)
open(os.path.join(output_directory, txtfile), “w”).write(modifiedcaptions[“response”])
progress += 1
print(f”Modified {txtfile} ({progress}/{len(txtfiles)})”)
print(modifiedcaptions[“response”])
txtfiles = [f for f in os.listdir(directory) if f.endswith(“.txt”)]
CaptionsEdit(txtfiles, directory, instruction, modelname, output_directory)

Config 1:

job: extension
config:
name: f1_char_hunter_v2
process:


type: sd_trainer
training_folder: output
device: cuda:0
network:
type: lora
linear: 16
linear_alpha: 16
save:
dtype: float16
save_every: 319
max_step_saves_to_keep: 4
push_to_hub: false
datasets:

folder_path: [path]
(bloodborne)\hunterbb_v2
caption_ext: txt
caption_dropout_rate: 0.05
shuffle_tokens: false
cache_latents_to_disk: true
resolution:

512
768
1024
train:
batch_size: 1
steps: 6000
gradient_accumulation_steps: 1
train_unet: true
train_text_encoder: false
gradient_checkpointing: true
noise_scheduler: flowmatch
optimizer: adamw8bit
lr: 0.0001
ema_config:
use_ema: true
ema_decay: 0.99
dtype: bf16
model:
name_or_path: black-forest-labs/FLUX.1-dev
is_flux: true
quantize: true
sample:
sampler: flowmatch
sample_every: 319
width: 1024
height: 1024
prompts:
The image is a digital drawing rendered in a dark moody style with a muted
color palette dominated by shades of brown and grey. The character stands
in a commanding pose clad in a long dark coat with intricate ornate details
reminiscent of a historical military uniform. The coat is adorned with silver
buttons and a high collar and a large dark hat with a feathered plume sits
atop the character’s head casting a shadow over their face. The hunter’s facehunter
Hunter stands in a powerful, poised stance, dressed in dark, worn armor with
detailed engravings on the bracers and boots. The character’s face is obscured
by a high-collared scarf and a battle-worn helmet with broken edges. Hunter
wields a large, serrated blade in one hand and a finely crafted pistol in
the other. The muted, smoky background enhances the grim and determined aura
of the figure.
Hunter is depicted in a dynamic pixel art style, standing in an aggressive,
battle-ready stance. The character’s flowing cape and dark attire are accentuated
with bold shading and highlights, giving depth to the design. Hunter holds
a large, serrated weapon in one hand and a gun in the other, both drawn in
intricate pixel detail. The overall composition, framed by a thin border,
captures a sense of action and intensity.
Hunter stands in a mystical and foreboding scene, surrounded by colossal,
shadowy statues of beasts emerging from swirling orange and blue mists. The
character is dwarfed by these monumental figures, yet their posture remains
calm and composed, holding a weapon at their side. A large, glowing moon looms
behind a domed structure, casting an ethereal light over the surreal landscape,
creating a vivid contrast between light and shadow, heightening the atmosphere
of mystery and tension.
Hunter is captured in a moment of intense agony or transformation, with their
head thrown back and hands clawing upward. The character’s form is engulfed
in darkness, contrasting sharply with the vivid, fiery red background that
swirls around them. Their eyes glow with a fierce, almost demonic light, and
the scene exudes a powerful sense of torment and rage. The dramatic use of
color and the dynamic pose create a haunting, visceral image of Hunter in
the throes of a violent and overwhelming force.
The image is a digital drawing in a dark gothic style featuring a male character
with a stern imposing presence. The character is dressed in a long dark coat
with intricate stitching and a high collar adorned with a large dark hat that
covers most of his face giving him a mysterious intimidating appearance. His
eyes are hidden behind the hat adding to the enigmatic aura. The coat is detailed
with gold accents on the shoulders and cuffs and a large belt buckle.hunter
The image is a digital drawing in an anime style depicting a young woman with
a pale complexion and long lavender hair styled in a braid with a blue ribbon.
She wears a black wide-brimmed hat with a dark almost black coat and gloves
giving her a mysterious and ominous appearance. Her eyes are large and expressive
with a hint of sadness or determination. The background is a fiery scene with
dark swirling smoke and flames suggesting a battle or a post-aphunter
Hunter is illustrated in a stylized, semi-realistic portrait, with a focused
side profile. The character has striking white hair partially concealed by
a dark, tattered hat, while their lower face is obscured by a black scarf.
Hunter’s piercing blue eyes stand out against the muted tones of their green
cloak and dark attire. The clean, minimal background enhances the character’s
sharp, detailed features, conveying a sense of calm determination and mystery.
neg: ‘’
seed: 99
walk_seed: true
guidance_scale: 4
sample_steps: 20
meta:
name: f1_char_hunter_v2
version: ‘2.0’

PS: If you want the dataset please contact me. I just don’t want to get CivitAI in copyright trouble.