Rank 128 LoRA extraction of https://civitai.red/models/2600510/juggernaut-z againt Z-Image-Base in order to be able to use it with Z-Image-Turbo. Mainly done because my setup (sd.cpp) don't work with quantized version of ZiB and derivatives 😭

All showcases pictures are using Euler+Smoothstep (~Beta), 8 steps, CFG 1, Flow shift 3, with a Q4_K quantized version of both ZiT and Qwen3-4B and were made on a GTX 1060 (6GB VRAM) using Stable-diffusion.cpp.

Full ownership of this LoRA stays with https://civitai.red/user/KandooAI, merge are forbidden, this is for your personal use with ZiT on-site or locally only.

The extraction was done using this script adapted to be able to handle ANY model as long as they have the same keys (which means i had to re-key Juggernaut Z to unfuse the qkv layers).

import sys, re
Very basic args check
if len(sys.argv) < 4:
print(f”Usage: {sys.argv[0]} origin.safetensors tuned.safetensors lora.safetensors”)
exit(1)
Importing the libs
from safetensors.torch import load_file, save_file
import torch
from tqdm import tqdm
No gradient needed
torch.set_grad_enabled(False)
The most BASIC svd extraction of lora weight
def extract_lora_weight(m, r):
u, s, vh = torch.linalg.svd(m.cuda())
B = u[:, :r] @ torch.diag(s[:r])
A = vh[:r, :]
return (B.cpu().contiguous(), A.cpu().contiguous())
Load models
print(f”Loading {sys.argv[1]}…”)
orig = load_file(sys.argv[1])
print(f”Loading {sys.argv[2]}…”)
tune = load_file(sys.argv[2])
wk = list(set(orig) & set(tune))
Get ready
lora = {}
rank = 128
count = 0
Extract LoRA
print(“Extracting…”)
for k in tqdm(wk):
diff = tune[k].to(dtype=torch.float) - orig[k].to(dtype=torch.float)
if torch.mean(torch.abs(diff)) < 0.0001 or len(diff.shape) not in [1,2]:
# Either not the right shape or not enough difference, let’s skip it
del diff
continue
if len(diff.shape) == 1:
count +=1
if k.endswith(“.bias”):
lora[k.replace(“.bias”, “.diff_b”)] = diff.to(dtype=torch.bfloat16)
else:
lora[k.replace(“.weight”, “.diff”)] = diff.to(dtype=torch.bfloat16)
del diff
else:
if diff.shape[0] <= rank or diff.shape[1] <= rank:
# Too small, let’s skip it for now
del diff
continue
count +=1
B, A = extract_lora_weight(diff, rank)
del diff
base_key = k.replace(“.weight”, “”)
lora[f”{base_key}.lora_B.weight”] = B.to(dtype=torch.bfloat16)
lora[f”{base_key}.lora_A.weight”] = A.to(dtype=torch.bfloat16)
Save LoRA
print(f”Stored {count}/{len(wk)} keys to LoRA.”)
print(f”Saving {sys.argv[3]} …”)
save_file(lora, sys.argv[3])
print(“Done!”)
exit(0)