Sophos Realism
Details
Download Files
About this version
Model description
Sophos Realism
Long time lurker, first time poster. I recently set out to create an Illustrious-based model that blends realism, Danbooru prompt comprehension, and character comprehension. I think this model achieved that goal enough to merit seeing the light of day, so here it is.
You need to use a VAE with this model; there is no VAE baked in. The default SDXL VAE works fine.
📝 Version Notes
Consult the notes below for the differences between the posted versions.
v1.4 - ExpressiveV2
A tweak to the v1.3 formula that I think improved it overall.
v1.3 - Expressive
This version achieves improved expressiveness over v1.2 while remaining a balanced checkpoint in terms of realism and prompt comprehension. It does character expressions especially well, but you might find v1.2 is better at some compositions. v1.1 remains the best checkpoint for "maximum realism" if that's your priority.
I'd say use v1.3 if you like v1.2 but you need a little more creativity or range of character expressions.
v1.2 - Balanced
This version sits somewhere between v1.0 and v1.1 in terms of realism and prompt comprehension. It seems to be about as flexible as v1.0 but with better overall realism.
v1.1 - Realistic
Use this version if you're willing to sacrifice some prompt comprehension and LoRA compatibility for more realism. It's great if you just need character portraits, and it can do some more complicated compositions if you work at it, but its prompt comprehension isn't as good as the other checkpoints.
v1.0 - Semi-Realistic, More Anime
This version should really be considered semi-realistic. It still looks very anime most of the time, but it can do almost everything an anime model can do, which can be fun. Use the recommended LoRAs below to nudge it more in a realistic direction.
Suggested Uses
You can use this model more or less like you would use an Illustrious anime model, but the results look realistic. It understands danbooru tags and does a decent job of realistically rendering popular fictitious characters. It's pretty good at realism too, just generally speaking. If you want a more realistic edge to the finished photo, try using this model to get the generation started and then switch to your favorite SDXL realism model for refinement of those fine details.
Usage Tips
Add Emphasis to Tags: You will have to add more emphasis than you might normally use to trigger the effect of certain danbooru tags for unrealistic things that you wouldn't normally see in real photos. In short, if it doesn't work at first, try upping the weight until it does (hopefully).
Boost Character Details: For characters, it helps to add some of their defining characteristics and also add more weight to the character name to guide the effect. For example, "aerith gainsborough" is okay, but "(aerith gainsborough:1.2), green eyes, single braid, pink ribbon in hair" is going to be even better. You don't really need to do that most of the time for the Illustrious anime models, but it helps with this one.
Use Concept/Chracter LoRAs: LoRAs are your friend. If a particular character or concept isn't well represented in the danbooru dataset, the effect of the trigger word might be too weak. Search Civit for a LoRA that can help. Odds are you'll find one... or dozens. Don't be afraid to apply those LoRAs at a lower weight. Sometimes a little dab will do ya.
👉 Use Recommended LoRAs: (This is only important for the v1.0 model.) See below! The v1.0 model thrives on two LoRAs in particular. The v1.1 model doesn't really need them.
⚠️ NSFW Note ⚠️
This model will do NSFW very nicely. It didn't lose any of that Illustrious potency, that's for sure. Please be responsible with what you do with it.
Note on Sample Images
I may have "cherry picked" my sample images. I haven't heard anyone give a robust definition for what that means in this context, so I'll just tell you the approach I took and you can decide for yourself whether my results are valid:
What you see is what you get in terms of the prompts and the LoRAs. No extra tricks, no targeted inpainting, no hidden loras for characters (if I used one, you'll see it in the prompt), no controlnets, nothing like that. Just hi-res fix upscale and ADetailer to sharpen the faces. Those are widely-available, well-known techniques, so I felt justified in using them.
I generated the images until I hit on the first one that felt good enough. Some of them were one-shots, but I definitely didn't one-shot the more complex compositions. If that's cherry picking, then guilty as charged, and I would simply say in my defense that I see no other way to use these tools. In my humble opinion, iterative prompt refinement and luck is required to get good results from any diffusion model.
The results you see are highly dependent on the LoRAs that I am recommending below. The model is still pretty good without them, but the LoRAs bring out the magic. I would have merged them in except I think there's so much value in being able to control their strength.
Recommended Prompts
Positive
8k UHD, (absurdres:1.2), (masterpiece:1.2), (best quality:1.2), (realistic:1.2), (intricate details:1.2), (film grain:1.2)
Negative
(text, logo, username, signature:1.2), photoshop, doll, (silicone:1.1), fake, filter, airbrush, 3d max, colourless, impassive, shaders, (anime eyes:1.2), (glossy skin:1.2), (plastic skin:1.2), cartoon proportions, illustration, digital art, simplified, abstract, (unrealistic:1.2), impressionistic, (lowres:1.2), (bad anatomy:1.2), (bad hands:1.2), (missing fingers:1.2), (worst quality:1.2), (low quality:1.2), (cartoon:1.1), (anime:1.1), drawing, sketch, (artificial:1.1)
Note: I think you could prune quite a few of those tags out of the negative prompt and still get perfectly fine results. More experimenting is needed.
Embeddings
In my testing, this model works fine with popular positive/negative embeddings like Realism_Illustrious_Negative_Embedding and Realism_Illustrious_Positive_Embedding. Give them a try if that's your thing.
LoRAs
LoRA
The LoRAs are the secret sauce for the v1.0 version of this model. (No longer needed for v1.1.) This LoRA combination really juices the v1.0 model and makes it shine. They are a must for getting the results you see in the example images.
You can go as easy or hard on the dark LoRA as you want depending on your needs, but ~0.4 is a fine setting for general purpose use cases to enhance any image.
The Stabilizer LoRA adds a little extra creativity and prompt adherence. Good values seem to be in the 0.1 - 0.4 range. If you go much higher than that, you'll get artifacts. I think ~0.2 is the sweet spot for general purpose usage.
<lora:illustriousXLv01_stabilizer_v1.185c:0.2> <lora:dark:0.4>
Other LoRA Compatibility
Generally speaking, any LoRA that works with the Illustrious anime models should work just fine on this model, although depending on the effect, it may not translate over nicely to the realism style. However, I've been pleasantly surprised with the results so far! Pretty much any Illustrious or SDXL LoRA, and even some Pony LoRAs, should work. Just remember that your mileage may vary.
Sampler Settings
Samplers
Play around with your favorites, but here's what tends to work for me:
DPM++ SDE, ~30-40 steps: Takes a while, but good results if you're patient
DPM++ 2M, 40-50 steps: A tried-and-true classic. Works great.
ER SDE, 30-50 steps: A newcomer that is my go-to these days.
CFG
I use 3.5 CFG with Mahiro CFG in reForge. If you use regular CFG, try it in the 4 - 5 range, or use whatever works best for you.
Clip Skip
You can do either clip skip 1 or clip skip 2. The difference is very slight, but I think I lean towards preferring clip skip 2.
Scheduler
I like the Phi scheduler if you have it available, otherwise you can't go wrong with SGM Uniform, Beta, or Karras. Ultimately use whatever works best for you.
Merge Details
Credit goes where credit is due. This merge wouldn't have been possible without the efforts of the people who finetune the base models. Please join me in saying thank you to these fine creators. 🙏
Illustrious Models
v1.0
v1.1
v1.2
This model is a blend of v1.1 with Hyphoria Real, which is a great Illustrious realism model in its own right. You should definitely check it out if you like these models at all.
v1.3
This checkpoint is a blend of v1.2 with DAMN! by coyotte, which is a solid Illustrious realism model that you should totally check out.
v1.4
Merged v1.3 with a blend of other Illustrious realism models, some of which were already present in the blend.
SDXL Models
v1.0 and v1.1 (present in other versions that merge v1.0 and v1.1)
Donations
It didn't cost me anything but time to do this, but if you're so inclined, I have a Kofi page. Thanks!