Judy Hopps (On Model)

This is my own personal attempt at creating a Judy Hopps LoRA.

While base models do have plenty of knowledge about Judy Hopps, and there exists tons of LoRAs already, a lot of LoRAs and models are trained mostly off of fanart.

As I consider myself to be a Judy Hopps enthusiast, I wanted to try and recreate Judy as accurately as I possibly could.

V3 is released!

I advise using this model: /model/3671?modelVersionId=1623867

After several months, I've finally made V3 of my On Model Judy Hopps LoRA. Efforts on this began back in October as soon as 2.5 was released. As it's a large dataset, it took a long time to go through and caption everything, though the captions are by no means as comprehensive as they could be (I just did Booru-style captions manually, without the use of any automatic tagging).

Still, it should have decent knowledge of poses and outfits.

I spent a while training this, trying out different models. I did it on Pony V6, Illustrious 1.0, then I tried NTRMix, and then Gehenna.

I had been considering releasing the Illustrious 1.0 version but instructing users to use Gehenna 2 as the model, as it was giving some pretty decent outputs but it was losing a lot of the look of the movie and wasn't correctly rendering the specific trigger words and outfits.

Unfortunately IL 1.0's outputs were far too unstable for me to recommend using the base model.

More recently I had been experimenting with YiffyMix, as I've been finding it has really good prompt understanding and, more importantly, great scene consistency. But I had been struggling with making outputs not look "cooked" despite following its provided settings.

But I wanted the improved prompt adherence and consistency of Yiffy, so I decided to train V3 on it, hoping that it wouldn't turn out burned.

And holy heck I wish I tried this sooner.

All that time spent banging my head against a wall, training on different models, hoping to get improved image quality and consistency, and the answer was to just train it on Yiffy, lol.

I believe that this is as good as it's possibly going to get until we get better base models, so this will most likely be the last version I'll do for now.

That said, there won't be a version of this for Pony V6: I did wind up training this initially on Pony, but for some reason adding captions had the complete opposite effect and the outputs turned out like total crap. So 2.5 is still the best version to use if you're on Pony. Otherwise, I really recommend giving Yiffy a spin.

(Previous V2.5)

2.5 will be the last version I do prior to manually going through and captioning everything.

This version was intended to add a bit more outfit control, as it was difficult in V2 to steer Judy away from her default cop outfit. In addition to adding tags for various outfits, I've also added some tags for a few different distinct poses from the movie. In my testing, this version also seems to significantly improve the coherence of generations, and as such every example image in this gallery was generated with base Pony V6.

For outfits, you may need to manually specify what you want in addition to using the trigger word. For example, "judyciv, white tshirt", or "judyfrm, pink shirt". See images for more examples.

Outfit Tags

"judyhopps": Standard trigger for all images.

"judyblu": Her cop outfit.

"judyfrm": Her farm outfit with the pink shirt.

"judyciv": The outfit she wears when she first travels to Zootopia, a white tshirt over a black shirt with black pants.

"judymtr": Her meter maid outfit.

Pose/Concept Tags

"judymicbang": Judy surrounded by microphones, which looks a bit sus.

"judyact": Her "playing dead" with her tongue out.

"judyback": Judy in the natural history museum, on her back.

"judychairclimb": The butt shot we get when Judy tries to jump onto the oversized chair in the ZPD.

"judycry": Judy crying when she apologizes to Nick.

"judydance": Judy dancing during the Gazelle concert at the end of the movie.

"judymultiply": The sassy pose she does when she jokes that bunnies are good at multiplying.

"judypeek": Judy trying to peek over the counter at the DMV.

"judysquish": Judy between two large mammals.

"judytrainlean": Rear view of Judy leaning on the console of the train.

"judytrainhang": Attractive pose of Judy hanging off the side of the train and looking back over her shoulder.

"judykneel": Another pose I enjoy, her kneeling on the sidewalk in her meter maid outfit.

"judytrainrecline": Judy laying on her back over the train console.

Some poses work better than others, so feel free to experiment.

Next version:

The only thing really left to do now is add captions, which should hopefully resolve all remaining issues with coherency and control.

(Previous version V2)

This version adds another ~500 pictures to the dataset, for 1,146 overall. The biggest improvements here are the rendering of her face, the accuracy of the character, and the overall quality of the render.

I had planned for there to be a 1.5 which would have been V1 but with captions, but the results weren't quite good enough and in some cases the captions made it worse.

For this release, there are still no captions and the trigger word is judyhopps. I've been generating mainly with AutismMix Confetti, but if you want to stay as close to the film's style as possible, I recommend using base Pony V6. If you have the memory for it, you can use Autism for the initial image and Pony V6 as a refiner at around 60%.

Since Nick appears in a variety of the screencaps, prompting "male fox" can render him pretty effectively.

Known issues:

-Although the accuracy of the outfits is improved here, it is more difficult to change the outfit. Specifically, it sticks very strongly to her standard cop outfit. Lowering the strength of the LoRA and upping the strength of the specific outfit you want to use can help. LoRA to 0.7 and outfit to 1.2 is a good place to start.

This should be more or less fixed when I finally get around to adding captions.

-Her ears can sometimes render in both positions, up and down simultaneously. This was also a problem with V1.

Next version:

The only real thing remaining is to go through and caption the images to improve the ability to change the clothes. Beyond that, I'm pretty much satisfied with these results.

(Previous version V1):
It's trained on over 500 screencaps from the movie, each cropped to 1024x1024, and trained on 20 steps each. It is by no means perfect, but so far the results have exceeded my expectations and I'm going to enjoy generating with this for a while, lol. No captions were used.

The trigger word is judyhopps though adding "bunny" to the prompt can improve the output.

Most of these images are generated with Autism Mix Confetti, as I find it's the closest to regular Pony without having as many anatomical issues. But Cyberrealistic Pony also works very well: /model/443821/cyberrealistic-pony

In both cases, I recommend adding "muscular, abs, ribcage" to the negative prompt if generating her naked, as otherwise it tends to generate some very prominent anatomy that I don't think aligns with the original style.

Known issues:
-The outfits aren't always completely accurate (it generates her wearing a tie sometimes, which would have come from Nick being in some of the screencaps. Adding captions should remedy this.)

-Face can get VERY wonky from certain angles or if too far away, though the LoRA was trained with a good mix of close up and medium shots.