Fdetailer - Adetailer for Furries

Version Differences

v1 and v1.1 are very similar. It's hard to say which is better, so try them both and decide for yourself. v1.1 was trained in an attempt to correct some minor issues, and while it has shown improvements over v1, it has also shown some regression. I think I slightly prefer the accuracy of v1.1, so if you only wanted to try one, I'd recommend that one.

What is this?

Fdetailer is an instance segmentation model trained on furry anatomy using yolo as a base model. The dataset is ~20k images with a variety of species and art styles, including both anthro and feral characters. The model is capable of detecting and masking faces, penises, pussies, anuses, sheaths and pawpads. It may struggle a bit with uncommon species or strange anatomy, but overall the accuracy has impressed me for being my first attempt at training a segmentation model.

Fdetailer is an all-in-one multi-class model. You can choose to only detect faces and pawpads, anuses only, or any combination of the classes available. Some interfaces may not support this, however. Multi-class models are relatively uncommon.

When given an image, Fdetailer attempts to find and mask objects it has been trained to detect. Instance segmentation is a bit different compared to bbox (bounding box) models that have been popular in the past. Bbox models find an object and draw a box around the object. This can be problematic for inpainting since the area around the detected object is also subject to inpainting and can create ugly seams and strange details outside of the intended object.

Instance segmentation attempts to solve this by creating point-to-point masks around the objects it detects. These masks are similar to ones you might create by manually painting over a character's face. This solves the issue of ugly seams or inpainting parts of the image outside of the face.

Who is this for?

Me. I created Fdetailer because inpainting is annoying. I'm a ComfyUI simp and I wanted to get more detail out my base images before I use other software to edit them. I've found every image I've generated could always use inpainting on faces and genitals, but I can't be arsed to manually inpaint every image before I decide if I even like it or not.

If you're the type of person that queues up a bunch of generations and then goes through them later to cherry-pick your favorites, you'll find this useful. You can automate the inpainting process of most of the common (in my opinion) parts of your character(s) you'd want to inpaint.

If you're the type of person to watch each image generate step by step, you may still find this useful. Why interrupt your gooning sesh to inpaint a butthole when tools exist to do it for you? I got you, homie.

Does it actually work?

Mostly. I'm just some dumbass pretending to be a pink cat on the internet. This is my first attempt at creating a segmentation model, and plenty of problems have come up during the process. I've attempted to fix most issues as they pop up, but my knowledge and time is limited, so this is an ongoing project.

Accuracy is okay. Pawpads are the worst offender due to how different they can appear depending on art style and species. Anuses can also be an issue, but in most scenarios accuracy is fine. Sheaths can be hit or miss due to lack of data, but I haven't had too many issues. Everything else is pretty damn good. I'm overall impressed with the current state of the model.

Uncommon species may confuse the model depending on how niche they are, but in most cases it works well enough.

How do I use it?

Reference your preferred UI's documentation for more accurate instructions. Your UI might give you the option to filter detections based on class id or class name. This is useful if you only want to detect faces, for example. The following list contains the class ids and names the model has been trained on:

0: face
1: penis
2: pussy
3: anus
4: sheath
5: pawpads

For confidence threshold, I recommend between 0.35-0.5.

ComfyUI: I recommend using the comfyui-impact-pack (you'll also need comfyui-impact-subpack for ultralytics support) set of custom nodes. Place the model in models/ultralytics/segm. Load the model using UltralyticsDetectonProvider and then use SEGM Detector (SEGS) to run inference, filter detections by class name, modify confidence threshold and change mask settings. Pipe the SEGS into Detailer (SEGS) to inpaint. You can use Impact's built-in wildcard processing to modify the prompt based on what is detected. For example, you can add detailed eyes to all face detections. Refer to the Impact wildcard documentation for all the things you can do.

SwarmUI: Documentation. Place the model in (swarm)/Models/yolov8. Add <segment:fdetailer_seg_v11.pt,denoise,threshold> to your prompt. Change denoise to your desired denoising strength and threshold to your desired confidence threshold. This will inpaint all objects detected in the image.

You can filter what you want detected by adding the class ids or names in between colons after the model name. For example, if you only want to detect faces and pawpads, you can add <segment:fdetailer_seg_v11.pt:face,pawpads:,denoise,threshold> to your prompt.

A1111 and Forge variants: You'll need the Adetailer extension. Place the model in models/adetailer. The Adetailer extension does not support class filtering. All classes detected will be inpainted. This cannot be changed until the Adetailer extension has been updated or forked with support for class filtering added. Additionally, the default settings are geared more for bbox masks. Since this model produces segmentation masks, I highly recommend increasing "inpaint only masked padding pixels" to 64 or higher.

Refer to the Adetailer documentation for further instructions because neither I nor my friends use a1111 or its variants.

Future Plans

Accuracy: I'm mostly happy with the current state of the model, but it could be better. I'm planning on doing another tuning pass over the current dataset to improve the accuracy of the masks and retraining the model to use a larger parameter version of yolo.

More Classes: With a larger parameter model, I'm more comfortable adding additional classes without the fear of it diluting the accuracy of other classes. I'm open to suggestions, but my current plan includes adding eyes, mouths, balls and nipples. I'm also considering adding a penetration class to mask both the object and orifice that's being penetrated, but I'm just dreamin' here.

HANDS?: Yes, but not anytime soon unfortunately. Hands are complicated and I'd need to do a bit of research on how best to tackle them.