Upload 4 Style References — Keep the Face, Change the Vibe

Description: This ComfyUI workflow creates highly customizable, stylized portraits by combining IPAdapterFaceID for accurate facial preservation, IPAdapterAdvanced for flexible style transfer based on user-uploaded reference images, and ControlNet with lineart preprocessing for clean, structured outputs. Transform a single input portrait (e.g., marie.jpg) into a unique artwork in any style you choose—cartoon, realism, abstract, or more—depending on the reference images you provide. Perfect for creating personalized posters, avatars, or artistic illustrations.

What It Does:

• Face Preservation: Utilizes IPAdapterFaceID with InsightFace to retain the facial identity of the input image, ensuring the generated portrait closely resembles the original face.

• Custom Style Transfer: Applies a user-defined style using IPAdapterAdvanced, leveraging up to four reference images (batched via ImageBatch) to shape the aesthetic. Swap the default images for any style you want—cartoon, oil painting, cyberpunk, or anything else!

• Lineart Control: Employs ControlNet (controlnet-union-sdxl-1.0 with lineart preprocessing) to guide the structure of the output, ensuring clean lines and a cohesive look.

• Image Preprocessing: Processes the input image with BiRefNetUltraV2 to remove backgrounds and ImageDesaturate+ to optimize it for style transfer, ensuring compatibility with your chosen aesthetic.

• Prompt Guidance: Includes a customizable positive prompt (e.g., "theatre poster with the text 'Varsailles' on top") and a negative prompt (e.g., "blurry, low quality, distorted faces") to refine the output.

• Model and LoRA: Built on the leosamsHelloworldXL_helloworldXL70 SDXL checkpoint with the araminta_k_midsommar_cartoon LoRA, which can be adjusted to suit your preferred style.

Key Features:

• Flexible Styling: Style is determined by your reference images, allowing endless possibilities—cartoon, realism, fantasy, or any visual aesthetic you provide.

• Modular Design: Organized into groups (e.g., "Image Preprocessing," "ControlNet," "KSampler") for easy customization and debugging.

• High-Quality Output: Generates 512x512 images (adjustable via EmptyLatentImage) with 17 sampling steps and a CFG scale of 6.3 for balanced detail and adherence to prompts.

• Preview Nodes: Includes PreviewImage nodes to inspect intermediate outputs (e.g., preprocessed image, face detection, lineart) for fine-tuning.

• Background Removal: Automatically removes the background of the input image using BiRefNetUltraV2 for a clean, focused portrait.

How to Use:

1. Input Image: Load a clear portrait image (e.g., marie.jpg) into the LoadImage node (ID 132). Ensure the face is prominent for optimal FaceID results.

2. Style References: Replace the default reference images (aether-illustration-new-nordic-style-*.webp) in LoadImage nodes (IDs 167, 168, 169, 170) with your own images to define the desired style (e.g., anime, watercolor, sci-fi).

3. Prompts: Update the positive prompt in CLIPTextEncode (ID 96) to describe your vision (e.g., "futuristic portrait, vibrant colors, detailed background"). The negative prompt (ID 343) is pre-set to avoid artifacts like "blurry, low quality, distorted faces."

4. Adjust Parameters: Fine-tune KSampler (ID 6) settings (steps, CFG scale) or IPAdapterFaceID/IPAdapterAdvanced weights to balance face preservation and style influence.

5. Run and Preview: Execute the workflow and check outputs in PreviewImage (ID 212) or save them via SaveImage (ID 279).

Requirements:

• ComfyUI Plugins: Requires comfyui_ipadapter_plus, comfyui-art-venture, ComfyUI_LayerStyle_Advance, comfyui_essentials, and comfyui-advanced-controlnet.

• Models:

  • SDXL checkpoint: leosamsHelloworldXL_helloworldXL70.safetensors

  • LoRA: araminta_k_midsommar_cartoon.safetensors (optional, replace with a LoRA matching your style)

  • IPAdapter: ip-adapter-faceid_sdxl.bin

  • CLIP Vision: CLIP-ViT-H-14-laion2B-s32B-b79K.safetensors and CLIP-ViT-bigG-14-laion2B-39B-b160k.safetensors

  • ControlNet: controlnet-union-sdxl-1.0/diffusion_pytorch_model_promax.safetensors

  • BiRefNet: BiRefNet-general-epoch_244.pth

• Input: A high-quality portrait image (ideally a single face) and 1–4 style reference images.

Tips for Best Results:

• Use a clear, well-lit portrait for the input image to ensure accurate face detection by InsightFace.

• Choose reference images with a consistent style to strengthen the style transfer effect in IPAdapterAdvanced (ID 356).

• Adjust weight and weight_faceid in IPAdapterFaceID (ID 347) to balance face preservation and style (e.g., 0.8–1.2).

• Experiment with ControlNet strength (ID 164, currently 0.75) to control lineart influence (try 0.5–0.9).

• If the output is too noisy, increase steps in KSampler (ID 6) to 20–30.