ComfyUI Local LLM Qwen Prompt Refiner (Offline LLM Node)

Details

Download Files

About this version

v2.0 - Major performance & multi-GPU update (nightly branch)

Key new features:

- Added torch.compile support (reduce-overhead + dynamic=True) for ~30–100% faster inference after initial compilation warmup

- Explicit SDPA (Scaled Dot-Product Attention) backend for better speed and memory efficiency on Ampere/Ada GPUs

- Multi-GPU support via device_map="auto" – toggleable new input "use_multi_gpu" (default True). Turn off for single-GPU only (e.g.users with only cuda:0 visible)

- Modern dtype options (bf16 default, fp16, fp32, auto)

- Better logging, error handling, and unloading when keep_loaded=False

This version focuses on stability, speed, and compatibility with multi-GPU setups while keeping single-GPU rock-solid.

Repo: link

Tested on RTX 3090 (single GPU) with PyTorch 2.7–2.9.

Feedback/PRs welcome for multi-GPU testing!

Model description

Local Qwen LLM loader & thinking prompt refiner for ComfyUI

Load any Qwen model (like Qwen3-4B-Thinking-2507) completely offline using safetensors. Perfect for refining Stable Diffusion prompts with full control: customizable instructions, visible chain-of-thought output, fixed seed for consistent results, and optional full memory unloading to free VRAM after use.

No API, no rate limits, no OOM headaches when running alongside heavy generation workflows.

Just drop the model files in models/qwen/model-main-folder-goes-here/repo-files-goes-here, restart ComfyUI, and start crafting better prompts locally.

location to place the model:
what goes inside of the subfolders:

Custom-node repo is here

Images made by this model

Sort by

No Images Found.

Model Type	Workflows
Base Model	Other
Published	1/11/2026