PonyDiffusion_V6XL inpainting

IMPORTANT INFORMATION I FORGOT TO ADD

AMD Users have to have the arg --no-half

Here’s how you set the COMMANDLINE_ARGS environment variable across different operating systems.

Windows (Command Prompt or Batch script):

set COMMANDLINE_ARGS=--no-half

Windows (PowerShell):
$env:COMMANDLINE_ARGS = "--no-half"

Linux/macOS (Bash or Zsh):

export COMMANDLINE_ARGS=--no-half

Other Unix-like OS (BSD, etc.):

export COMMANDLINE_ARGS=--no-half

(Less Common)

Example for Fish shell:

set -x COMMANDLINE_ARGS --no-half

FOR THE BEST RESULTS USE THE SAMPLER EULER A

The samples are from Forge/Auto1111 img2img though i did most of my testing using openOutpaint because i prefer it over the default plus outpainting is so much easier but i chose to upload the img2img samples because openOutpaint doesn't hold the metadata in the images you generate and just want to be as transparent as possible, whether you use img2img or openOutpaint or even comfyui (although not tested on,i don't see any reason it wouldn't work)..

One thing i forgot to add for those that don't know, you should have your Inpainting conditioning mask strength set to 0.99 which you can do by going to settings and i believe it's in the img2img section, mines been moved with the Quicksettings list... So if you use openOutpaint like i do anytime it brings up that screen to change it to 1 just click no.

This took me over 2 months of trial and error with so many merges and combinations trying to get a ponydiffusion inpainting model because as I'm sure for the ones that have tried to create a ponydiffusion inpainting model the normal way, it will generate images but its basically not usable. It was trained on sdxl so idk why it's the only model that does this and It's not just ponydiffusion but any fine-tuned ponydiffusion model has the same problem but i came across two fine-tuned pd models called

luminaverse

and

T-ponynai3 (V3.5)

that was very very close to the default ponydiffusion model but the problem wasn't nearly as bad but still unusable as it was so these were the steps that i took if you want to for whatever reason recreate it yourself. you'll also need

4thTailHentai_03

which is another fine-tuned ponydiffusion model and lastly
AnieaseXL_V2.0

which is the only model that's not trained on ponydiffusion but i used it because its a pretty good model overall with characters it already knows but mostly for the way the inpainting model's color blends into your image its very clean and seamless

• This step idk if its mandatory but I'll list it, one test i did was i used the Model Toolkit Extension loaded the normal ponydiffusion_v6 model clicked on the advanced tab and then in the import i selected the t-ponynai3_v3.5 model and selected UNET-XL and clicked save and used this model in the merge i mention below (when i reference using the t-ponynai3 model, it was technically the ponydiffusion_v6XL model with a swapped UNET-XL)

so using the checkpoint merger in Forge/Auto1111 i started by creating the first inpainting model setting Add difference with a strength of 1 checking Save as float16 this will prune the model and create an fp16 model (i tried baking in a vae and not baking one in and since i use a normal sdxl vae by default i didn't really notice a difference)
for
model A = sdxl.inpainting
model B = luminaverse
model C = sdxl base model (specifically sdxl.09vae)
1. luminaverse.inpainting
Now that you have that you'll need to create an aniease inpainting model using the same method above.
2. Aniease.inpainting

then you'll need the 4thTailHentail_03 model but this one doesn't need an inpainting model (i tried creating the inpaint model and the results were just as bad)
3. 4thTailHentail_03

so first i took the luminaverse.inpainting model and the aniease.inpainting model and merged them together using the checkpoint model merger
Model A = luminaverse.inpainting
Model B = aniease.inpainting
Weighted Sum of 0.3

which resulted in luminaverse-x-aniease.inpainting model
then to add some of the pony back and also further help the color problem because the colors for the normal models are fine it's only after you create an inpainting model with them that it ends up the way that it does and i have no idea why...
but i took luminaverse-x-aniease.inpainting and used checkpoint model merger again
Model A = luminaverse-x-aniease.inpainting
Model B = 4thTailHentai_03
Weighted Sum of 0.2

luminaverse-x-aniease-x-4thTailhentai_03.inpainting
The results after this was alright, color was close to normal but it seemed to effect the inpainting model a little too much so i then used checkpoint model merger again

Model A = luminaverse-x-aniease-x-4thTailhentai_03.inpainting
Model B = T-Ponynai3_V3.5.inpainting
Weight Sum = 0.5
luminaverse-x-aniease-x-4thTailhentai_03-x-T-Ponynai3_V3.5.inpainting

so just a 50/50 split between the two which fixed the inpainting side of things but brought back the color issues so this is when i dropped checkpoint model merger and used the Model Mixer Extension

and im only listing the successful merge receipts if i listed every single merge i tried i would have to write a book on the process and no one wants that. and i would just like to stress im not an expert or even knowledgeable on Block Weights or the different things like Normal,Cosine,Inv.Cosine ect. all the tests that lead to this next merge were all trial and error and over 2 months of using different merging techniques and i almost gave up but thought i might try something i still don't fully understand, not even a little bit actually which is the DARE option that the extension model mixer recently added. and i also played around with the Adjust settings that it gives you which look like
i also used the elemental merge which was also just trial and error im sure someone that actually knows about what these functions do can make a way better inpainting model, one were the artists styles are close to the same because they do still work but are just weaker as i'll show in my xyz plot.. that's just the default settings I'll just post the settings that were in the Metadata but if you just wanted to copy the settings over this part specifically is the "ModelMixer adjust" part which i bolded the important parts but what worked was
Model A = luminaverse-x-aniease-x-4thTailhentai_03-x-T-Ponynai3_V3.5.inpainting (i had a different name for it but it's the same as the Final-TestPonydiffusion_V6XLMix.inpainting model)
Model B = 4thTailHentaiModel_03.safetensors
ModelMixer merge mode b: DARE
ModelMixer alpha b: 0.5
ModelMixer mbw mode b: True

ModelMixer use elemental b: True

ModelMixer calcmode b: Simple Cosine

ModelMixer mbw b: ALL

ModelMixer mbw weights b: "0,0.125,0.15,0.125,0.15,0.2,0.25,0.2,0.15,0.2,0.25,0.125,0.15,0.125,0.15,0.2,0.25,0.2,0.15,0.2"

ModelMixer elemental b: "IN00-IN03 OUT06-OUT08:skip_connection out_layers op in_layers emb_layers:0.25"

ModelMixer model a: Final-TestPonydiffusion_V6XLMix.inpainting.safetensors [9fefd616ba], ModelMixer max models: 3, ModelMixer adjust: "0.05,0.05,0.15,0.1,0.025,-0.01,0.025,-0.15", ModelMixer use model b: True, ModelMixer model b: 4thTailHentaiModel_03.safetensors [a1f96244d3], ModelMixer merge mode b: DARE, ModelMixer alpha b: 0.5, ModelMixer mbw mode b: True, ModelMixer use elemental b: True, ModelMixer calcmode b: Simple Cosine, ModelMixer mbw b: ALL, ModelMixer mbw weights b: "0,0.125,0.15,0.125,0.15,0.2,0.25,0.2,0.15,0.2,0.25,0.125,0.15,0.125,0.15,0.2,0.25,0.2,0.15,0.2", ModelMixer elemental b: "IN00-IN03 OUT06-OUT08:skip_connection out_layers op in_layers emb_layers:0.25", ModelMixer use model c: False, ModelMixer use model d: False

After this merge i ended up with Mix100+Test4FixPonydiffusionXL.inpainting.safetensors

But still not done yet, i had to be sure it was basically still the pony model which has all the artist styles (over 750+ I'll post below) and we all know pony is a perfect nsfw model so that was the first test i did using different artists tags and I'm actually still shocked that for the most part work. I can't say they all work as i couldn't test all of them it would have taken to long, all the ones i did test worked so i figured It's good enough.

Last step - used Supermerger extension to merge 4thTailHentai's Lora Extract with the inpainting model which seemed to really help bring out some of the artist styles that were really hard to tell if they had any effect or not

THIS MODEL IS NOT USED FOR GENERATING IMAGES, FOR THAT USE THE NORMAL PONYDIFFUSION_V6XL MODEL

Sorry I forgot the artist list and i tried to upload the txt file but don't even know if you can on civitai so sorry in advance for having to scroll so much

• Steps to Format this list is after it at the bottom. It's only 2 steps using Notepad++ Which I'll Screenshot.

matsunaga_kouyou m-da_s-tarou cutesexyrobutts gweda fumihiko_(fu_mihi_ko) kantoku cle_masahiro kame_(kamepan44231) homare_(fool's_art) 92m personal_ami sciamano240 kuro_kosyou bigrbear samip wisespeak asutora hayabusa fujima_takuya varniskarnis 40hara seraziel kairunoburogu nottytiffy asanagi vahn_yourdoom kase_daiki cluseller iwbitu themaestronoob syhan niliu_chahui ibuki_notsu tama_(tamago) ke-ta scottie_(phantom2) ricegnat ishikei as109 abmayo lvl_(sentrythe2310) mignon chita_(ketchup) nekoda_(maoda) aki99 tab_head mashuu_(neko_no_oyashiro) ndgd doitsuken6 chahan_(fried_rice0614) kukie-nyan beijuu yoako harris_hero missile228 nagishiro_mito mao_lian_(nekokao) puuzaki_puuna yoshizawa_tsubaki yukito_(dreamrider) hu_dako omone_hokoma_agm kfr alp sabamen deaver ouma_tokiichi ruu_(tksymkw) bluefield sukemyon tachimi_(basue) kisaragi_nana sakuraba_hikaru_(loveindog) casino_(casinoep) unfairr null_(nyanpyoun) mochirong koyuki_(kotatsu358) sasaame pottsness miyase_mahiro ergot cowfee re:shimashima shpo tony_guisado roropull everyntrge yuano hater_(hatater) monobe_yuri takunomi z.taiga jeneral camonome nekoya_(liu) sayori_(neko_works) aoi_ogata kaedeko_(kaedelic) himitsu_(hi_mi_tsu_2) misaka_12003-gou neko_neko_koneko ratatatat74 afrobull rokugou_daisuke masami_chie huke ushiyama_ame mathias_leth sakura_chiyo_(konachi000) cirilla_lin xiujia_yihuizi umakuchi_shouyu deel_(rkeg) masabodo chanta_(ayatakaoisii) sendo_(sendrawz) saburou_(hgmg) tateha_(marvelous_grace) nanoless sub-res wlop sogaya ge-b ateoyh idaten93 dearmybrothers pizza_(pizzania_company) ringeko-chan poco_(asahi_age) maccha_(mochancc) sigm@ neropaso tarakanovich putama beeeeen koretsuki_azuma nat_the_lich dearonnus naga_u usagihime watao subachi omoomomo horn/wood minaha_(playjoe2005) huyumitsu hoshizuki_(seigetsu) catstudioinc_(punepuni) efe cube85 starraisins muk_(monsieur) munashi_mujou hara_kenshi 3_(sanyako1) allenes danimaru greatodoggo nibiiro_shizuka kuria_(clear_trip_second) yaman nakamura_rohane mushi024 monaka_curl kanachirou nashidrop nokoru_sora suerte turewindwalker miko_(royal_milk) oopartz_yang ehrrr hinata_(user_rjkt4745) kanpa_(campagne_9) merrytail poligon_(046) nikorashi-ka shishi_juuroku genkung yuzuriha_(active_homing) chups mathew_(srmmk_mce) yukineko1018 shiratsuki_shiori zankuro lds_(dstwins97) akai_sashimi fal_maro skywalker0610 arpeggio_kaga jovejun nekoyashiki_pushio tidsean ogre_(illustogre) lunch_(shin_new) nopetroto songchuan_li yuuji_(yukimimi) mamuru hammer_(sunset_beach) kink_(tortoiseshell) inre_kemomimi shirokitsune merunyaa cenangam asou_(asabu202) rizento ddolbang kana616 hplay meion ohako_(ohako1818) midori_(misuriru8) observerz sheya nanashin_naomi mikozin raptor7 pinkcap kazami_karasu masanaga_(tsukasa) ebonyxh youmu-kun kanon_(kurogane_knights) onono_imoko aramaru namagome_negi welt_(kinsei_koutenkyoku) gustav_(telomere_na) ryota_tentei nito_(nshtntr) ooji_cha darumoon kanta_(kanta_077) chilakkk hotel01 tandohark kinhasu gijang nanana_(nanana_iz) raki_kr arinu mikisai ru_zhai liu_liaoliao oxxo_(dlengur) ao_oni_(onioni-aoi) satou_(3366_s) amashiro_natsuki iroha_(iroha_matsurika) ajifurai e.o. yanagi_yuu pixel_(yuxian) suminagashi ejami kanopan kaigen_1025 ibaraki_natou horokusa_(korai) ojay_tkym radioneet sonri megateru tksand joshua_(shisanli934) shone kokihanada rukako sp123 mafuyu_(chibi21) sy4 shiro_youduki akeyama_kitsune roke_(taikodon) rustle kird_a tanabe_(fueisei) da_mao_banlangen aos etsunami_kumita kamiyoshi_rika nokoppa kibisake zooanime fiz-rot suraimu_(suraimuraimu) kawakami_rokkaku liya itomugi-kun elsi kyou_039 born-to-die massakasama ayaoshiro rushian piyodesu momoko_(momopoco) kiriko_(araragikoyomi) hospital_king suzuno_(bookshelf) distr beudelb yua_(checkmate) mildt bartolomeobari neko_lu_(wofud453) senmura syuri22 syope krirk ayakura_juu pandarou mirin_chikuwa uechin_ewokaku mata_(matasoup) liclac mauve iesupa yan_pai amagi_(amagi626) fusu_(a95101221) kurousagi_tetora gusha_s sasaki_(glass1138) hiyashi_mirano rotte_(1109) sakuramochi_(sakura_frappe) garuku mamemochi kou_mashiro koume_keito sakurato_remi ayahi_4 robot_cat_(robotcatart) dan-98 nama_udon tyakomes enkyo_yuuichirou fh_moya akino_ell ke-su zunusama kamukamu_(ars) peso_(cheese_company) dio_nand terasu_mc damda gyouza_(mhea5724) mabing hanamoto410 miyo_(ranthath) ddddecade kutan toi1et_paper mizuga spam_(spamham4506) silver_(chenwen) shundou_heishirou arim0k0 nuu_(nu-nyu) doribae binggong_asylum kawayabug kasuya_baian tenmu_shinryuusai shika_miso hira_(nanika_no_heya) ransusan satsuki_mayuri kagiyama_shachou masuyama_ryou nakkar juejue syroh ma_draws shirosato laserflip deku_suke yume_shokunin muku_(muku-coffee) moda_(mo_da_3) sao_satoru aohashi_ame paraffin hatoyama_itsuru shikito mintoaisu tsune_(tune) paintrfiend maboroshi_mochi letdie1414 hagure_kedama nero_augustus daidai_ookami momozu_komamochi ini_(inunabe00) ribiadan trimbil kajimakoto xe_(execut3r) auguste primsla hongbaise_raw yana_mori kani_bonara oppaiserothicc gokuu_(acoloredpencil) ganesagi deetamu totoharu_(kujirai_minato) keita_naruzawa tohoyuukarin gaothun komaku_juushoku sincos inyuppo daizu_(melon-lemon) makuran kokuto_(kurousagi1210) hisona_(suaritesumi) gawako kuro_suto_sukii yokaze_(yokajie) kentllaall faustsketcher mirufui ramudia_(lamyun) haseru_(ginku_mh) movik dikko kaptivate hidebo akitsuki_karasu fufumi morino_hon minakami_(flyingman555) furukawa_wanosuke dombear melaton shiraue_yuu muuran mizuki_(kogetsu-tei) less ringouulu ayamy dokomon kochiya_(gothope) slugbox zerocat gins bee_(deadflow) arind_yudha aogisa mayogii tony_taka e_sdss misaki_yuu chirang matsuoka_(mtok_0) otokuyou suzuiro_(chocolatte) fieryonion xiaosamiao himajin_noizu paryi zap-nik luren_max colon_br miike_(992058) kaamin_(mariarose753) eddybird55555 yuccoshi kuya_(hey36253625) harmonist11 melon22 funa_(sakana) na_tarapisu153 ren_(gh) npcpepper shitacemayo tokiani jikuno shino_duka roki_(hirokix) yabby togekk0 fuuzasa tomatolover16 second_heaven sumishi_(sumisi_3) coreytaiyo pumpkinspicelatte k0ng club3 hecha_(01964237) wolflong kazana_(sakuto) nagomian 7saki kutsuki_kai kamehito togemaru34 shounen_(hogehoge) taisa_(kari) min_(120716) timitarcat sakura_ani akuma_(st.takuma) asanogawa_(tutufcc) sarasadou_dan ameto_yuki dokonjou_(odossan) itou_yuuji mcgunngu haryuu_(poetto) shuuko_(s_h_uuko) mitake_eil jun_(aousa0328) kakaricho_dairi yom_(ymayma00ss) kanijiru falken_(yutozin) ougiikun tofuubear yagoro_kusuriya whoru nyong_nyong ebifurya sasaoka_gungu tostantan chamaruk spacezin kazuki_seihou floofsmear momendoufu chiroshiron beleven nori_tamago rottenweiler calm_mashiro nejime poronegi yoshida_hideyuki optionaltypo tanaka_kusao shimada_fumikane koizumo konata_gazel chii_(tsumami_tsumamare) gurajio hukahire0120 ikeda_ruriko chikado nuko_miruku kitazinger supershrimpcakes mozukuzu_(manukedori) hano_haruka suzune_hapinesu techi_(techi35499) moyoron daichi_(tokoya) mochizuki_riku barbariank saeki_touma bori_(3910234) sakuma_hiragi tonchamon_san arnest peroponesosu. tochinoko 258n susukitten chima_q mirukurim ghost_(tama) tamahana ken_(1057168252) mazume hira-san shio_(shiofeifei) keiki8296 akisome_hatsuka shibori_kasu iganashi1 b_nosk101 fir3born azuki_osamitsu ichirugi maru_(maru1625) meng_ziya yamauchi_(conan-comy) shinmon_akika ni_(221) kkato lemtun yatanukikey kuri_dora harunoha shironeko_yuuki miura-n315 oeyama wada_arco kinkymation someyaya toutenkou mokoiscat kurotofu kolshica red_medicine mikan_(manmarumikan) mirufuaa haruwaka_064 niliu_chahuiobaoba_(monkeyix) risui_(suzu_rks) hamico torakichi_888 rariatto_(ganguri) roshin molyb hihiyama_yokikana katsushika_pachi guo582 dodok_(gj77230) sh_(shinh) rimukoro ryuuichi_(f_dragon) soranamae hjz_(artemi) fukahire_(ruinon) sinkai sifeizui sabaku_chitai done_(donezumi) gunnjou_yosio u_u_zan mame_komari ramie_(ramie541) mo_ne rank_alee wing228 meta_meta le_(huanglongen) maturiuta_sorato top-exerou gnai tabayashi hase_neet s16xue steed_(steed_enterprise) hijikawa_arashi shiratama_(hockey) wowoguni kemo_chiharu monaka_(gatinemiku) coldcat. akaoni_(zumt3548) nyantcha kyabekko sawaya_(mizukazu) neko_no_youchuu aoshima naguru_(cyoroama) dog-san yonago_miko daisy_cutter himanemuitoma y2 sznkrs dfra rtil 0_(znanimo) yuuppi shinonome_asu rylaera it_po_(sss_cmy) sunhyun foxykuro shikido_(khf) kujira_pasta micon n0r0 kakult2017 usaka_ray azure-maya kame_(kamepan44231) setakman drlee_lili kei_kei suzume_(simple0091) lily0428 fumei_(mugendai) 360_(taiyo360) ten_no_hoshi kurokan_(kokkyou_oudan) 54hao garasuno k_mugura mikoma_sanagi otomeza_ryuseigun xi_ying fenrir_(fenriluuu) kyogoku-uru yuuki_mix zhazhapao carbohydrate_(asta4282) op_na_yarou kyuu_(plastic_night_q) yunmi_0527 kajuu ichikushi_mojibake ebi_193 amazuyu_tatsuki yukinoshiro onkn_sxkn umyonge_(lkakeu) ya_kexi agahari alexi_oekaki lingshalan yukishiro_haku fujiko_(emnprmn) binware reijing_etrn nanome_to mitama_mudimudi ohno cactus41747280 monety sekiyu_(spartan) chis_(js60216) sakurasaka gomeifuku felutiahime qinglai_haiji tomyoo jytniliu_chahui namiki_(remiter00) pudding_(skymint_028) chanifge sthk iop5509 seika_okawari zounose bbc-chan nanashi_(nlo) foxyreine kagami_toufu matsuda_(matsukichi) sarukana rikuguma you_(noanoamoemoe) cine_(wazd0183) hitobashira_(1746164306) tetsurou_(fe+) itsumizu doraeshi bodhi_wushushenghua 463_jun amisu orchid_(orukido) dior-zi kaleka kaguura_(kagu) sumiyao_(amam) xing kuraki chintora0201 sora_mame_(princess_0sora) wlper dishwasher1910 umberblack izumi_sai mizore_arius vamjulian mikan_(chipstar182) k@bu cerevevisiae hyouta_(yoneya) nephlite ootsuki_wataru houkai_(collapse_illust) g_(genesis1556) tama_(soon32281) zuo_daoxing natsuba002 hews fumitsuki_(minaduki_6) fujisaki_hikari bannouyakunta if_f minazuki_mizu renxzd amino_dopple shironekoban kanzakietc parijennu222

copy the entire list, open up notepad++ (if you don't have it, i recommend getting it. It's free and can be very useful).
After opening Notepad++ paste the entire text and then go to Search tab and click on replace.
For Your Settings

Find What**:**

Replace With**:\n**

(it might look empty but you have to put one space in the find what for it to work)
(You Don't Have to worry about the Wrap Around, it's not needed but doesn't hurt anything)

select Extended
and then click replace all and that's it.

If anyone has any problems with this or the model itself please leave a comment