Lune - flow matching - sd15-Flux
Details
Download Files
About this version
Model description
sd15-lune-flux-v01
This is the first iteration of a bulk flux-schnell extracted images used for baseline, and then finetuning atop of those with about 40k portrait images specific to higher-quality eyes, faces, hairstyles, and so on.
The portraits were targeted for timesteps 0-500 to allow additional quality to cut through for the later generation timesteps.
The system is trained primarily with plain English, so speak to her and she might just make what you want.
You NO LONGER require latent multiplication on output.
This is still required, and is best set to 2.50 instead of 2.0 for this version.
Have fun. She's growing in power rapidly.
Diary of a mad scientist
Her early iteration is ready for toying with. I advise using res4lyfe in comfyui.
This is most definitely flow matching sd15 - a large percentage of sd15's core model distilled into flow matching.
https://huggingface.co/datasets/AbstractPhil/sd15-latent-distillation-500k
You can play with the model here if you don't have the hardware, but be warned ZeroGPU is limited.
https://huggingface.co/spaces/AbstractPhil/sd15-flow-matching-lune
This playground includes both Lune and the prototype VAE named Lyra.
Every single prompt used to train this model is compounded into lists attached to the repos where the checkpoints are stored. You can see a direct system related to laion flavors and synthetic data from A to B.
Licensed MIT - a cite would be nice but not required.
The most recent finetune is here;
and it does not require latent multiplication.
Using This Model
https://huggingface.co/AbstractPhil/sd15-flow-lune
https://huggingface.co/AbstractPhil/sd15-flow-lune-flux
All the PT and safetensor checkpoints are hosted here. I'll be uploading 52000 to civit prior to the flux-schnell finetune.
You will want to multiply the output latents FROM the ksampler. If it's too bright or looks pixelated, reduce the latents a bit.
For sd15-flow-lune you will need this. For flux you will not need this.
You will also want sd15's shift - trained with 2.0 and 2.5. It will respond to gradients between both.
Will work with normal ksampler without res4lyfe too. Just make sure the shift runs from the core model to the shift, and then the shift to the sampler's model input.
Lune's History
Lune is the offspring of prototype twins. I distilled the two models using a collective based on classification through collective census. This collective I dubbed Geofractal-David Collective. This collective's entire purpose is to watch each block of sd15 to learn patterns and timesteps.
David
David went from a prototype to a fully working model capable of many tasks. David can classify seemingly any form of data in collectives - assuming the data CAN be classified, and David can accumulate his own learnings of how to classify those behaviors due to the methodology behind the classification heads and the processes of geofractal behavioral response classification.
100 entries and buckets for timesteps, 10 dim entries and buckets for patterns. All shared space - was how I trained these models.
Each layer was latched directly to sd15's layers as it was frozen and issued prompt request, after prompt request, after prompt request. Hundreds of thousands of requests later with huge batch sizes - and David was capable at classification with a high enough accuracy to begin the first experiment.
https://huggingface.co/AbstractPhil/sd15-flow-matching
Flow Matching Fumbles
The journey wasn't an easy one. The model went through multiple phases of seemingly good response to completely geometric flat responses. The system learned, and yet did not seem to understand details. The patterns specific to those details were never targeted and I learned why post-train, but during the training I pushed through until it was to epoch 50.
During training there were multiple faults with the trainer that caused resets. Multiple problems with the connections that caused overflows or failures. Multiple small tidbits like failing to upload the pts and almost wasting 16+ hours of training time. %debug was learned FAST to fix this one using colab.
I never gave up though. I refused to let this model die.
Flow Matching Try 2
The first version seemed to be failing, so I began another to test the hypothesis. The second version was using a new format of timestep training with weighted buckets - ensuring less accurate buckets weren't used, and the buckets that met a certain threshold were instead treated like difficulty targets.
The train was more successful with geometric structure but the patterns never held. The patterns themselves, I thought, weren't deep enough. However, further training showed that I was incorrect. The patterns held, and I was assessing the incorrect valuations. The incorrect methodology for assessment. The incorrect format of geometry.
This version, became Lune. Cooked to about epoch 28 before I halted - while the sister cooked to epoch 50 instead.
Renewed Determination
I began cooking the big set, which ended up being around 400k 512x512 latents extracted directly from the output of sd15. These are a combination of scaled and unscaled.
https://huggingface.co/datasets/AbstractPhil/sd15-latent-distillation-500k
This is a dataset of basically MODEL POISON. However, it's more than enough to test the hypothesis. sd15-flow-matching-try2 DID learn the shapes, the geometry, and the fractals saved the space. However, it did not learn the global meaning of those without it's adjacent pattern-driven output structure.
It did not retain the globalized order as expected, and that is what caused it to fail. The flow matching interpolation was a superb success - and was simply missing a piece of important information.
I ran it, and it cooked. This first release IS THIS VERSION. You are seeing the outcome. It requires two specific configurations to function.
The Breakthrough - Cantor Fractals
During post-assessment I deemed certain elements invalid, and certain elements useful within David's structure for full assessment. I then attempted to attach global attention to a David collective to pure failure - it OOM'd even on small collectives. The global attention could not hold, and thus I had to find a new solution. A better solution.
VAE Lyra
Cantor Attention - a mechanism I designed for global attention, has created the Lyra VAE. A truly magnificent and breathtaking invention that allows the cohesive fusion of T5-Base and CLIP_L features - to directly encode a meaningful differentiation into a CLIP_L feature without completely destroying the representative understanding of that feature.
You can also experiment with Lyra in the huggingface playground, but this is not a VAE in the sense of traditional "PLUG AND PLAY" vae's. You need a special set of nodes for ComfyUI which is available in my repo here with some baggage.
MMAE Beatrix
This is is a cantor regulated multi-modal auto-encoder prototype in her early stages that will;
She will accept multiple clip features from multiple scales at the same time.
She will accept multiple variations of encoder simultaneously - such as the T5, Bert, Lyra, and essentially anything finetuned and snapped onto her system as a potential access gate.
She will emit deterministic sharded fractal geometric features - having fully merged the learned behavior of all of those structures into a cohesive shape for ingestion downstream.
Behave as a vit for images, circle of fifths encoder for midi, and containment unit for text features.
The entire process will be inversible - meaning the same encodings will be decodable downstream.
This is only possible due to the cantor step process. Without cantor stepping, the flow matching would fail. Without flow matching, the system cannot work in it's current phases.
With the cantor gating and sparse global attention, this will allow cross-contaminated regions of normally sparsely ignored cantor space - to geometrically align from any space in the dimensional vectorization.
The losses will group typed systems together while meshing the text encodings throughout the whole system as tokenized pathways of shared fractal access. These latent access spaces will be highly deterministic and optimized using the most robust and utilitarian systemic accesses I can muster in modern day technology.
This is, shortcuts incarnate. The system I've been building towards, and it's on the radar now. Not a blip, not a confusing experiment, a direct on the radar potential that I can in fact build sooner than later.
I will require grant funding for the full high-end training, but the baseline will be trainable with nothing more than an a100 and some sessions finetuning each portion.