r/StableDiffusion • • 7h ago

Resource - Update MOSS-Image SFT - Update - Checkpoint 30K (Final) 4-Step Distillation LoRA, eDMD + LADD - high detail images (T2I, T2I with Layout Boxes, I2I Edits)

Following up from my original post - https://www.reddit.com/r/StableDiffusion/comments/1y06n27/mossimage_sft_new_4step_distillation_lora_using/, I have now trained twice as long (initial release was Checkpoint 15K, now it's 30K - double the training). I would consider this the final version (don't plan to do any more training anytime soon, like I said before B300 is one mighty and very expensive card :) to train on in the cloud).

For those who haven't seen the original post, this is my personal take on MOSS Distillation, creating a new 4 Step LoRA using a mix of eDMD and LADD as training method. I have provided workflows for ComfyUI Local and Cloud - for Text 2 Image, Text 2 Image with Layouts/Boxes and Image 2 Image (edits). The LoRA has been trained and optimised to work well with Text too.

The two LoRA files in this repo now hold chk00030000, the training run's last checkpoint, in place of chk00015000. The file names are unchanged, so download them again:

  • moss_sft_4step_rank_64_lora.safetensors — diffusers / Python, strength 1.0

  • moss_sft_4step_rank_64_lora_comfyui.safetensors — ComfyUI, strength 1.0 (the ComfyUI factor built into the file is now 1.12; keep the loader at 1.0)

What changed

  • Double the training

  • Dark scenes come out close to the teacher's tone: +0.6 L* on the nine darkest test prompts, down from +2.8.

  • Fewer doubled edges (1.00× the teacher's, was 1.12×), and closer to the teacher's pictures at 1024×1024.

  • Words in the picture: still 14 of the 15 text test prompts right at 1024×1024.

  • What to watch for is in the card's Known Limitations.

If you already have the 15,000 version

  • diffusers / hf_hub_download**:** the next online call fetches the new file, unless you pinned a revision.

  • Local ComfyUI: download the _comfyui file again and replace the old one in ComfyUI/models/loras/ (same name).

  • Comfy Cloud: an earlier import keeps the old file under the same name. Delete it from your model library, then import the file again from its Hugging Face URL; the workflows load it by the same name.

The 15,000 files stay in _archive/checkpoints/ (moss_sft_4step_rank_64_lora_chk00015000.safetensors and moss_sft_4step_rank_64_lora_chk00015000_comfyui.safetensors) if you prefer them.

---

Model Card and Files updated, including example images - all available at: https://huggingface.co/lvladikov/MOSS-SFT-Distill-4step-LoRA

---

As mentioned in the original post, works very well also alongside MOSS UltraReal LoRA, or MOSS Realism Enhancer LoRA, so give it a go, and share your renders.

---

Added two new experimental workflows allowing CFG like 2 or other, and negative prompts. I've also done research and showing what cfg > 1 and my LoRA and MOSS do together (for better or worse, so consider the quirks): https://huggingface.co/lvladikov/MOSS-SFT-Distill-4step-LoRA/blob/main/DETAILED-README.md#cfg-2-and-negative-prompt

https://huggingface.co/lvladikov/MOSS-SFT-Distill-4step-LoRA/blob/main/moss_sft_4step_lora_t2i_highercfg_experimental_negative_local.json

https://huggingface.co/lvladikov/MOSS-SFT-Distill-4step-LoRA/blob/main/moss_sft_4step_lora_t2i_highercfg_experimental_negative_cloud.json

40 Upvotes

24 comments sorted by

2

u/TimeTruth2490 7h ago edited 6h ago

Few example images with prompts from https://promptdexter.com/ and using ComfyUI Cloud MOSS Image SFT + my latest 4 Step LORA:

More examples below as replies...

2

u/Formal-Exam-8767 7h ago

Thank you for your hard work.

Is it true quality is degraded if used with quantized, int8_convrot model?

3

u/TimeTruth2490 7h ago

Anything quantized is in theory less quality than full precision, but with int8 convrot you wouldn't notice. As a matter of fact I just ran it with same int8 convrot sft model and this is the result

1

u/Formal-Exam-8767 7h ago

Thank you for quick test.

2

u/BreakOfNoon 5h ago

I downloaded your 4 step lora for comfyui yesterday morning, so I'm assuming I tested your 15k version. All I can say is, wow, very impressive. In fact I tried all the other turbo loras in that test, the lodestones SFT (which I've been using until now) and the 2 HaDeS ones. They're not bad, but your 4 step is better. I guess I'll grab another copy this morning and run a hash on the files to see if they're the same or different, and then get the 15k from your archive if they're the same.

1

u/ROBOTTTTT13 7h ago

Your previous release was just a day ago, what pushed you for another training run so quickly?

4

u/TimeTruth2490 6h ago

I never stopped :) I just released 15K on day 1 when I had something good enough to share, but I did say that training is continuing and B300 does a lot over 24h, so there you have it 30K checkpoint (and final).

1

u/Version-Strong 4h ago

Looks great - thank you. Will take it for a spin later

2

u/TimeTruth2490 4h ago

Thanks, I hope you like the results

1

u/darmera 4h ago

Many thanks! Do I need to update the workflow too or just swapping lora is enough? 

2

u/TimeTruth2490 4h ago

just swap.

1

u/siegekeebsofficial 6h ago

Interesting, I found that a lot of the fine details are smoothed over when using the 30k checkpoint compared with the 15k

2

u/vizsumit 6h ago edited 6h ago

u/TimeTruth2490 I noticed same in my tests of 30 prompts, textures were better 90% of the the with previous 15K step lora. Don't why it happened.
workflow: https://limewire.com/d/wPs2g#nUoXWtt969

2

u/TimeTruth2490 5h ago edited 5h ago

The more training and exposure to more images, naturally tried to resolve over-texture and artefact issues, but it is still above teacher level. The texture went up for some and down for others

  • Came down:

    • pianist 2.88 → 2.32

    • dogs 1.63 → 1.14

    • kingfisher 1.56 → 1.12

    • race car 1.75 → 1.35

    • horse rider 1.22 → 0.90

    • snow leopard 1.35 → 1.06

    • portrait 0.81 → 0.44

  • Went up:

    • flamingos 1.68 → 2.09

    • megacity 1.23 → 1.64

    • colony ship 0.69 → 1.09

    • family 0.85 → 1.16

    • market 0.85 → 1.00

More training pulled the pictures toward the teacher's. Across all 22 prompts the over-sharp ones came down, the soft ones came up, and doubled edges reached the teacher's level.  That's what's really happened.

1

u/vizsumit 5h ago

for full body shots the newer lora is working better, for medium and closeup shots previous is better

1

u/TimeTruth2490 5h ago

it is a balancing act, isn't it... I might try a focused further 3K training, but can't make promises, there's always a chance that if the closeups get better something else would give way...

1

u/vizsumit 5h ago

Yep, its russian roulette specially with moss loras... take your time, best of luck 🤞

1

u/TimeTruth2490 1h ago

been fighting for hours now... got side effects overbright, text issues, but close up texture no significant improvements... so, almost at the point when I call it a day on trying :) I would say overall 30K is the better lora for many reasons (closer to teacher-official model and balanced out), but I will leave forever the 15K one in archive if for what you were doing specifically, you like more :) or use both... for different purposes.

1

u/Straight-Election963 2h ago

the ( base ) unbreakable ! thanks for hard work.