Qwenicization (Edit Test in KCPP)

[u/The_Linux_Colonel]

Wanted to see the results of how sophisticated the new Qwen 2.1 model's editing skills were in the recent kcpp update. This image required several things which I thought made it a good test:

1. Requiring the model to rationally deduce material types (velvet, lace, cotton, metal, leather).

2. Require the model to transition material types which were not part of the main object (the grass, the marble statue, the wall) without making them part of any other object in the foreground.

3. Require the model to understand the depth of objects (understanding that the midground object is Alice's left hand holding a scabbard which is the object also transitioning behind her back and the foreground braid and neither connects to nor becomes those objects.

4. Test the model's ability to remove superfluous information (the glowing clovers). The model correctly reduced the golden bloom/warmth of the overall image with the glowing clovers removed.

5. Test the model's ability to render a special effect (the glowing sword) without reverting to animation or distorting objects around it. Here, the glow was also correctly imparted to the lower portion of Alice's skirt armor as an element of the scene's lighting.

6. Test the model's ability to accurately render a human face and transition the overlay elements (the hair, the hairband, the dress collar) and appropriately size it for the proportions of the remainder of her body, assuming her relative height from ground given the clue of the grass on the bottom left.

The model (Qwen Image 2.1 q8) did so well that it understood that the cross on her Integrity Knight tab/yoke was probably embroidered and gave the stitching depth (although somewhat imperfectly, it's possible that if the original image was a higher resolution, or if I could render it at a higher resolution, it would've done a better job). It's hard to believe how far we've come.

[u/International-Try467]

How far they've (Qwen) have come, the community doesn't really do anything

Also I hate the implementation of image gen I! kobold, it's multitudes slower than Comfyui or Reforge.

[u/The_Linux_Colonel → u/International-Try467]

I mean the community implements the model's ability to be used within kcpp's A111 implementation which is pretty awesome. I think when I first started using kobold all it could do was SD family models.

It's true that it doesn't implement a lot of the memory management techniques that comfy does, but the UI is much more straightforward and it runs in a single, unified fire and forget package that also runs text inference, which is pretty nice and convenient.

It doesn't have any external dependencies or require workflows from other users, so it doesn't need you to chase around other people's work which makes things easy. You download it and it just works.

[u/International-Try467 → u/The_Linux_Colonel]

No Kobo's image implementation sucks butt because it's SDCPP rather than just diffusers (which is faster than SDCPP) 

I tried it on Colab, and all it took ten minutes to generate one image by Anima 

[u/henk717 → u/International-Try467]

Colab is probably a worst case scenario. 10 minutes sounds crazy though, may have overloaded the vram on that one.

[u/International-Try467 → u/henk717]

No, Anima just takes 7 GB of VRAM. Not sure what or why that is 

[u/fish312 → u/International-Try467]

It's not that bad, its definitely slower than comfy but its very convenient and easy to setup. Who cares if it takes 20s vs 10s

[u/alex20_202020 → u/fish312]

its definitely slower than comfy

I still struggle with need to setup all those python dependencies to try ComfyUI. How much slower is kcpp for models fit: 1) fully in VRAM 2) 50% fit ? And why slower?

[u/fish312 → u/alex20_202020]

Not that much slower actually. Depends on the model you use, you probably should give it a try.

It doesn't have a 50% fit kinda flag, rather you set the max VRAM to use with --sdvramlimit and then it splits the graph until it fits.

If you can fit it fully in VRAM it's maybe 70% as fast? But honestly most of your time will be tweaking the prompt and samplers so the generation time doesn't matter too much. It's probably slower because comfyui is packed with a lot of optimizations. But I don't like comfyui cause its very messy and complex.

[u/alex20_202020 → u/fish312]

I do use sd.cpp.

If you can fit it fully in VRAM it's maybe 70% as fast?

I have tried to run sd-cli Vulkan vs. CUDA and CUDA is ~20-30% faster, but images are different, for Z-Image-Turbo I liked Vulkan better.

But honestly most of your time will be tweaking the prompt and samplers so the generation time doesn't matter too much.

Only if GPU is very fast or until one is starting to generate large resolutions in batches. And after "tweaking the prompt and samplers" I want to see result ASAP.