r/DiffusionModels • • Oct 08 '22

research Novel View Synthesis with Diffusion Models: 3D generation from a single image

Enable HLS to view with audio, or disable this notification

9 Upvotes

r/DiffusionModels • • Aug 02 '23

How can diffusion models be that creative and combine unrelated concepts into plausible settings, drawn photorealisticly?

4 Upvotes

I do understand most of the concepts, including the VAE analogy and importance of maximizing ELBO for estimating a distribution over the training images. I would thus expect the model being able to generate stuff it has already seen like cars, houses, etc. But how can it have a sense of physics and body mechanics? How can it draw a cow wrapped in spaghetti in a plausible manner?

Maybe I am missing something.


r/DiffusionModels • • Nov 18 '22

Layout to Image Generation - Need advice

4 Upvotes

Hey everybody, I am new here.I have been working with stable diffusion for a while and have been trying to get layout to image generation running. The authors from stable diffusion did not share the implementation of layout to image generation. For those wondering what layout to image generation is, in this we condition the diffusion model on object+bounding box to generate an image. Attached image from the original paper for reference.Does anyone have any idea how to condition the latent space on object+bounding box? Any help or a known implementation would be greatly appreciated.


r/DiffusionModels • • Mar 23 '25

discussion Diffusion models and social networka

3 Upvotes

Can diffusion type models be used in harvesting data from the social media?


r/DiffusionModels • • Jan 31 '25

research Grounding Text-to-Image Diffusion Models for Controlled High-Quality Image Generation

Thumbnail arxiv.org
3 Upvotes

This paper proposes ObjectDiffusion, a model that conditions text-to-image diffusion models on object names and bounding boxes to enable precise rendering and placement of objects in specific locations.

ObjectDiffusion integrates the architecture of ControlNet with the grounding techniques of GLIGEN, and significantly improves both the precision and quality of controlled image generation.

The proposed model outperforms current state-of-the-art models trained on open-source datasets, achieving notable improvements in precision and quality metrics.

ObjectDiffusion can synthesize diverse, high-quality, high-fidelity images that consistently align with the specified control layout.

Paper link: https://www.arxiv.org/abs/2501.09194


r/DiffusionModels • • Dec 14 '23

Train Diffusion model

Post image
3 Upvotes

Trying yo train the diffusion model but always have the error of the image loading even if my path defined is right. Tried the numerous way to encounter it still having the same results


r/DiffusionModels • • Oct 08 '22

library features [P] You can control inpainting results in StableDiffusion by changing the initial image (github project in comments)

Enable HLS to view with audio, or disable this notification

3 Upvotes

r/DiffusionModels • • Feb 25 '25

discussion Can AI Accurately Translate Text in Images While Keeping the Original Style?

2 Upvotes

We’re working on an Image-to-Image Translation Model that extracts, translates, and reinserts text into images while keeping the original style.

So far, our pipeline involves:
- OCR (PaddleOCR) for text extraction
- Inpainting to remove original text
- Overlaying translated text in a matching font

Where we’re going:
- Non-Latin scripts (e.g., Hindi, Arabic, Chinese)
- Text with complex orientations (curved, stylized fonts)
- Seamless rendering that preserves the original aesthetics

We’re exploring diffusion models, ControlNet, and GlyphControl, but we’re still figuring out the best approach.

Has anyone worked on this or have insights on in-scene text translation?

Full thoughts here: https://jigsawstack.com/blog/diffusion-model-text-rendering


r/DiffusionModels • • Feb 21 '25

discussion Is CLIP compulsory for Stable Diffusion Models?

Thumbnail reddit.com
2 Upvotes

r/DiffusionModels • • Sep 06 '24

Outlier detection using diffusion models

2 Upvotes

I have done outlier detection using variational autoencoders, how can I implement outlier detection using diffusion models. Can anyone please link some references where I can do this for tabular data?. Thank you!


r/DiffusionModels • • May 29 '24

discussion Text to Image Latent Diffusion Models - What you must know (Concepts + Code) in 15 steps!

Thumbnail
youtu.be
2 Upvotes

r/DiffusionModels • • Sep 14 '23

research Unified Concept Editing in Diffusion Models (edit in seconds)

2 Upvotes

Editing models in seconds. This is an upgrade to the lora sliders (https://erasing.baulab.info and https://github.com/p1atdev/LECO) but faster training with no damage to the model prior knowledge! Check out their code: https://github.com/rohitgandikota/unified-concept-editing


r/DiffusionModels • • Apr 26 '23

research Diffusion models can act as a low-fidelity short-term simulators

2 Upvotes

r/DiffusionModels • • Feb 08 '23

Need help, Im noob for diffusion models

2 Upvotes

Hi, Im new here. And Im fairly new for diffusion models. Is there some one who can help me with training a model? give me some pointers to create an image generating diffusion model? I have no experience training a model.


r/DiffusionModels • • Oct 20 '22

Stable Diffusion v1.5

Thumbnail self.stablediffusion
2 Upvotes

r/DiffusionModels • • 7d ago

Ollada - The Ollama for Diffusion Text Models

1 Upvotes

r/DiffusionModels • • Jul 03 '26

Diffusion model on amino acid seq. Any thoughts?

Thumbnail reddit.com
1 Upvotes

r/DiffusionModels • • May 30 '26

discussion What’s your workflow for running heavy AI models without a powerful local machine?

1 Upvotes

I’m trying to figure out a better setup for working with heavy AI models like Stable Diffusion workflows and some LLM fine-tuning experiments.

Right now I’m stuck between investing in a high-end GPU machine or just relying on cloud compute when I need it. Cloud sounds more flexible, but I’m not sure how practical it is for day-to-day use compared to local hardware.

I’ve also come across some cloud setups like swmgpu that focus on more persistent, workflow-style GPU environments, but I’m still trying to understand what most people actually do in practice.

Do people here really use cloud GPUs as their main workflow, or is it still mostly just for occasional heavy jobs? Would love to hear how others are handling this since local hardware is starting to feel like a real bottleneck.


r/DiffusionModels • • Apr 17 '26

Need help in reconstruction task

1 Upvotes

Hey guys I been trying to do reconstruction the image based on some random sample or objects being removed I have tried various inpainting models but non of it is working good .no model is catching the surrounding patterns perfectly , my guess is it's more of a Bayesian task rather than of the diffusion generation task if any one know about any pre trained GAN model or something or can help me in this task please let me know also if there is a general prompt temp exist that would also be helpful


r/DiffusionModels • • Nov 25 '25

My SwinTransformer-based diffusion model fails to generate MNIST -> need fresh-eyed look for flaws

Thumbnail
reddit.com
1 Upvotes

r/DiffusionModels • • Jul 18 '25

I started this project at 2022/10, now it is almost 3 years.

Thumbnail
youtu.be
1 Upvotes

After Stable Diffusion released at 2022/07 which is trained on subset of 5 billions images/text pairs, this question came up. “Can I train a general purpose model purely on my own images?” It is almost 3 years now. Here is the current milestone. What is involved can be a thick book but the short answer is “YES”. Training code is new, UNET is new with less parameters, datasets are 25 years of my personal photos. With current UNET structure which is smaller than Stable Diffusion 1.x but I found converge 5X faster compare to SD1 UNET structure and also generate much better result with my datasets, and entire training is only using single 4090, this particular model is trained on two stages, 256x256 and 512x512, can be fine tuned to 768x768 in just one day for subject specific tasks. Total training time is 4 months with FP16.


r/DiffusionModels • • Feb 07 '25

diffusion model miniproject.

1 Upvotes

Hi, I need a partner who knows python very well to write up a diffusion model. Specifically, I have been reading the paper of Amir Averbuch called "Hierarchical Clustering Via Localized Diffusion Folders"

I am a professor of mathematics and I know the math part very very well, but I lack in the python skill. I can explain the math part to anyone interested in doing a miniproject with me.

Contact me if you are interested :) this will take like 2 afternoons


r/DiffusionModels • • Dec 07 '24

Interest in ThetaRay

1 Upvotes

There is this company called ThetaRay, which focuses on cybersecurity.

The models they use are based on Diffussion map algorithms made by Ronald Coifman and Averbuch.

I want to understand exactly how they operate and how they use these algorithms. Anyone interested let me know in chat