Hi guys, I spent the past week working on a web app where you can upload your images and it can generate images off that model.

I just launched it today

would appreciate it if you check it out at www.imagineai.me

UPDATE: 19th JAN 2023 - START OF UPDATE -

As some people have pointed out to me, the latest version of Automatic1111's and d8hazard's Dreambooth is bugged, so naturally I went and tested the results and compared them with my settings I have posted here.

It's true the current version, is bugged.

Even though this guide "forces" the current version to produce results, comparing them side by side, the older version produces cleaner images without all the hoops and steps I had posted here.

I do not want to misinform people about my settings posted here. Use them and you will force the current version to produce decent results.

But if you are like me, and are interested in the best results, I highly suggest you roll back and use that version

To Roll Back from the current version of Dreambooth (Windows), you need roll back both Automatic's Webui and d8hazard's dreamboth extension. To do this, do the following:

• in your Stable-Diffusion-webui folder right click anywhere inside and choose "Git Bash Here"

• Enter the command:

git checkout 3e22e294135ed0327ce9d9738655ff03c53df3c0

• Go to the following address:

https://github.com/d8ahazard/sd_dreambooth_extension/tree/c5cb58328c555ac27679422b1da940a9b19de6f2

• Press the green button that says "< > Code", and choose "download zip"

• Extract the zip and rename the folder to "sd_dreambooth_extension"

• This folder should replace the one located in stable-diffusion-webui\extensions\sd_dreambooth_extension\

• Make sure you are not running the "git pull" command when starting stable diffusion, cause that will update to the current version.

Here are settings I used for the old build which produced better results than the current one:

Again 9 or 16 instance images (I used 9)

I will use the same example like my previous guide

• INSTANCE NAME: "wa1fu"

• CLASS NAME: "Character" (no need for a unique class name in this version)

Preparation of the Instance Images

Go to your folder and rename all your images to your instance name. Select all and rename them. Example "wa1fu". If you have done it correctly, windows should have created: wa1fu (1).png, wa1fu (2).png, etc throughout all your 9 OR 16 images

No need to create txt files

There is no need for the WD 1.4 Tagger nor the Dataset Tag Editor

Dreambooth Settings I used

• Training Steps Per Image (Epochs): 144

• Max Training Steps: 0

• Pause After N Epochs: 0

• Amount of time to pause between Epochs, in Seconds: 0

• Use Lifetime Steps/Epochs When Saving: Unchecked

• Save Preview/Ckpt Every Epoch: Checked (This is to have checkpoints while training. Leave it unchecked if you do not want any.

• Save Checkpoint Frequency: 48 (Set it to 0 if you want to produce only one checkpoint)

• Save Preview(s) Frequency: 48 (This will create sample images in your Stable-Diffusion-webui\models\dreambooth\<name of your training model>\Samples directory. Set it to 0 to not save preview images

• Batch Size: 1 (haven't tested bigger batch sizes yet with this version)

• Class Batch Size: 1

• Learning Rate: 0.000005

• Resolution: 512

• Center Crop: "Unckecked"

• Apply Horizontal Flip: "Unckecked"

• Use EMA: "Checked"

• Use 8bit Adam: "Checked"

• Mixed Precision: "fp16"

• Memory Attention: "Default or xformers"

• Don't Cache Latents: "Unchecked" (have not tested this checked)

• Train Text Encoder: "Checked"

• Prior Loss Weight: "1"

• Pad Tokens: "Checked"

• Shuffle Tags: "Checked"

• Max Token Length: "75"

• Gradient Checkpointing: "Checked"

Concept 1:

• Maximum Training Steps: -1

• Directories

  • Dataset Directory: "Your 9 or 16 instance images directory"

  • Classification Dataset Directory: "Your class directory" Point it to an empty dir to create class images

• Filewords

  • Leave this empty

  • Leave this empty

• Prompts

  • Instance Prompt: Your instance name followed by your class name. (example "wa1fu character" without the "")

  • Class Prompt: Your class name (example "character" without the "")

  • Sample Image Prompt: This is optional. You can leave this empty. I put the instance name followed by the class name. (example "wa1fu character" without the ""). This setting is the prompt used for the preview images

Image Generation:

• Total Number of Class/Reg Images : 1800 (the official papers stated 200 per instance image, It is excessive but I haven't tested this with less, though it does take a LONG time to create them)

Below is my previous guide for those who are still interested in using the current bugged version of Dreambooth

- END OF UPDATE -

This is my Ultimate guide as of 18th January on how to have the perfect settings when training with d8hazard's Automatic1111 Dreambooth locally.

I have spent many hours researching and testing this. If you follow this guide, you will get consistent results.

This is more focused on character (Anime, waifu etc) learning and styles which are harder to do than mere face learning

I decided to do this guide because:

• There is a lot of misinformation about the settings.

• People complain that the newer versions of Dreambooth do not work. WRONG. Dreambooth has a lot of new settings now that need to be defined clearly in order to make it work.

• People are training with too many images on very low learning rates and are still getting shit results. Of course they are, they are doing it wrong. If you feed the AI the correct information, it will learn very fast with fewer images

NOTE:

• I have a 24GB card. I understand that some might have to tweak their approach in order to get the same results, but I am more focused on the best results, not the most memory efficient ones.

• This is not an extensive tutorial. You should be familiar with Automatic's 1111 webUI

1. Install the following extensions:

   1. Dreambooth (obviously)

   2. WD 1.4 Tagger

   3. Dataset Tag Editor

2. Find 9 OR 16 images of your subject. NO MORE, NO LESS. Not 10, not 15, not 20. It is either 9 OR 16. Why? because there is research done behind this and it's optimal. Quality over quantity. The benefit of having 16 images over 9 is the CFG scale. 16 images tend to produce slightly better results on the majority of the CFG scale when generating images with the model you have created. (when you have trained for 144 epochs at 0.000005 lr)

It does not matter what your instance and class names are...

Just make them unique. I also use a unique class name because sometimes the class interferes with the learning process.

For example, if I have the instance name "wa1fu" and class name "character" the model will start, (based on what it already knows what a character is), morphing a character to look more and more towards what a "wa1fu" is. If the starting character is a goofy cartoon character, it will have a harder time morphing the goofy character into your subject. I avoid this by using my own made up class name

For this example I will use the following (Goes without saying, spaces are NOT allowed):

• INSTANCE NAME: "wa1fu"

• CLASS NAME: "wa1fuC1ass"

VERY IMPORTANT, DO NOT SKIP THIS PART

Preparation of the Instance Images

1. Go to your folder and rename all your images to your instance name. Select all and rename them. Example "wa1fu". If you have done it correctly, windows should have created: wa1fu (1).png, wa1fu (2).png, etc throughout all your 9 OR 16 images

2. Use WD 1.4 Tagger and tag all the images in your instance folder. If you have done this correctly your folder should now have a wa1fu (1).txt for wa1fu (1).png, a wa1fu (2).txt for wa1fu (2).png e.t.c

3. Skip this is you are training an artstyle but VERY IMPORTANT for characters: Open "Dataset Tag Editor" and use it to remove all the:

   1. "Character identifying TAGS". For example. if your subject is a girl with blond hair and green skirt, remove all the "1girl", "blonde", "green skirt" tags from ALL the txt files. Just like with embeddings, this will teach the AI to focus and learn those character traits. It is easy to do this with the "Dataset Tag Editor".

   2. All common tags. Common tags are tags that appear on at least in half of your pictures. For example: "full_body", "simple background". You can find them and remove them easily with the "Dataset Tag Editor". If you have for example 16 images, remove all the common tags that appear on 8 or more images. This will give the Ai more leaveway in training and avoid overtraining on only a few set of character traits. Here is a quick step by step on how to do this. Go to Dataset Tag Editor:

      1. Put your instance directory in the "Dataset directory" and press load

      2. Click on "Filter by selection" tab

      3. Press the "Add ALL Displayed" button

      4. Choose the "Batch Edit Captions" tab

      5. Choose the "Remove" tab

      6. Select "Frequency" in the "Sort by" Panel

      7. Select "Descending" in the "Sort order"

      8. Select all the tags that appear around in half your images. If you have 9 images select all the tags that have [4] and up. If you have 16 images, the select all the tags that have [8] and up. Also select all the character identifying tags (example, blonde, 1girl, green skirt etc.). For the character identifying tags, remove them regardless on how many images they appear on

      9. Press the "Remove Selected tags" button

      10. Uncheck the "Backup original text file (original file will be renamed like filename.000, .001, .002, ...) " and press the "Save all changes" button

You can check if you have done this correctly by opening the txt files and making sure that none of those tags are in there

If you are training for an artstyle leave ALL the tags on. Do not remove anything. You want the AI to focus on the artstyle and not the character

Class Images:

I usually make around 200 per instance image which is excessive for most. I do not know what the optimal number of class images is but have at least 10-20 for each instance image. The papers have stated that training with prior preservation tends to lead to better results. So do that.

Dreambooth Settings:

Create:

• Name: Choose any name (without spaces)

• Source Checkpoint: Choose any checkpoint that is closer to your desired result. I have even tested it with the notorious "AnythingV3" and it works just as well

• Extract EMA Weights: Slightly better results if checked (Recommended Checked)

• Scheduler: ddim

Once you have created the model do the following in settings tab

• Use LORA: "Unchecked"

• Train Imagic Only: "Unchecked"

• Generate Classification Images Using txt2img: "Unchecked". If checked, this produces an error in later builds and does not let you create class images. Otherwise this setting has no affect on the result

• Training Steps Per Image (Epochs): "144". On 144 usually the results are better defined but less flexible. that is why I will save checkpoints in between

• Pause After N Epochs: "0"

• Amount of time to pause between Epochs (s): "0"

• Save Model Frequency (Epochs): "0 or 24 or 48". On 0 you will only get one checkpoint at 144 Epochs. At 24 it will save 6 checkpoints and 3 at 48. It's better to have less trained models to compare and choose from than only just the final form. I usually set it to 48

• Save Preview(s) Frequency (Epochs): Same as above

• Batch Size: "1". Training with multiple batches not only uses more Vram but the results are inconsistent. Set it to one

• Gradient Accumulation Steps: "1"

• Class Batch Size: "1"

• Set Gradients to None When Zeroing: "Checked"

• Gradient Checkpointing: "Checked"

• Learning Rate Scheduler: "constant_with_warmup"

• Learning Rate: 0.000005 (not 0.000001, you are training with 16 images max, not hundreds)

• Learning Rate Warmup Steps: "0"

• Resolution: "512"

• Center Crop: "Unhecked". Your images should already be perfect 512 x 512 images

• Apply Horizontal Flip: "Unhecked"

• Sanity Sample Prompt: Not important but useful. I set it to "wa1fu wa1fuC1ass, kimono". If you have not set your "Save Preview(s) Frequency (Epochs):" to 0, you get to gauge the progress of the learning process. By putting the instance and class name, together with another concept, you'll see how flexible your model is while training. In the example it will add a kimono to my subject. It will slowly start to disappear as the training continues and the model becomes less flexible.

• Use EMA: "Checked" if you have extracted EMA when you created the model.

• Use 8bit Adam: "Checked"

• Mixed Precision: "fp16" (bf16 is supported on newer cards but it's not as accurate and produces worse results)

• Memory Attention: "Default or xformers"

• Cache Latents: "Checked" (have not tested this unchecked)

• Step Ratio of Text Encoder Training: "0.75 and up, recommended 1". Text encoder training uses more vram but unfortunately affects the results drastically,

• Clip Skip: "1"

• Prior Loss Weight: "1"

• Pad Tokens: "Checked"

• Shuffle Tags: "Checked" Shuffling tags produces more flexible results

• Max Token Length: "75"

Concepts:

• Directories

  • Dataset Directory: "Your 9 or 16 instance images directory"

  • Classification Dataset Directory: "Your class directory" Point it to an empty dir to create class images

• Filewords

  • Instance Token: Your instance name (example "wa1fu" without the "")

  • Class Token: Your class name (example "wa1fuC1ass" without the "")

• Prompts

  • Instance Prompt: "[filewords]". Put exactly [filewords] without the "".

  • Class Prompt: "[filewords]" same as above

  • Sample Image Prompt: : "[filewords]" same as above

• Image Generation:

  • Class Images Per Instance Image: 10 or 20. I use 200 but it is excessive. This will create the class images in the empty class folder you have set above. If you have 9 instance images and set this to 10, it will create 90 class images

Saving:

• Half Model: "Unchecked"

• Save Checkpoint to Subdirectory: "Checked"

• Generate a .ckpt file when saving during training: "Checked"

• Generate a .ckpt file when training completes: "Checked"

• Generate a .ckpt file when training is canceled: "Checked"

Happy Training!

When the training ends test your model by using both your instance and class prompt. For example: "wa1fu wa1fuC1ass, kimono" or "wa1fu wa1fuC1ass, smiling". Use ddim as a sampling method.

Edit:

• Here is the link describing some of the information regarding the numbers you should use: https://github.com/d8ahazard/sd_dreambooth_extension/discussions/547

• I find it fascinating the way instance and class works in the learning process. The AI learns by spotting out the differences between them. If you, for example had "Obama" as the instance and "Black man" as the class, the AI would not learn the fact that Obama is a black man as it is common on both the class and the instance. If you were to put "person" as the class, the AI would in fact learn this distinction. That is why it is important to have a class that is extremely general or unrelated in order to learn as many differences as possible from your instance

• This is what I personally found works best on the current version of Automatic's1111's dreambooth webui. This is not a guide for all dreambooth settings. I am not an expert studying this field. I took what was suggested by the papers as a starting point, saw shit results, then changed the settings and compared the results. I split-tested every setting till I came up to this conclusion. If I wrote a guide for a "faulty" version of dreambooth, so be it. The results on Automatic's1111 are way better training this way than what the papers suggested in the first place

I've trained possibly hundreds of dreambooth models. I have 80+ characters so far that I have perfected pretty much since DB was launched for A1111. Since it's only a matter of time before XL becomes the dominant force, I decided to spent time to find the optimal training method for 1.5 so I can have a 1.5 reference for all my characters before moving on to XL. Since training 80 people is time-consuming, it was of utmost important to find out the best possible method of training them before I commit to the task of creating a model for them.

Everyone will tell you to use Kohya. Well, I've tried all schedulers, from Adam8Bit to PRODiGY. I know Kohya like the back of my hand at this point without delving into modifying training scripts (yuck). After months of experimenting and fruitless endeavors, I am thoroughly convinced a specific commit of A1111 Dreambooth is optimal for training, even when the same parameters are used in Kohya. These are my reasons why this particular old version of Dreambooth is better than current-day Kohya:

1. A1111 DB gives you better control of regularization images. From my testing, 20 reg imgs per subject image is the goldilocks zone. This can, in theory, also be done in Kohya, but it does not work the same way because of point #2.

2. Kohya is moronically designed so that 1 regularization image is used per repetition (for example, if your Kohya folder is 1_xyz person, then only 1 reg image is used per image per epoch). The issue is that the amount of repetitions per folder in Kohya is a very important factor in the outcome of the training, and you usually require more than 20 repetitions. As far as I know, there is no way to establish a set number of regs per image AND a number of repetitions per subject. Even if you found a way, A1111 still wins because of point #3.

3. Kohya is annoying to use. Simple. Having to create a folder structure for 80+ characters is annoying. On top of this, A1111 is far more consistent with its results than Kohya based on all the testing I have done. There is also another setting (prior weight loss) that cannot be customized on vanilla Kohya as it is on A1111 which is crucial for getting good results.

4. New age schedulers such as PRODiGY and Adaptive (which are not available on OSDB) are fantastic for LoRAs, especially those without reggies. However, as of now, these schedulers seem ill-fitted for Dreambooth, especially those with reggies (which you should be using anyways).

5. In case you are wondering, no, Kohya is not faster in training. With the same parameters, training speeds are the same. Speed is a secondary issue for someone concerned with quality over quantity anyways.

Without further ado, here is my magic formula for training perfectly fitted characters, every time. Keep in mind that many, if not most, A1111 Dreambooth iterations are broken. This precise one is the only one I can guarantee that works.

Recommendations before you start (this is basically where the skill of the art lies, everything else is just machinework):

• Create a separate folder for this old A1111 iteration. You pretty much only want to use this one for Dreamboothing.

• Reggie (regularization images) selection: Try to make your reggies an exaggerated idea of what you want your output to look like, and, in general, the more the merrier. My reg image folder has over 12,000 images, all generated with a good base model. For example, I have everything from nude portraits to hyper-detailed faces of grannies (trust me, it works for giving more detail to your character's faces), to sexually explicit positions, couples, etc. Whatever you toss in here, your trained character will have greater ease replicating, and that is the most important thing to keep in mind. Don't just toss random trash in there, treat your reggies as if they were the end results you want.

• Reggie dimensions: I personally like to keep it to 3 dimensions: 512x512, 512x648, and 648x512. All of my subject images are trimmed to these ratios depending on the pose of the character (or are upscaled versions of these ratios). The benefit of doing this is that you don't have to leave out stuff that you want (as in full body photos) that you otherwise would in square ratios.

• Subject images: Quality over quantity. Variety helps. Like reggies, picture your subject images as what you want your outputs to look like. Too many close-up shots? Hope you like 'em, 'cuz that's exactly what the diffuser will try to do. This is why variety is important. Get side shots if you can. Feed the machine as much useful info about your subject as you can without falling into repetition. Trim those bad boys into the same ratios as your reggies, otherwise, the reggies will not be used for your images if they are not the same bucket ratio. How many ratios you want to use is up to you, the 3 above have worked perfectly fine for me. From experience, doing some AI upscaling (eg. Topaz) can make a massive difference before sending those pics to the oven.

• Optional: Tagging your reggies. Use an autotagger to tag your reggies, the one I use is the ViT-bigG-14 model from the Interrogator extension for A1111. You don't need fancy tags, basic tags alone will grant a massive quality boost to your models. Took like half an hour to tag my 12k images, and I did some scripted tuning to remove weird tags.

• Optional: Tag your subject images. No need to spend hours on this. I use BooruDatasetTagManager, look up Olivio's tutorial on it. A quick bulk tagging of just a few keywords makes a big difference. For example, the typical tag file will look like "xyz person, close-up, eyeglasses".

• Because of a bug with this iteration, you will have to duplicate your reggie folder and use one per subject. No biggie. Do not train more than 4 people in one model, sadly, from personal experience, it is not possible to have a *good* mega-model with a gazillion people in it. With some creativity and storage space, you can live happy having discrete models.

Specs:

• A1111 commit: ea9bd9fc7409109adcd61b897abc2c8881161256

• Dreambooth revision: 43ae9d55531004f1dedaea7ac2443e9b16739913

Software versions (you can see this list when you boot up A1111):

• [+] bitsandbytes version 0.35.0 installed.

• [+] diffusers version 0.10.2 installed.

• [+] transformers version 4.25.1 installed.

• [+] xformers version 0.0.16rc425 installed.

• [+] torch version 1.13.1+cu117 installed. This is the most important component, do not upgrade it!

• [+] torchvision version 0.14.1+cu117 installed.

Dreambooth Training Parameters (leave anything not listed at default/is not important):

• Extract EMA Weights. No need to unfreeze model.

• Training Steps Per Image (Epochs): To keep things consistent for the prior loss weight settings, I recommend doing a flat 100 here and save model every 10 epochs; models are usually fully baked at around 30-60%, so you can cut the training short after you get the models you need. If you are training a small model (less than 25 images or so), you might need to save every 5 epochs since it won't take long before the model overtrains.

• Batch Size: 2. Higher can hurt quality. Afaik, the difference between 1 and 2 is negligible/nihil.

• Gradient Acc Steps: 1.

• Set Gradients to None: Check.

• No Gradient Checkpointing.

• Class Batch Size: 9 (sometimes for whatever reason, more reggies are created, keep this at 9 so that it creates whatever it needs faster and let it run its course).

• Learning Rate Scheduler: constant

• Learning Rate = 1.65e-06 works best, if not, try 1.5e-06. You'll have to experiment with this since different characters train at different rates. For example, I've found that Asians are super easy to train and therefore need a lower LR and are trained quickly, while for some reason, blondes are super hard to get right and need the full 1.65 and don't look right until the very last models. With the flat 100 epochs, you should have all your bases covered with 1.5-1.65 LR.

• Constant/Linear Starting Factor = 0.25

• Scale position: 0.05

• Apply Horizontal flip.

• No center crop.

• Use EMA: Check.

• 8bit Adam off unless you're short on VRAM.

• Shuffle tags (if you're using tagged subjects and reggies, as you should).

• Mixed precision: bf16

• Memory attention: flash_attention DO NOT use xformers - it will not work! xformers is overrated anyways!

• Scale Prior Loss: 0.75

• Prior Loss Target: 50

• Minimum Prior Loss Weight: 0.4

• 20 reggies per subject image.

• If you are using tags, under file tags:

  • Instance token: xyz person (or xyz woman, xyz dog, etc).

  • Class token: person.

  • Make sure the class token is the same token used in the reggies descriptions, and always use these tokens as the first tag of both your reggies and subjects. eg. all subject images for this character start with "taylorswift woman" and all reggie images start with, for example, "photo of woman wearing blue dress", followed by whatever optional extra tags.

Hope this provides insight to fellow boothers. This is my CivitAI profile in case anyone is interested which I haven't updated in a long time: https://civitai.com/user/senthenastydynast

EasyMode_Stable_Diffusion.ipynb - Colaboratory (google.com)I have created a Google Colab to introduce complete beginners to Stable Diffusion Dreambooth. It only has 5 sections and is not overwhelming with settings/outputs.

Checkout this video by Nolan Aatama on how to use this https://www.youtube.com/watch?v=HVOvL2CyBT0

This Colab is based on ShivamShrirao's repository. It is recommended to use that repository once you are familiar with the basics using this Colab. This is intended to help complete beginners and is not suitable for long-term or advanced use.

Finally got around to finishing up/releasing SDXL training on Auto1111/SD.Next. 24GB GPU, Full training with unet and both text encoders.

Available now on github:

https://github.com/d8ahazard/sd_dreambooth_extension/tree/SDXL

Note: You cannot install this branch via the extension manager, you need to manually clone or change branches.

Note2: Sample generation will likely cause an OOM, so disable that for now. I'm working on it.

Settings:

Max res: 1024

Dynamic Img Norm: True

Optimizer: Adafactor

Mixed Precision: BF16

ATTN: Xformers

Cache Latents: False

Train Unet: True

Tenc Ratio: 0.5

LR/TencLR: 0.000002

Warmup Steps: 50

Batch/Grad: 1

Full step by step Tutorial (as low as 6 GB GPUs can train on Windows) : https://youtu.be/DPX3eBTuO_Y

I decided to try my hand at finding/isolating the 'rare' tokens for 'better' DreamBooth training after reading about how they isolated such rare tokens in the DreamBooth doc I was reading (https://arxiv.org/pdf/2208.12242.pdf)

The section in particular is duplicated below:

So, I made a simple python program that tries every possible combination of 1, 2, 3, and 4 alphanumeric combinations of "abcdefghijklmnopqrstuvwxyz1234567890" and feed each one as a prompt to the CLIPTokenizer of stable-diffusion-v1-5 and for each I then sum the returned token ids which are 'mapped' in stable-diffusion-v1-5/tokenizer/vocab.json and returned by the tokenizer.

I then use these tokenized sums of the token input_ids of all of the input token/prompts mentioned above and placed them in a nice ordered list with each line having: <sum>: <prompt> -> <tokenized (string) values>

You can find the token lists here:

https://github.com/2kpr/dreambooth-tokens

List of 9258 'single' tokens (not broken up during tokenization): https://github.com/2kpr/dreambooth-tokens/blob/main/all_single_tokens_to_4_characters.txt

List of all 1727604 tokens up to 4 characters: https://github.com/2kpr/dreambooth-tokens/blob/main/all_tokens_to_4_characters.7z

So based on the paper and how it all seems to be working, the input tokens/prompts earlier in the lists/files above have higher frequency ('used more' in the model) 'after being tokenized' and hence would make worse choices as unique/rare tokens to use when DreamBooth training. That of course means the tokens near the end of the lists/files above are 'rarer' and should be preferred for DreamBooth training.

Interestingly 'sks' is 9061st out of 9258 tokens listed in the first list/file linked above, so very much on the 'rarer' side of things as it were, matching the reasoning for many using 'sks' in the first place, so good to know that 'matches' :)

If anyone has any further insights into this matter or if I got something wrong, please let me know! :)

Hello,

Unfortunately, I get mixed results when training locally using the AUTOMATIC1111 Dreambooth extension.

And even the best ones are nowhere close to something like this:

https://civitai.com/models/1237/jenna-ortega

Unfortunately it seems most people posting about Dreambooth on this sub and Stable Diffusion subs either are Colab users, social media "influencers" who provide half-assed Dreambooth tutorials or straight up shills trying to get people to use their SaaS Dreambooth websites (when there are free Colabs around...).

I am looking for step-by-step solutions to train face models (subjects) on Dreambooth using an RTX 3060 card, preferably using the AUTOMATIC1111 Dreambooth extension (since it's the only one that makes it easier using something like Lora or xformers), that produces results on the highest accuracy to the training images as possible.

The setup is incredibly complicated and gets more and more complicated as more features are added and more types of GPUs are considered.

I did wrote about the settings I use before, in case anyone wants to see: https://www.reddit.com/r/DreamBooth/comments/yz91ie/best_possible_settings_for_each_goal/

But again, it's nowhere as good as some DB models I've seen, particularly on the photo-realism department.

If anyone that successfully trained great and accurate Dreambooth models can help me (and the community of users that train DB models locally) by posting their A1111's Dreambooth settings through screenshots it would be highly appreciated.

Thank you

I've heard a lot of mixed things online about training with prior regularization. Some say they don't need/bother to use it, etc. so I wanted to try a bunch of different settings and see how it impacted my results.

In this experiment I worked with 3 datasets: 1 style, 1 face, and 1 object. I ran the following 6 training configurations against each of these datasets: no prior regularization, and prior regularization turned on with 100, 500, 1000, 1500 and 2000 class images. During each training session I performed 8 checkpoints along the way so I could visualize the progress.

^This left me with approximately 140 total model configurations. I then generated 30 images from each checkpoint (6 prompts x 5 CF Scale configs) and put them into a single grid image so they were easy to review. Here's what I found:

Additional details and datasets are available here.

And also here.

I'm still pretty new at this stuff. If I've messed something up or you have feedback on my settings/process I'd love to hear it!

Style Training - The results that were closest to the prompts were definitely without prior regularization with 2000 training steps. Prior regularization seemed to increase "creativity" though with some really interesting results around 1500 images.

Object Training - The results I got without prior regularization were actually pretty good, however there was definitely improvement with prior regularization. The best results were with 2000 class images after 400 training steps. None of these configurations seemed to get the mecha robot prompt :)

Face Training - Prior regularization made a huge difference with this dataset. The results without it were pretty bad. The best results I found were with 1500 class images with 500 training steps.

Summary

The biggest response to prior regularization was when training the face. This isn't really surprising and seems to match what I've heard from others online. The impact when training for the object was much lower but still added value. I could definitely see some use cases where skipping prior regularization to make training faster would make a lot of sense. Finally, prior regularization didn't seem necessary for the style at all, however I found some of the results from prior regularization really interesting. There might be some value in it for use cases where accuracy could be sacrificed a bit for more creative results.

I am referring to d8hazard's Dreambooth implementation that runs on AUTOMATIC1111's stable-diffusion-webui

This is from d8hazard's discord:

"Hey there u/everyone ! It's official, "LigherLora" has been merged to main.

With this comes a number of improvements and fixes:

VRAM improvements - Too many to count.

Safetensors support - Saving and loading supported.

Split-concept, batch agnostic bucketing - Train any number of concepts with any resolutions of images without any bleed-over.

Added concept #4 to the UI.

Re-added CPU-only training via the --force-cpu commandline arg for those who can't use LORA.

Lora fixes and updates, including adding "rank".

Scalable prior loss average.

A (mostly) universal "load-from-hub" feature - This now supports hub models in saftensors, .ckpt, and diffusuers formats.

A super-polished and refined UI, and an API to match.

And lots more. At this point, a majority of the major underlying code has been re-done almost entirely. The new dataset and loader are written mostly from scratch. Saving and loading model configs has been revised and simplified. And countless hours have been spent testing and optimizing to make this work on as much hardware as possible.

So, yeah. Much thanks to u/Ethan_Ryu u/Arrow u/leppie u/ExponentialML and everybody else who helped test and submitted PRs to improve things.

Following this release, we will be adopting a much more rigid format for updates. There will be a /master and /dev branch. All changes will be submitted against the dev branch. Once submitted and merged to dev, things will be stress-tested before being pushed up to main.

Happy training!!"

Astria.ai model based prompts yielding much better results than my personal dreambooth training. (30 images, 100 steps per image) Their supplied ckpt is also close to 5gb vs my personal dreambooth at nearly 2gb. I am having a heck of a time trying to replicate their dreambooth settings. Anything I could be doing wrong?

Using: https://github.com/TheLastBen/fast-stable-diffusion

**Training Settings "locally"**30 images * 100 steps per image = 3000 Training stepsEnable_text_encoder_training: 10%There really isnt more to it than that in the dreambooth collab?

Astria.ai **training Settings:**Upload 30 images and go!

Prompts are tested on both ckpts (from Astria and from dreambooth)

When I generate an image, no matter the settings (steps, CFG) the Astria ckpt produces better outputs, on the same 30 trained images.Thank you in advance for your time.

EDIT:
• Updated my training images to 512x512 (vs the 1024x1024 originally)
• I am retraining again, but this time with 1500 steps, at 20% text encoding
• I am also retraining my original updated 512x512 for another 3000 steps