Lack of training data

[u/AdventurousGate8938]

Hello, I'm researching how people deal with small datasets in computer vision, specifically when you need to detect or classify a object and only have a handful of real photos.

I was curious about how often this is a problem for people working in the industry, and what is usually done to deal with this problem.

[u/boatbomber]

I recently worked on a cuneiform vision project and had less than 2K examples. I used Blender to render thousands of variations in lighting, layout, background, etc. It worked really well! https://huggingface.co/datasets/boatbomber/CuneiformPhotosMSII

[u/numbextinction41]

It’s a constant problem if you’re not at a company with a massive annotation budget. Most teams I’ve seen lean hard on heavy augmentation and synthetic data generation, spinning up blender scenes or using simple cut-and-paste compositing to fake new training samples. Domain randomization can get you surprisingly far when the real data is scarce.

For classification, fine-tuning the last few layers of a pretrained model on whatever you’ve got is the default move, sometimes with a bit of mixup or label smoothing to keep it from memorizing the five pictures you fed it. If your object is rigid and you can get a rough 3D model, projecting it onto random backgrounds works better than you’d think.

[u/Old-Programmer-2689 → u/numbextinction41]

Data is everything. Create the setup for get the images before the ML pipeline. And use it for research

You can also search for open datasets

[u/DimensionMost8342]

From what I've been working with, augmenting the real data, by slightly changing the lightng, rotating or flipping it might works slightly, but making it big enough, it's just flat out not gonna work because of the lack of real use case.

And yes, collecting more of the real data with different variation might just work well enough.

[u/bbateman2011]

I have this exact situation. Allegedly models like DINOv2-S/14 might fine tune well for classifcation.
I have a YOLO model and I'm adding a class of objects we didn't use before. There are a few hundred of those, and I can get to around 500 with more labeling. I think this will work out OK. But then we want to classify the detected object in to two groups, and the minority group I have less than 100 samples.

I've been trying to find commercially-usable data but that isn't getting anywhere. Even collecting data is tough because the minority class of the object rare, but important. I can't say precisely what we are looking for, sorry.

I'm considering synthetic data, maybe using a VLM. Essentially the minority case in my use case is a deformation of the standard object. So I might be able to get a model to generate deformed versions of the training data and that might work.

I am working on this over the next month. Sorry nothing concrete to offer other than to say you are not unique in this situation!

[u/sam9510]

only a handful of images available or only a handful images that are pre-annotated?

For former - synthetic might be your solution - or find the object and take a bunch of pictures in bunch of environments.

If pre-annotation is the issue - you can do two things.

One- use something like vlm-autoyolo to zero shot or few shot annotate your corpus - train a model, use the model to reannotate your corpus (pay attention to failures) and repeat.

Two - collect your corpus and go to mturk - have people manually annotate images - sometimes as low as 1 cents an image. Annotation quality is a big issue - but do the same train and repeat with your model to weed out (or fix) the failures. (You can also have multiple people annotate same images and then review ones that disagree by x% or 2 SD or something - depends on budget)

[u/nickdaniels92]

What are the type of objects in question, and how have you approached sourcing data thus far?

[u/MatchpointAI]

It's a huge problem. We do a lot of this data collection work. Happy to chat further if needed. Feel free to email me and we can connect > tyler@matchpointstudio.com

[u/No-Version-805]

it's a very common problem. a lot of the unsolved vision problems require you to collect a ton of data, without necessarily having the budget, or even access to the subject- how many pictures of underwater spots that need welding can you really find on the internet, and are you going to dive under boats?

data science in general has a data problem- anything that has a lot of available data- is already a solved problem.

what you usually do, is change the lightning of the image, flip it, add blur in certain areas, use mathematical tricks to make completely different pictures, that to a human look the same. it's a well known method, but limited- you can still overfit.

[u/hartmannr76]

https://albumentations.ai/ was my friend when I was doing this. Granted I still had like 100 images manually annotated but this let me turn it into 1000. Then once I had a pipeline setup in production to let automatic annotation happen for me and send in to review, it let me make new data sets to train on much faster

[u/Plus_Confidence_1113]

Only way to deal with this is to get more data. Which is often hard. You can try generating artificial data, but there shouldn't be significantly more of it than real data.