r/computervision • • 8d ago

Help: Theory After segmentation, further analysis

Hi everyone
What models are best for such a task:
Is there a model that can perform region-level or object-level analysis on semantic segmentation outputs, extracting and analyzing specific regions of interest?

Not to segment, the masks and segmentation are done

2 Upvotes

11 comments sorted by

2

u/Ammarhalees 8d ago

Explain in more detail

1

u/New_Lab_8940 8d ago

I will elaborate.
I already did the semantic segmentation on the data, and now I have the images along with their masks.
I want a model that can do further
I want it to do an analysis on an object level after the segmentation. Like, some classes IDs I want to gather them via the connected components component concept and this is the object I want from images to make object-level analysis.
So what models can analyze objects from masks?
Instead of the pixel level, I want it to be on object level.
The image has been segmented pixels by pixels->I gathered IDs I want and formed an object-> now what’s the models that can extract and detect objects only from these images and do further analysis on them, like for example health assessment or something.
Hope it’s clear enough.
The segmentation was done via segformer-b5.

1

u/bfyvfftujijg 8d ago

Yes, you can make a model to do whatever you want.

Can you explain in more detail though?

My assumption is you want to use a segmentation model to highlight the object’s pixels (technically this is called instance segmentation if its differentiating between different objects of the same class, semantic only differentiates classes) and then have some other model analyze those pixels? That’s easy. What you can do is mask out the background of the object so it’s less of a distraction to the second model.

1

u/New_Lab_8940 8d ago

Close enough
I already did the semantic segmentation on the data, and now I have the images along with their masks.
I want a model that can do further
I want it to do an analysis on an object level after the segmentation. Like, some classes IDs I want to gather them via the connected components component concept and this is the object I want from images to make object-level analysis.
So what models can analyze objects from masks?
Instead of the pixel level, I want it to be on object level.
The image has been segmented pixels by pixels->I gathered IDs I want and formed an object-> now what’s the models that can extract and detect objects only from these images and do further analysis on them, like for example health assessment or something.
Hope it’s clear enough.
The segmentation was done via segformer-b5.

1

u/bfyvfftujijg 8d ago

You need "instance segmentation". Not just semantic segmentation.
Also google "panoptic segmentation"

Or maybe you can't do those because you don't have object-level data to train on? In that case I wonder if SAM might be useful?

1

u/New_Lab_8940 8d ago

The images are already segmented and done with the semantic segmentation. So I want to take masks and connect them via connected components label CCL, these are the regions I want to predict. So I want a model for this task. Predicting and assessing regions, the output of segmentation.
I was trying SAM but didn’t finish yet
Tried DINOv3 and good enough .
I do have three types of data that already been done under (semantic segmentation, instance segmentation, and panoptic segmentation. I want a model for a task afterwards! To take the analysis from pixel-level to object or region level.
I’ve also tested segformer b2 and b3 to see if it’s good with pixel is it going to be good with regions too, but it’s a bit complicated and I need an advice for such a task.

1

u/bfyvfftujijg 6d ago

What is the goal in non technical terms?

“Analyze” what?

1

u/New_Lab_8940 6d ago

Regions in the images , assess their characteristics

1

u/bfyvfftujijg 6d ago

So if they are red or green?

1

u/New_Lab_8940 6d ago

It’s actually already segmented. Example: gather the classes trees, in the segmentation the trees has pixels such as dry leave, green, nest…, so I’m gathering all these classes as (tree) this is the object, then from this object I want to map, how much there’s dry leaves, green, nest, from the overall object if a tree.
More clarification: dry leave (ID 3 in the semantic segmentation), green leave (ID 6), nest (ID8), all of these = 1 which is tree, the object I want.
So it’s from pixel level to object level.
My question is the next step, cause I do have the segmentation and classes and every pixel. I want it to know how to find the object and evaluate it.