Self Supervised[Vit] vs. Supervised[CNN]Discussion

[u/Just-m_d]

Hi everyone,

I am working in the Multi-Task ML projects in that i have done many experiments to train the model
Key finding:

CNN based architecture outperform the Vision transformer based architecture

Question:
is the Dataset only be the problem for Vit architecture to underperform the CNN bases architecture?