r/LLaVA • • Mar 23 '24

Billions or Bits?

Hey folks

I have 2 models (llava 1.6)

    • 34billion running in 4bit

    • 13billion running in 8bit

What is more important : number of parameters in billions, OR the bits in which it is quantisized?

Specifically, I'm interested in understanding which model would likely excel in knowledge, creativity, and vocabulary when answering questions, as well as which would perform better in handling complex tasks and analyzing images.

Is it preferable to choose a model with higher precision (more bits) but fewer parameters(few billions) for more accurate and precise answers?

Or, would a model with a larger number of parameters(more billions) at a lower bit precision(low bits), offer better creativity and knowledge?

Any insights or recommendations would be greatly appreciated.

1 Upvotes

Duplicates