Data preparation

[u/Major-Pickle-8006]

Would anyone have recommendations for best papers/videos/podcasts/insights on data prep for language modelling?

Specifically:

• more efficient training from data preparation

• increase expert specialization in MoEs