2

Soup-of-Experts: Pretraining Specialist Models via Parameters Averaging