I found that if I didn’t use a primer then the paint would chip off on me. It didn’t matter so much on the complexes because you don’t move them around and pile them with all of the other units in your tray, but certainly on the infantry it made a difference. So yes I recommend you use a white primer. Ask at the model paint store which one would be best for you.
Curiously though, some brands would chip off more than others due to the type of plastic they used. Airfix chipped off big time. OOB units chipped off a lot too. Caesar didn’t chip at all.