Code belongs in pretraining, not in a later phase
A model that meets code only after language has settled has to learn it against established representations. That is more expensive and gives a worse result.
It is tempting to train language first and add programming later as a specialisation. It is also more expensive than doing both at once.
Representations formed early are the ones everything later is built against. A model that has spent its entire budget on prose has settled on structures that treat indentation as noise and identifiers as rare words. Teaching it code afterwards means fighting those structures rather than forming them.
The mix is a zero sum choice, and pretending otherwise is how mixes drift. Every point given to code is a point taken from prose. The split is written down in one file, and a new capability means a new entry there, not a change in code.
There is one asymmetry worth exploiting. Deduplication is mandatory for prose and wrong for code: similar source files are usually legitimate variants of the same pattern, and those variants are exactly what teaches the pattern.