continuous integration is not for compilers #4
Description
I don't think continuous integration should be applied to compilers.
A compiler should be upgraded very consciously. The program compiled with it, should be checked carefully against a list of modifications and fixes that is going with the upgrade.
Compilers should be the most reliable pieces of software to begin with. So upgrades should probably years apart with years overlap between versions.
Few compilers are so fully defined that a large program would not be afected by some minor change. Especially if there is the attitude of "we'll fix it next time" there may be no reliable version of the compiler at al, at any time.
That there an automatic test is sufficient is also a dangerous illusion. I perform a number of manual test in addition for a new release of ciforth's and they always detect issues. In addition nothing is released after a considerable in-house burn-in period.