The dynamic mean field theory of the spin glass phase is generalised to n-component spins. Exact averaged equations of motion are derived within a purely relaxational model. It is shown that dynamic stability enforces a violation of the fluctuation-dissipation theorem for any finite n and T<Tc. The marginally stable state is characterised by algebraic decay of correlations t- nu with an exponent which depends continuously on temperature and the number of components, nu (T,n)=1/2-3 mod T-Tc mod / pi n(n+2)+O( mod T-Tc mod 2)+O(1/n3).