We study the consequences of rotational symmetry breaking in isotropic vector spin glasses. Starting from a microscopic model we identify the underlying symmetries of time-dependent (replica-dependent) SO(m) rotations. The hydrodynamic triad theory is confirmed and the spin-wave stiffness is related to a generalized transverse susceptibility. An expansion around mean-field theory is used to calculate the stiffness approximately.