Abstract: In this work, we explore the benefits of Variance-Covariance Regularization in Continual Learning (CL). Neural networks suffer from abrupt performance loss when updated with additional data.