Skip castle rights update when not needed