It depends on what the covers are made of and/or plated with. Brass, nickel and gold are non-ferrous, and have no effect. Chromium will. So, a nickel, nickel plated or gold plated brass cover won't have an effect, but chrome plated steel or brass will. Of course, wood and plastic will not.
The effect it has is to roll off the highs very slightly due to eddy currents in the cover. It's not something you'd really hear, but the effect does take place. So, of course the fanatics and zealots blow it all out of proportion and take heroic steps to remove their covers and point out why: their ears are so much better than yours.
Back 100 years ago, we used to pull covers because they often made the pickups microphonic. They usually weren't potted back then. Today, there's no reason for it unless you've got some ultra-cheap Chinese part that didn't get potted.