One could argue that if it's playable with no fret buzz, then there's not a problem. It takes a very level set of frets to get away with that, which is a Good Thing. Relief is a compensatory adjustment, not a requirement. In a perfect world, you'd need no relief at all.
The problem comes when we talk about "playability", which is highly subjective. One guy's "playable" guitar is another's useless nightmare. Often, you can compensate for a less than ideal fret plane or lack of available relief by raising the bridge high enough that there's sufficient string clearance to avoid interference with string vibration. But, is it playable set up that way? Depends on the player, but most would prefer the strings fairly close the fretboard.
If the thing is playable for your comfort level using the strings necessary to get it to behave, then you don't have a problem, right?
Personally, I'd have a pretty rough time playing 11s, but that's just me. There's certainly a population of players that are comfortable with that gauge set, or there'd be no such thing.