Back on topic.....
If you have fretwear, you'll see it. You'll see either very flat fret-tops from bending, or grooves in the frets from pressing strings into them. This "usually" occurs first at the low frets where we tend to play more.
I'm wonding if fret wear is the culprit... because... bad adjustment of the neck relief will also cause the same thing. You can in fact, have the relief set such that the middle of the neck is high, then raise the action to accommodate that high middle... and its just terrible.
Try tuning to pitch - with the strings you normally use. Changing string gauge/brands/winding WILL cause changes in relief. Minor sometimes, major sometimes, but its always a change... even if only a few .001's of an inch. So stick to one string type. Get it tuned to pitch and stretched in. Then check the relief against itself, lightly pressing fret 1 and fret 21 (or 22) at the same time. Around fret 8 or so, you should get right about the thickness of a B string between the top of the fret and the bottom of the string. Some folks like a little bit more - just a very little. Some like a little less, but you have to have a very "light" touch to play buzz free with that sort of relief. Just thickness of a B string, or a little more is about perfect for most folks.
Then set your string elevation, at fret 12 as follows - 3.5/64ths of an inch at the high E (and all the plain strings) - between the string and fret top. Go 4/64 on the low E (and all the wound strings). That should get you really close to a perfect setup. You'll have to look REAL CLOSE with a magnifiying glass and a machinists ruler to get the spacing in 1/64 increments "just right". Remember - its string bottom, not the middle of the string that gets set at those spacing... to the top of the fret.
If you do that, then we'll know if you got fret wear.