Larger pots do several things in both tone and volume controls. They present less of a load on the pickup, so frequency response improves and output increases. Also, the output response curve changes. The improved frequency response and output is not dramatic, but it's perceptible even to a casual ear. The change in response curve can be problematic, though. You may end up with most of your control at one end of the pot's throw, rather than evenly spread throughout the pot's rotation. With some pickups, it'll behave as though you've put a linear taper pot in place of a log (audio) taper pot. With the tone control, you can often mitigate that behavior by changing the value of the filter cap, but you could end up with too much cut, making the thing sound dead.
Any pot at all is going to load the pickup(s), though, which is why there's such a thing as a "no-load" pot. With those, they actually open the pot up at one end so when you're at that extreme, the pot isn't even in the circuit anymore. It's essentially an "infinite megohm" pot, which = no load at all. Lets the pickup fly free. "No-load" pots aren't easy to find, though. You can make them yourself, but I don't know if it's worth it. If you really want to dump that load altogether, it's easier to put in a mini-toggle.
Another side effect of high resistance or no-load pots is that your guitar's output impedance goes up some, so it's a little more susceptible to noise. But, since you're getting more output the S/N ratio probably doesn't change much. Still, the amp will see it and amplify it.