Yes, as you just confirmed in your message, regarding rarer notes if you spend the extra money to get a possibility inflated grade that results in more money than it makes sense for that seller who is clearly trying to profit. For notes valued under one thousand, it may not make sense to invest that much into the holder.
REMEMBER: Always buy the note, not the holder!