Your baby should let you know when he is hungry. If your baby eats frequently during the day, he may fill up and not be hungry. My son used to eat every 2 hours when he was an infant and then he would sleep all night.
I agree with the others. My pediatrician always asked if we "trusted" our baby. As in, does the baby tell you when he is hungry. If you feel confident that you understand your babies cues and he lets you know loud and clear when it is feeding time, then I say let him sleep. Also, consider the weight gain. If he is gaining appropriately (about 1/2 an oz. a day) then you are fine. Enjoy your sleep.