Traditionally, they even have names: Gaspar, Melchior and Balthazar. But who says there were three of them? Certainly not Matthew.
The idea that there were three wise men (magi) probably arose from the fact that Matthew identifies three gifts that they brought on the occasion of the birth of this new King.
Their story is fascinating. They were probably some kind of astrologers who had stumbled on the news of a significant birth and had headed off to find the future King whose birth had been announced.
Rather than see their story as a biblical endorsement of astrology (!), we should see their worship of the Christ as a further preview of what had already been hinted at in the genealogy of chapter 1 and would come to fruition in the words of the Great Commission: the gospel is for the nations.