We do not know when exactly the star showed up. We know that the wise men did follow the star to Jesus before returning home towards Persia.
Star of Bethlehem Background
Now, after Jesus was born in Bethlehem of Judea in the days of Herod, the king, behold, wise men[a] from the east came to Jerusalem, 2 saying, “Where is he who has been born king of the Jews? For we saw his star when it rose[b] and have come to worship him.” 3 When Herod the king heard this, he was troubled, and all Jerusalem with him; 4 and assembling all the chief priests and scribes of the people, he inquired of them where the Christ was to be born. 5 They told him, “In Bethlehem of Judea, for so it is written by the prophet:
6 “‘And you, O Bethlehem, in the land of Judah,
are by no means least among the rulers of Judah;
for from you shall come a ruler
who will shepherd my people Israel.’”
7 Then Herod summoned the wise men secretly and ascertained from them what time the star had appeared. 8 And he sent them to Bethlehem, saying, “Go and search diligently for the child, and when you have found him, bring me word, that I too may come and worship him.” 9 After listening to the king, they went on their way. And behold, the star that they had seen when it rose went before them until it came to rest over the place where the child was. 10 When they saw the star, they rejoiced exceedingly with great joy. 11 And going into the house, they saw the child with Mary, his mother, and they fell down and worshiped him. Then, opening their treasures, they offered him gifts, gold frankincense and myrrh. 12 And being warned in a dream not to return to Herod, they departed to their own country in another way.
The Star of Bethlehem was only seen by the Magi. When they arrived in Jerusalem, they visited Herod and inquired about the birth of Jesus.
After leaving Herod's palace, the star appeared again and would lead them to the place where Jesus was staying. He was no longer a baby, but insteadof 2 or 3 years old.
After giving their gifts, the Magi returned to the East, which was probably Persia at the time.
After the Magi left Jesus, the Lord had Joseph and Mary leave Bethlehem for Egypt until the death of Herod.
To go further in-depth about the Star of Bethlehem, read this article at Got Questions.