Day of Atonement

And when he has made an end of atoning for the Holy Place and the tent of meeting and the altar, he shall present the live goat. ²¹ And Aaron shall lay both his hands on the head of the live goat, and confess over it all the iniquities of the people of Israel, and all their transgressions, all their sins. And he shall put them on the head of the goat and send it away into the wilderness by the hand of a man who is in readiness. ²² The goat shall bear all their iniquities on itself to a remote area, and he shall let the goat go free in the wilderness. (Leviticus 16:20-22)

This is part of the ritual for the Day of Atonement. After the High Priest has sacrificed one goat as a sin offering for the people, he then takes another goat to symbolize the carrying away of the people's sins. Together the goats signify Jesus' atoning work on the cross. Upon the cross he was sacrificed as a sin offering for us. Upon the cross he bore our sins and carried them away into the "wilderness." 

God explains what these acts on the Day of Atonement mean. In verse 30 he says, "For on this day shall atonement be made for you to cleanse you. You shall be clean before the Lord from all your sins." What the Day of Atonement signified, Jesus accomplished in fact. His atonement has made us clean before the Lord from all our sins.