After the Maha parinirvana of Gautama Buddha, according to the legend, the tooth relic was preserved in Kalinga and smuggled to the island by Princess Hemamali and her husband, Prince Dantha on the instructions of her father King Guhasiva. Hemmamali hid the relic in her hair on the way to the island. They landed on the island in Lankapattana during the reign of Sirimeghavanna of Anuradhapura (301-328) and handed over the tooth relic. The king enshrined it in Meghagiri Vihara (present-day Isurumuniya) in Anuradhapura. Safeguarding the relic was the responsibility of the monarch from then, therefore over the years, the custodianship of the relic came to symbolize the right to rule the island. Therefore, reigning monarchs built the tooth relic temples quite close to their royal residences, as was the case during the times of the Anuradhapura Kingdom, Kingdom of Polonnaruwa, and Kingdom of Dambadeniya. During the era of the Kingdom of Gampola, the relic was housed in Niyamgampaya Vihara. It is reported in the messenger poems such as Hamsa, Gira, and Selalihini that the temple of tooth relic was situated within the city of Sri Jayawardenepura Kotte when the Kingdom of Kotte was established there.

During the reign of Dharmapala of Kotte, the relic was moved and was hidden in Delgamuwa Vihara, Ratnapura, in a grinding stone. It was brought to Kandy by Hiripitiye Diyawadana Rala and Devanagala Rathnalankara Thera. King Vimaladharmasuriya I built a two-story building to deposit the tooth relic and the building is now gone. In 1603 when the Portuguese kingdom invaded Kandy, it was carried to Meda Mahanuwara in Dumbara. It was recovered in the time of Rajasinha II and it has been reported that he reinstated the original building or built a new templeThe present-day temple of the tooth was built by Vira Narendra Sinha.The octagonal Paththirippuwa and moat were added during the reign of Sri Vikrama Rajasinha. The royal architect Devendra Moolacharya is credited with building the Paththirippuwa. Originally it was used by the king for recreational activities and later it was offered to the tooth relic, it now houses the temple's library.

The brick wall which runs along the moat and the Bogambara, the lake is known as the water waves wall. Holes in this wall are built to light coconut oil lamps. The main entrance gate which lies over the moat is called Mahawahalkada. At the foot of Mahawahalkada steps, there is a Sandakada pahana (moonstone) which is carved in Kandyan architectural style. Mahawahalkada was totally destroyed in a 1998 bomb blast and rebuilt afterward along with Sandakada pahana other stone carvings. Elephants are depicted in stone on either side of the entrance. A Makara Torana and two guardian stones are placed on top of the staircase. The Hewisi drummers' chamber is situated in front of the main shrine. The two stories of the main shrine are known as "Palle Malaya" (lower floor) and "Udu Malaya" (upper floor) or "Weda Hitina Maligawa".The doors of the Weda Hitana Maligawa are Ivory carvings. The actual chamber in which the tooth relic is kept is known as the "Handun Kunama".