Near the Castle are the ruins of the Old Castle which dates back to the 11th century and belonged to the patriarchy of Aquileia. The Castle itself dates back to 1389, when the Wallsee family commanded the construction of a strong fortress. Over time, the Wallsee family disappeared and the Castle, after having been used as a prison, became the residence of the Hofer, Captains of Austria.
Mathias Hofer’s daughter marries Raimondo della Torre Valsassina who receives the Castle at the end of the 16th century. The Thurn-Hofer Valsassina family keeps the Castle which will officially be attributed to them by Emperor Leopold in 1660. Teresa, the last of the Thurn-Hofer Valsassina, marries Prince Egon zu Hohenlohe mid-19th century.
The Castle goes to the Thurn und Taxis family in 1875 when their daughter, Princess Marie zu Hohenlohe, marries Prince Alexander von Thurn und Taxis from the Czech branch of the House of Thurn and Taxis. It remains with the family to the present day with his great-great-grandson Prince Dimitri della Torre e Tasso, born in 1977.
The Castle and its park have been partially opened to the public since 2003.