The castle is situated on a naturally protected slope, at the top of a steep hill above the Lešnica stream.

It was first mentioned in 1300 in relation to its masters, the Altenburgs. In 1350, Jakob Altenburg sold the castle to the Ortenburg family; in 1418, the Counts of Celje inherited it and confided its managing to Jurij Scheyer and Peter Obračan who later bought the castle. Among its owners, the family Breckerfeld has also been mentioned; Janez Sigismund (1689-1760) who wrote about his homeland regarding the economic, political, legislative and genealogic aspects, leaving many manuscripts dealing with the states and nobility of Slovenian land and its regions as well as a dictionary of four main European languages. The lexicographer and topographer Franc Anton Breckerfeld (1740-1806) wrote the Appendix to Valvasor's Topography and the Novo mesto Topography. In the 19th century, the castle was inherited by the Counts of Marghera di Commandona; they gave it as a gift to the monastery of the Brothers of Mercy between the World War I and II.

The oldest part of the castle has an original early Gothic tower from the 13th century, yet in the 14th and the 15th century, two flanking residential wings were added, and a connecting arcade staircase from the 16th century. In 1632, an early Baroque chapel was built inside the castle outline; it was rebuilt in the 1930ies. The castle was burnt down during World War II and partially rebuilt in the post-war period.

The castle was declared a cultural monument of local importance by the Act of Declaration of Natural, Cultural and Historic Monuments in the Novo mesto Community, Uradni list RS No. 38 / 1992.