First mentioning of lords of Svibno dates back to 1169 when Konrad of Svibno is mentioned. Next mentioning of Konrad (himself or his son) is from 1175 and in the years 1182 and 1191 knight Henrik is mentioned. In the first half of the 13^th century his son, who carries the same name as his father, is mentioned. It is not until 1372 that the castle is explicitly referred to (vest, haws Sherffenberch). Valvasor, on the other hand, mentions that Svibno castle was built by Arnulf of Svibno already in 928 but this information is not very reliable because fortified constructed castles did not yet exist in that era and also there is no information of lords of Svibno.
For a certain period of time lords of Svibno had in posession or in lease many castles: Žebnik, Gamberk near Zagorje, Planina near Sevnica, Podsreda, Smlednik and others and they also had in posession feudal estates from Salzburg, Freising and Krško. Most famous among knight of Svibno is Viljem. In 1270 he commanded the nobles who alongside Filip, brother of Carinthian duke Ulrik III., fought against Bohemian king Otokar II. Otokar, who wanted to acquire Carinthian, Carniolan and Slovenian mark soon suppressed the revolt. He returned the estate ti Viljem, but in a year he revolted again. Viljem bravely died on a battlefield in 1293 in the battle at Grebinje.
The power of the lords of Svibno began to blunt in the 14^th century. They were losing one estate after another, some by pawning, some by selling, and a good part gradually went into the hands of skilful and ambicious lords of Celje. In 1349 they pawned their castle Svibno to lords of Celje even if the castle was accounted as ducal feud. The castle was dispossessed from the last of the lords of Svibno by Leopold III. of Habsburg and assigned a new residence for them in Maribor. Pavel Ramung, who got the castle from Henrik of Rogatec as pawn property was murdered by lords of Svibno who unexpectedly returned to the castle. To Viljem of Svibno and his wife was in 1396 allowed lifelong living on the castle, which remained in posession of provincial prince after the death of Viljem. The castle was at the time managed by castellans in the name of lords of Svibno.
In the second half of 15^th century many pieces of firing weaponry were handed to castle's caretaker Peter Obračan and hence it can be concluded that there were tumults in the Svibno area. Auerspergs were the biggest problem for Obračan, because they unrighteously took cattle from his people and burden them with various taxes.
In the begining of the 16^th century the owner of the casle was Jurij Turn, a man known for his cruelty and ruthlessness. When he was murdered in 1515 in peasant revolt his daughter Ana took over as a caretaker. From the year 1529 on regional leader of Styria, vicedom of Celje and landlord Ivan Ugnad is testified. In the meantime the castle was dilapidating which was also established by committee in 1575 and was used only as a depository at the time. In 1579 when the castle was re-examined by a committee, the Svidno dominion was leased for 20 years by provincial councillor of Carniola Krištof Gustičič but because of that had to assure 15.000 goldinars of loan to Felicita Lamberg. After Krištof's death the dominion was succeeded by his sons Andrej and Jurij. Jože Koropec who studied matter documents of Inner-Austrian court chamber and Vicedom archive of Carniola reports of unsuccesful attempt to renovate the castle in 1597. The family died-out by male line in the 19^th century.

The original castle had a character of regulat castle with palatium and bailey. This medieval core was later extended towards south on the slope of Ostri vrh. At the time the castle was additionaly fortified and protected by a new aouter defence wall and embedded renaissance towers. Nowadays the original form of the castle is barely recognizable from the ruins. Some of the damage on cultural monument was caused by the locals who leveled the top of stack with the castle remains an in the proces destroyed a big part of old walls. Together with three sketches Valvasor prepared unusualy extensive description of castle building along with some other details.

Some words about Svibno mansion. Lords of Svibno established a borough with the same name at the side of hill below their castle and according to dominions land register from around 1400 only 32 subjective families lived there; among professions of boroughers are scribe, guard, tollman and hunter. Subjective families had to perform varoius tasks like driving cereals to Ljubljana, gathering woods, fixong the fence and a few among them had to mow, drive woods and fodder but all of them had to do a special serfdom when requested. The mansion, of which no trace can be found nowadays, was located close to cart track which leads to a nearby church of St.Cross and pass the church to the castle ruins. Valvasor depicted it as a typical renaissance 3x7 axial, on square ground plan constructed single-storey building with balconies in the front, covered with steep roof with four watershoots.

Knight of Svibno also found their way into Slovenian literature. They entered it with Ostrovrhar, Rozamundino snubičenje ("Rozamunda's proposal") by Prešeren, story of Viljem Of Svibno was also subject of tragedy in verses Viljem Ostrovrhar by Anton Medved.
Resident of Svibno is very well known among admirers of medieval era, blacksmith Oblak.