Castle with strategically important location is built on territory with old settlement continuity witnessed by the remains of old castle place with moat at its foot above Drava bank. Its name is linked to the coat-of-arms of lords of Vurberk which has a worm (ger. Wurm) in it, mythical serpent - dragon. Castle is indirectly mentioned for the first time in 1238 when its castellan Wigand (Wigandus de Wrmberch) is mentioned even though it was probably built as early as in 12^th century because already in 1204 we encounter a dragon in a coat-of-arms of first known owner Amelrik from Hollenburg. First explicit mentioning of the castle dates to 1244 whet the castle is reffered to as Wuermwerg;at the time Amelrik of Stubenberg pawned the castle to Swiker of Humperk for the amount of 130 marks. Two years later, after Swiker's death the castle Vurberk - castrum in Wurmberch was sublet as a fee by archbishop of Salzburg to Hartnid of Ptuj in exchange for his hereditary estates in Lungav (Tamsweg) in Salzburg.
At the time of the war between Ottokar II. of Bohemia and Hungarians Vurberk has fallen into the hands of Hungary. In 1260s at the time of noblemen unrest king Ottokar of Bohemia supposedly gave the order to demolish also Vurberk. Rudolph of Habsburg who was king of Germany at the time took advantage of noblemens discontent. In the battle on Marchfeld Ottokar has lost and Styria along with other Ottokar's estates came under Habsburg's supremacy. Lords of Ptuj now rebuilt Vurberk. Vurberk remained as a property of Salzburg until 1438 ant then it was in 1441 passed over by hereditary contract to Friderik's sister Neža, married to Leutold of Stubenberg. It then remained as a possession of Stubenbergs until 1616 when Jurij of Stubenberg sold the dominion to Filibert Schranz noble Schranzenegg. In 1619 it was gained by Wolfgang Žiga baron noble Herberstein, in 1627 colonel baron Johan noble Wechsler and his daughter, »huda Liza« (»bad Liza«) sold the castle back to Herbersteins. When in 1715 Kristina Krescencija noble Herberstein married count Ignacij Marija Attems the castle became a property of this notable family of Styria and remained in their possession until 1885. The castle was the in civic hands until 1907 when it was again gained by Herbersteins. It was in their possession until the last war when it was a place for sanatorium of Russian red cross. At the time of occupation it was used as Nazi political school and on 22^nd February 1945 the castle was heavily damaged during the bombardment. Ruins were removed after the war. Nowadays the castle is mainly used for entertainment events because the "amphitheater" is located on its courtyard. Local tourist society also has a place in castle's bastion.

At the time of Turkish invasions the castle was an outpost between Krapina and Upper Styria where bonfires on the top of hills were used to warn of the approaching enemy. But Turks never conquered the castle, also not in 1532 when they were robbing and murdering in Slovenske Gorice. Of course the castle was properly prepared for such invasions. In castle's inventory from 1525 among other things, its weaponry is fully listed: in the armory were armours and also their belongings, other weapons were in small weaponry closet, in the house in front of a chapel spears and lances were prepared. Also mentioned are Polish snake (sort of a cannon), small cannons on wheels, two bombards and many kartauns and also other cannons with belonging ammunition..

Former image of the castle can only be reconstructed using old depictions, photos and descriptions. Four-winged castle core was built on rectangular ground plan and reinforced on south corner by round defence tower; castle prison was in the ground floor of the tower. The castle was build on bedrock and was two storeys high. On the ground floor mainly administration was located and 22 lordship's rooms were on the first floor and 14 on the second. Courtyard was on all four sides decorated with arcades and one of courtyard walls had Renaissance three-light window. A cistern with forged crown was standing in the middle of courtyard. On presumably older northeastern wing two plates were embedded with coat-of-arms of lords Stubenberg and on one of courtyard extensions was a year 1592 which maybe dated the complete Renaissane adaptation of the castle. Of older castle components in castle core a Late-Gothic chapel is proven, built by Gašper Stubenberg and consecrated on 26^th of August, 1510 by Pewerl, Levantine bishop.
Castle core was surrounded by Renaissance defence ring, defence walls with towers which partially remained until present day as the only part of former castle. Keyhole-shaped arrow-loops are in defence walls and once two cannons and two Polish snakes dating to 15^th century were built into the wall. This »artillery« was in use until 1814 when they were last used at a celebration of Victory over Napoleon. Later the cannons were transferred to Ptuj castle.
72 meters deep well was located on the courtyard which was according to a legend dug by two criminals or captured Turks and thus saving their lives. Water was pumped from the well with the help of the wheel powered by pedals - the only known specimen of its kind in Slovenia and a rarety also in wider European area. Stone staircase built in 19^th century led from outer courtyard to inner castle and they ended at the stone, rustic Renaissance entrance. Exposed tower seen on Vischer's depiction and primarily had a function of front fortification is not preserved anymore. A rich collection of old furniture, weaponry, torturing devices, art works and small equipment was once on the castle. Part of this equipment is now on Ptuj castle.

In the near past some energy points were discovered in the area surrounding the castle and their number is now already 31 and are visited by many people for healing purposes. Oral tradition is that during the reign of Herbersteins castle doctor sent his patient with lung illness to walk in the forest, growing on castle elevation, every day. They had to walk on certain paths leading among the trees and rest on designated places.