In 1369 the immensely wealthy Sir Thomas Hungerford - John of Gaunt's steward and first Speaker of the House of Commons - purchased the manor of Farleigh and set about rebuilding and enlarging the castle.
Almost immediately he found himself in trouble, when he was charged with crenallating without a licence. But, having obtained a pardon in 1383, he was able to pass on a substantial stronghold to his son, Walter.
Having further enhanced the family seat, Walter was created Baron Hungerford in 1426, and thenceforward, the building became known as Farleigh Hungerford Castle.
Walter fought alongside Henry V at the Battle of Agincourt in 1415.
His grandson, Robert, although inheriting his fighting genes, lacked his ability to back the right side. He was captured in the last battle of the Hundred Years War, and spent seven years as a prisoner in France.
Returning to England, he joined the Lancastrian cause in the Wars of The Roses and, on 29th March 1641, following the virtual annihilation of Henry V1's army at the Battle of Towton, was captured and executed.
Farleigh Hungerford Castle then passed into the hands of Richard, Duke of Gloucester - later Richard III - who retained it until his defeat at the Battle of Bosworth Field in 1485, whereupon the new King, Henry V11, returned Farleigh Hungerford Castle to Robert's grandson, Walter Hungerford.
In 1516 Walter's son, Edward, inherited the estate and married Agnes, the widow of one John Cotell.
But, following Edward’s death in 1522, Lady Agnes Hungerford was arrested on suspicion of murdering her first husband.
It was alleged that she had had him strangled and his body burnt, in the kitchen furnace at Farleigh Hungerford Castle. Found guilty of murder, she was hanged at London's Tyburn.
Her stepson, another Walter, was married three times. Nothing is known of his first two wives, but his third spouse created a sensation when she wrote to Thomas Cromwell complaining that her husband had imprisoned her in a castle tower for the previous three years and had even tried to poison her. But since Walter and Cromwell were friends, the latter ignored Lady Hungerford’s plight.
Then - in 1540 - Thomas Cromwell fell from Royal favour and, shortly afterwards, Walter was executed on charges of treason and "unnatural vices."
Lady Hungerford emerged from her captivity and married Sir Robert Throckmorton, by whom she had six children.
Having passed to the Crown, Farleigh Hungerford Castle was sold back to Walter's son Edward, who kept alive the family's tradition of intrigue by accusing his wife of trying to poison him. The case was dismissed in court and Edward, having refused to pay his wife's legal costs, was sent to prison for debt.
But the curtain was fast descending upon the almost Shakespearean plot that is the history of this remarkable clan.
In the latter half of the 17th century another Sir Edward, whom posterity remembers as "the Hungerford waster", inherited the estate and proceeded to gamble away the family fortune.
He sold the castle in 1686 and, thereafter, it fell into ruin until only the bare ribs of the once magnificent pile remained.
The ghost of Lady Agnes Hungerford is still said to return here on occasion, and appear in the vicinity of the chapel, a serenely beautiful shade who flickers briefly in front of astonished witnesses, before fading into nothingness in those twilight moments when day turns into night.