Moated Beaumys Castle stands next to the A33, just within the bounds of the parish. The original 13th century house was owned by Geoffrey Le Despencer, Lord of Martley in Worcestershire. The moat was dug for his nephew, Hugh Le Despencer, the favourite of Edward II. When disgraced by Queen Isabella in 1322, Hugh fled the court and Mortimer, her lover, raided many Despencer lands including Beaumys. It was later the home of the De La Beche Family. Sir Nicholas rose to the rank of Lord De la Beche and was made Constable of the Tower & Seneschal of Gascony. He oversaw the education of the Black Prince, but died childless in 1345. His widow, Margery, remarried twice in short succession and her husbands are believed to have died of the Black Death. It was while staying at Beaumys with Prince Lionel, and several other children of King Edward III, that this widowed lady was abducted by her lover. Sir John Dalton broke in with sixty-four Berkshire and Lincolnshire squires and made off with, the not so reluctant, Margery to Scotland. Edward III himself held the adjoining Swallowfield Manor in the 1350s when it became one of his Royal stud farms, supplying war horses for the Hundred Years War under the watchful eye of Edmund Rose and Sir John Brocas.

This latter manor was owned, in Edward the Confessor’s reign, by a Danish Huscarle named Sexi! Throughout most of the Middle Ages, the St. Johns were Lords of the Manor. Their main homes were at Stanton St. John (Oxon) and Lagham (Surrey), but King John did visit Roger St. John at Swallowfield in 1205 & 1206. Roger’s daughter-in-law, Emma married, as her second husband, Geoffrey Le Despencer from nearby Beaumys. He appears to have been more attached to Swallowfield than his predecessor. In 1253, Emma was given charge of King Henry III’s deaf and dumb baby daughter, Princess Katherine. At the age of two, the poor little girl became rather ill and was sent off to her surrogate-parent at Swallowfield. She had a baby goat from Windsor Great Park to play with and some other children for company. Katherine did not survive and died at Windsor Castle in 1257.

Emma’s son, John Le Despencer, one of the barons who rebelled against Henry III, lived at Swallowfield in his half-brother’s house. Despite the parish church’s east window suggesting a date of about 1130, he is known to have written to the Pope, in 1256, to complain of the distance he had to travel to church in Sonning and gained permission to build a place of worship of his own. His grave was found under the church floor during restoration work in 1869. John lay in a stone coffin with a large flat cross upon it. He was surrounded by cloves and a wooden dish, probably for salt, rested on his chest. Due to its ruinous state, the building was almost pulled down in the last century but, Charles Kingsley, the Vicar of Eversley (Hants) and a close friend of Lady Russell of Swallowfield Park, saw its merits and had it restored. Another friend of Lady Russell was Mary Russell Mitford, the author of Our Village. She retired to Swallowfield (which gains a mention in her book) to be nearer her friend and was later buried in the churchyard. Locals long remembered her books being pushed through the Berkshire lanes in a hand-cart, all the way from Three Mile Cross.

Swallowfield Park is a beautiful old manor house built for the 2nd Earl of Clarendon in about 1690, though much altered in 1820. The previous house had seen the parliamentary army camp in the park on 23rd October 1644, just four days before the 2nd Battle of Newbury. In the 18th century, the house was owned by Thomas Pitt, grandfather of the Prime Minister of that name who lived in Binfield. He was a successful merchant who became the President of Fort Madras for the British East India Company. While there, he was given the nick-name of Diamond Pitt after acquiring the now famous Regent (then Pitt) Diamond in 1701. Being the largest diamond in the World at the time (now the 15th largest), there were naturally many rumours concerning its origins. The diamond was said to have been found by a slave who hid it in a wound in his leg. Having made a fateful pact with a ship’s captain, the poor man was murdered and the diamond sold on to Pitt for £1,000. Alexander Pope (also of Binfield) thus wrote of him:

Asleep and naked as an Indian lay,
An honest factor stole a gem away;
He pledged it to the Knight, the Knight had wit,
So kept the diamond, and the rogue was bit.

Thomas, however, claimed that he had bought the diamond for £20,000 from an Indian Diamond Merchant. He eventually sold it to King Louis XV of France and it became part of the French Crown Jewels. Napolean used to wear it in his sword hilt, but today it is safely on display in the Louvre in Paris. With the proceeds of the sale, Pitt bought Swallowfield Park. The house is now split into retirement flats, but the grounds and some rooms are open to the public on midweek afternoons. One of the estate barns (now a private house) has an interesting old dovecote.