The name of the village, and also of Letcombe Bassett, probably derives from the Saxon ‘Ledge Valley’, or possibly incorporates a personal name as ‘Leoda’s Valley’. It could also be Celtic Leito-Cwm, that is ‘Grey Valley’, or possibly Leito-Camp meaning ‘Grey Open Land on the Outskirts of a Roman Settlement’. The Roman site might be Belmont, at Wantage, or Segsbury Castle hillfort. An old legend says the name really stems from a battle at this latter site between Danes and Saxons, when the enemy’s blood ran down the valley and the villagers shouted, "Let it come! Let it come!"

Segsbury or Letcombe Castle is a huge Iron Age hillfort covering twenty-seven acres. The name ‘Segsbury’ may be partly Celtic and partly Saxon in origin. Sego-Byrig literally means ‘Powerful Fort’, but the two halves should not be translated together. To the Saxons it would have been ‘The Fort called (by the Celtic British) Seg. . .’ The Celtic suffix has been lost forever, but if the Saxons had made a straight translation, the name would have been Segedunum originally, like Wallsend in Northumberland. The deep hollow near Segsbury is known locally as the Devil’s Punchbowl. It was into this that Old Nick apparently disappeared after ploughing the Devil’s Ditch across the Downs.

The Regis suffix shows the manor was a Royal one. Its court rolls are thought to be the oldest in the country. The present 'Old Manor' is an early 17th century building with a late 17th century granary on staddle stones and an early 18th century dovecote. 'Letcombe Manor' that replaced it, in status, was a late Georgian building, now replaced by retirement homes. Antwick’s Manor (previously the 'Moat House') is largely a Victorian Jacobethan mansion built by ‘Boss’ Croker after making his fortune in New York. It stands on the site of a medieval hunting lodge, used by King John while riding the Downs, which survived with its moat and fortified gatehouse into the 18th century. This was at one time the home of John Estbury who founded the well-known almshouses at Lambourn. His aunt has a memorial brass in the parish church. Estbury was followed in his residence by one of the many branches of the Berkshire Hyde family. The relations of both families, the Fettiplaces, owned the minor manor of Woodhill and have a good baroque wall monument also in the church. This 15th century building has a massive 13th century tower. Internal treasures include remnants of 14th century stained glass, including a figure of Christ and some heraldic shields. Ashmole recorded a now lost figure of John Mermill with his hunting horn. An obelisk in the churchyard stands as a memorial to a Maori chief, George King Hipango, who died of tuberculosis at the vicarage whilst training to be a Christian missionary. He was only nineteen. Other notable buildings in the village include the Greyhound Inn, with its unusual early 19th century brickwork using black glazed headers and Flemish bond. From its steps, the Riot Act was once read, after some of the villagers got a little out of control one Bonfire Night in the early 1900s. The burning effigy of the unpopular owner of Antwick’s Manor, which they pushed to its gates on a cart, was apparently not best appreciated.