The name Ilsley appears to be derived from Hilde-Laege meaning 'Place of Conflict'. One of the villages bearing this name is therefore thought to be the site of the Battle of Ashdown (AD 871) which is known to have taken place somewhere on the Downs. It was here that Prince (later King) Alfred won his magnificent victory over the invading Danish armies; though there are good reasons for supposing this took place in adjoining East Ilsley.

The Italian Archbishop, Marco Antonio de Dominis, apparently became a resident in the village in 1616 when he was appointed the rector. He had had a row with the Pope and turned to Protestantism for a while before eventually recanting. He was also made Dean of Windsor though, so he probably wasn't in Ilsley that often. He still managed to make himself unpopular however, searching local leases for legal loopholes which enabled him to extract some £1,600 from the tenants. The villagers eventually complained to the Church. When he returned to Rome, he was tortured by the Inquisition.

Prior to the Civil War, the villagers met on the Downs near this village to discuss the hardships imposed upon them by the King. They decided to petition him for exemption. In 1644, Charles I was actually entertained, at the Rectory here, by the rector, Dr. Godfrey Goodman. He paid dearly for his loyalty for he was ousted from his position by the civil authorities two years later. He later became Bishop of Gloucester. William of Orange and his Dutch troops also passed this way whilst on their march towards Reading and then London where he was to take the throne as William III.

West Ilsley was the original home of the Morland Brewery of Abingdon – now owned by Green King, but remembered in their Morland Original and Old Speckled Hen brands of beer. John Morland, a local farmer, started brewing in 1711 and his ale and porter became much sought after in London pubs. The business was only relocated to Abingdon in 1887.

In the mid 19th century, West Ilsley had its own village witch. She was said to turn herself into a hare and race across the Downs. One day, however, she was chased by some dogs. She managed to escape by jumping through the keyhole of her cottage door but, as she passed through, one of the dogs bit her. She was found by her neighbours, in her own form, sitting by the fire nursing a dog-bite! Thereafter she was never able to cast her magic again.