James was the eldest son of James, 8th Lord Chandos of Sudeley, and Elizabeth the eldest daughter and co-heir of Sir Henry Barnard of Bridgnorth (Salop). His father was sent as Ambassador at Constantinople in 1680 and died 16th October 1714. The son was elected Member of Parliament for the City of Hereford in 1698 and sat for the same place until the accession of King George I when, on 19th October 1714, he was created Viscount Wilton and Earl of Caernarfon. On 30th April 1719, he was created Marquis of Caernarfon and Duke of Chandos. In 1707, he had been appointed Paymaster-General of the Forces Abroad, a lucrative office which he held until 1712. He employed his wealth in building a splendid house, Canons at Whitchurch, near Edgware (Middlesex), and began another, of which only two 'pavilions' were finished, in Cavendish Square. The last was discontinued upon his buying the Duke of Ormonde's house in St. James's Square. Three architects were employed and the Italian painters Purgotti and Paolucci. One of 'the ablest accountants in England' was appointed to superintend the expenses, which are said to have amounted to £200,000. Alexander Blackwell laid out the gardens. There was a magnificent chapel, in which was maintained a full choir. Handel spent two years at Canons. He composed twenty anthems for the service, and there produced his first English oratorio, 'Esther'.

In December 1731, Alexander Pope published his 'Epistle to Lord Burlington,' in which occurs the famous description of Timon's villa, and Timon was at once identified with the Duke of Chandos. It was noted that Chandos had made a present of £500 to Pope. In the year 1732, appeared a spurious edition of the epistle, to which Hogarth prefixed a caricature representing Pope bespattering the Duke's coach. Pope indignantly denied the report in a letter to John Gay, signed by his friend William Cleland, and published in the newspapers of the day. He denied it also in his private correspondence to Lord Oxford, Caryll and Aaron Hill. He inserted a compliment to Chandos in the epistle on the 'Characters of Men,' first published in February 1733: "Thus gracious Chandos is beloved at sight".

In spite of certain inapplicable details, there can be no doubt that Pope took some hints from Canons and should have anticipated the application. There is, however, no reason to suppose that he had received any favours from Chandos. A refusal to answer the charge would have been better than a denial which rather strengthened the general belief. The point is discussed in Mr. Courthope's introduction to the 'Epistle to Burlington'. Warburton, in a note to the edition of 1751, stated that some of Pope's lines were fulfilled by the speedy disappearance of Canons - thus, by an odd oversight, confirming the application which he denied.

Daniel Defoe, in his 'Tour through Great Britain' (1725) describes the splendours of Canons in terms which recall Timon's villa. He says that there were 120 persons in the household -though Pope tells Hill that there were not 100 servants - and says that the choir entertained them every day at dinner. A poem called 'Chandos; or, the Vision' (by Gildon), was published in 1717, and another, on the same subject, by S. Humphreys, in 1728. Chandos got into difficulties by speculative investments and, in 1734, Jonathan Swift, in his verses on 'the Duke and the Dean,' says that 'all he got by fraud is lost by stocks.' He accuses Chandos of neglecting an old friend on becoming 'beduked.' On 31st August 1734, he had asked Chandos to present some Irish records, formerly belonging to Lord Clarendon, Lord-Lieutenant in 1685, to the University of Dublin. The failure of the request probably annoyed him. Swift, in his 'Characters of the Court of Queen Anne,' had called Chandos 'a very worthy gentleman, but a great compiler with every court.'

In April 1721, the Duke was appointed Governor of the Charterhouse and, on 25 August, Lord-Lieutenant of Herefordshire and Radnorshire, offices to which he was again appointed in 1727 on the accession of George II. He was also Chancellor of the University of St. Andrews. He was thrice married: first, on 27th February 1697, to Mary (d.1712) the daughter of Sir Thomas Lake of Canons; secondly, to his 2nd cousin, Cassandra (d.1735) the daughter of Sir Francis Willoughby of Wollaton (Nottinghamshire); and thirdly, to Lydia Catherine the daughter of John Van Hatten and widow of Sir Thomas Davall of Ramsey (Essex). The 'Princely Chandos' died at Canons on 9th August 1744. He was buried under a gorgeous monument at Stanmore Parva, in the church which he had rebuilt in 1715, and was succeeded in the Dukedom by his second son, Henry, who was famous for buying his second wife from a Newbury ostler.

Canons was sold for its materials by auction upon the 1st Duke's death. One William Hallet built a house with some of them on the vaults of the old one. The staircase was re-erected in Chesterfield House and the statue of King George I helped, until 1873, to make Leicester Square hideous. The Dowager Duchess took up residence at Shaw House, near Newbury, a fine country property which her husband had purchased in 1721 but legal complications had prevented him from occupying until seven years later. The couple used to spend about six weeks of the year there, a convenient stopping place on their travels from London to Bath. The Duchess was buried in the adjoining parish church there upon her death in 1750 and her heraldic memorial slab can still be seen there, sadly relocated to outside the south wall.