The first building on the site of the Royal Lodge was erected during the Commonwealth when Cromwell divided up the Great Park and sold it off as large estates. This was known as 'Watkins' House,' being the residence of John Watkins, a tenant of John Barry from Byfield Lodge (now Cumberland Lodge). When the Crown took back the park in 1671, Watkins' became the 'Garden House' for the chief gardener at what was, y then, the Great Lodge (now Cumberland Lodge). By 1750, it was known as the 'Dairy,' a large two-storeyed place with a hall, dining room, little hall and two parlours beneath the main 'alcove' bedroom, two further bedrooms and a passage closet. The occupant was probably the steward of the Duke of Cumberland, then resident at the Great Lodge, whose role was equivalent of Deputy Ranger. In 1764, this position was given to the Duke's architect, Thomas Sandby. By 1770, he was living at the Dairy and the rise of the building's importance began. A low single storey building had already been added to the main house and, in the 1790s, Sandby enlarged the complex still further; building a new two-storeyed south facing block with elegant bows flanking a pedimented and columned entrance porch as appears in paintings of the time. After his death, the park bailiff, Joseph Frost, took up residence in what was then known as the 'Deputy Ranger's Lodge'. However, in 1811, frost was ejected as the place was considered to fine a residence anyone beneath the rank of a gentleman.

At the time, the poor sick George III was installed at the Castle and Prince Regent needed a suitable country residence of his own. It was decided to refit Cumberland Lodge for him but, while work was in progress, he took up temporary residence at the Deputy Ranger's House. This was eminently suitable for riding, boating, fishing and other leisurely pursuits that the prince so enjoyed. However, even this house required a little work to make it fit for a prince. So, in 1812, John Nash was engaged to repair and rearrange the house and add a veranda for £2,750. Within a year mere alterations had ballooned into an extension that doubled the size of the building. The 60ft long façade became 170ft long, including a vast conservatory and 'eating room' (see illustration), with a new northern entrance between parlour and waiting room and adjoining offices and servants' quarters. The whole was thatched, so it became known as a 'cottage,' all be it probably the largest cottage ornée ever built. Building work continued throughout the following year with many proposals and changes until the bill amounted to some £30,000. The prince then spend £17,000 on furnishings; finally staying there for the first time during Ascot Week 1815.

By 1820, the prince had inherited the Crown as King George IV and, with Queen Caroline's trial dominating London society, he was spending more and more time at what was now termed the 'Royal Lodge'. Building work continued, of course. Jeffrey Wyatt took over as chief architect and switched the thatch for slate, as well as undertaking many other alterations, including converting a lodge into a chapel and adding more staff accommodation. However, these must have been trifles compared major works at Buckingham Palace and other Royal residences, which, between 1825 and 28, kept the King most at the Royal Lodge. He spent over £3,000 on new furnishings and moved in his best paintings. There were constant parties, with a private band playing in the conservatory, and the Duke of Wellington, Princess Lieven and, of course, the King's mistress, Lady Conyngham, were frequent visitors. In the last two years of the King's life, however, he was prevented from residing there for long periods due to the erection of a new kitchen, staircase and a dining room behind the conservatory, with a small octagonal 'tent room' adjoining.

When King William IV inherited the Throne in 1830, he pulled down the vast majority of the building, which had always suffered from damp and had additional structural problems. He retained only the new dining room, tent room and conservatory. Jeffrey Wyatt (now Wyatville) was again employed to patch up the scars on the remains. What was left became a popular spot with the King and Queen for al fresco dining whilst out riding in the park. Queen Adelaide fitted out the 'tent room' with draperies and pretty sofas and installed glass doors and windows; but they never spent much money on the place and, when Prince Albert came on the scene, he found it almost in a state of collapse.

Queen Victoria remembered visits to see her uncle at the Lodge and encouraged her husband to save the building from demolition, which he did with three years of essential works, including the erection of several bedrooms, from 1840 to 1842. His private secretary, George Anson, lived there for a while in this "compound summerhouse, tent and richly adorned country house" with splendid flowering plants in the conservatory that were reflected in the ornamental mirrors that filled the connecting dining room. By 1844, the Queen and Prince Albert were being urged to convert the house as their own country estate rather than go through the expense of purchasing Osborne on the Isle of Wight, but it was not to be.

In 1856, the lodge was turned into a 'tutorial house' for the young Prince Alfred, Duke of Edinburgh, who needed to concentrate on his studies for two months before he left for his Naval training in Portsmouth. Later, the Prince & Princess of Wales used it during Ascot Week and a more permanent residency was suggested. Major additions would be required at a cost of £10,000, but, although in 1866, the dining room was sub-divided and wooden accommodation for servants put up, the couple never moved in. In the late 19th and early 20th centuries, it became the home of various Royal equerries: Col. George Grant Gordon, Lord Bridport and Sir Arthur Ellis. In 1905 & 6, the latter himself paid for extensive alterations, including a first storey above the dining room. After his wife's death in 1917, various minor servants in the Royal household were in residence until 1931.

In that year, King George V offered the Lodge to his second son, the Duke of York, and his family. The Duke demolished the Georgian conservatory to make way for a new family wing, removed the partition from the old dining room to make it a new living room and made other adjustments that made the place fit for use as a Royal residence. His daughter, the present Queen, grew up there. The family moved out when the Duke became King George VI in 1936, but it once more became his widow's official home upon his death in 1952. As the Queen Mother, she lived there for fifty years. Two years after her death, Prince Andrew, the current Duke of York, moved there, with his daughters, from Sunninghill Park, and instigated a major programme of restoration.

The Royal Lodge is a private residence belonging to the Crown. It is hidden from view within its surrounding estate.