Under the Second Charter issued by James II in 1609, Virginia's western boundary extended "up into the land, throughout from sea to sea, west and northwest."1
The colonial government began to establish local governments in 1734/35 (at that time, the "new year" started on March 1), but waited 100 more years before including any of the far western lands within any county. Governor Spotswood and then Governor Gooch were successful in inviting settlers to come into the Shenandoah Valley. Orange County was needed because there finally was a demand for a local court to confirm the acquisition and transfer of land claims west of the Blue Ridge.
In 1734, the General Assembly divided Spotsylvania County and included all the colony's territory west of the Blue Ridge within the new Orange County. It was named after William of Orange, who had ruled initially with Queen Mary and then solo as King William III.

Orange County originally was bounded by the Rappahannock River on the north, the Rapidan River on the south, and on the west by the "utmost limits of Virginia"
Source: Memoir of Col. Joshua Fry (p.26)
To encourage settlers to move to the western lands, those who arrived in Orange County by January 1 of 1734/35 were exempt from taxes for the next three years. The delay made it difficult to start the new county's operations, limiting its ability to raise funds for those three years.2
The county boundaries extended "northerly by the grant of Lord Fairfax and westerly by the utmost limits of Virginia." The vague description reflected the confusion created because the boundaries of the Fairfax grant had not been defined yet.3

Gordonsville in 1878
Source: David Rumsey Map Collection, OW Gray & Son Atlas - Gray's New Map of Gordonsville and Harisonburg, Virginia (1878)

Orange Court House in 1878
Source: David Rumsey Map Collection, OW Gray & Son Atlas - Gray's New Map of Orange Court House, Virginia (1878)

since 1845, wings were added to Montpelier - and then removed to restore its appearance to the days when President Madison lived there
Source: Historical collections of Virginia, Montpelier, the seat of President Madison (p.422)

Gordonsville served as a hospital center for the Confederacy during the Civil War, because railroads could transport the wounded there from battlefields in northern and central Virginia
Source: Library of Congress, Atlas of the War of the Rebellion, Southeastern Virginia and Fort Monroe, Va. (1892)