Sir, I do not know whether you will die on the gallows or of the pox
From The Art and Popular Culture Encyclopedia
The exchange between John Wilkes and John Montagu, 4th Earl of Sandwich ("Sir, I do not know whether you will die on the gallows or of the pox." "That depends, my lord, on whether I embrace your lordship's principles or your mistress.") is also attributed to Samuel Foote; the same story was told of Mirabeau, answering Cardinal Maury, during the French Revolution.
Yale Book of Quotations, p. 281-2 doubts; but it is ascribed to Wilkes by Henry Brougham, Lord Brougham and Vaux in 1840 (Historical sketches of statesmen who flourished in the time of George III, third series), who claims to have heard it from an ear-witness, and by Charles Marsh (Clubs of London) in 1828.