Brian Fairfax, who was the eldest son of Brian Fairfax, author of the Life of the Duke of Buckingham and other works, was born on the 11th of April 1676. He received his early education at Westminster School, where he entered as a Queen's Scholar, and from whence he went to Trinity College, Cambridge, taking the degrees of B.A. in 1697 and M.A. in 1700. He became a Fellow of his College in 1698. In 1723 he was appointed a Commissioner of the Customs, a post he held until his death on the 9th of January 1749.

Fairfax collected in his house in Panton Square a very valuable library, which, together with a considerable fortune, a gallery of pictures, a fine collection of Greek, Roman, and English coins and medals, and other curiosities, he bequeathed to his relative, the Hon. Robert Fairfax, of Leeds Castle, Kent, afterwards seventh Lord Fairfax. Robert Fairfax intended to sell the library by auction on the 26th of April 1756, and the seventeen following days; but after having advertised it, he privately disposed of it for two thousand pounds to his kinsman, Mr. Francis Child,[64] of Osterley Park, Isleworth, Middlesex, and the printed catalogues, with the exception of twenty, were suppressed.[65] The title to the catalogue of the intended sale reads: 'A Catalogue of the Entire and Valuable Library of the Honourable Bryan Fairfax, Esq., one of the Commissioners of His Majesty's Customs, Deceased: which will be sold by Auction, by Mr. Prestage, at his great room the end of Savile Row, next Conduit Street, Hanover Square. To begin selling on Monday, April 26, 1756, and to continue for seventeen days successively. Catalogues to be had at the Place of Sale, and at Mr. Barthoe's, Bookseller in Exeter Exchange in the Strand. Price Six-pence, pp. 68. 8°.' In a copy of the catalogue mentioned by Dibdin in his Bibliographical Decameron, the price at which each article was valued is given for the express purpose of the purchase of the whole by Mr. Child. Among the prices thus noted are those of the nine Caxtons which the library contained, which altogether amounted to thirty-three pounds, four shillings. The Recuyell of the Histories of Troye was valued at eight guineas, the Confessio Amantis at three pounds, and the Histories of King Arthur at two pounds, twelve shillings and sixpence. The prices obtained for these books at the sale of the Osterley library in 1885 were eighteen hundred and twenty pounds, eight hundred and ten pounds, and nineteen hundred and fifty pounds, respectively. The collection became part of the Osterley library, of which a catalogue was made in 1771 by Dr. Thomas Morell, assisted by the preceding labours of the Rev. Dr. Winchester. Only twenty-five copies of this catalogue were printed.

Brian Fairfax's pictures, statues, urns, and other antiquities were sold by auction on April the 6th and 7th, and the prints and drawings on May the 4th and 5th, 1756.

In 1819 the library passed by marriage into the family of the Earls of Jersey, and on the 6th of May 1885 and seven following days it was sold by Sotheby, Wilkinson and Hodge. The sale consisted of one thousand nine hundred and thirty-seven lots, which realised the large sum of thirteen thousand and seven pounds, nine shillings.