The first Isle of Man Times was launched by William Shirrefs in 1847. He had established other newspapers at his company, the Manx Steam Press, during the 1840s, exploiting the extension of postal privilege to send newspapers free to mainland Britain. The imposition of a standard letter postage to the mainland in April 1849 resulted in the winding up of Shirrefs' business in July 1849.

James Brown re-launched the title on 4 May 1861. He had moved to the Isle of Man in 1846, and set up his own printing office. By 1854, he was the printer/publisher of the Isle of Man Advertising Circular . Brown used the Isle of Man Times to campaign for the reform of the House of Keys (similar to the House of Commons), which was self-elected and unrepresentative. In 1864, the Keys summoned Brown for libel and contempt, and he was sentenced to six months in prison. Brown continued to run the Isle of Man Times , writing editorials in support of freedom of the press. The newspaper expanded its content over the years, from four pages (twenty-four columns) in 1861 to eight pages (forty-eight columns) in 1867, with sixty-four columns being achieved in 1875. In 1888, a second weekly issue was begun, on Tuesdays.

In 1870, Mitchell's Press Directory wrote of the paper: "Advocates the rights of all parties without favour or affection. The local news of the island is given at length, with full reports of public meetings, general intelligence, etc." The price of the Isle of Man Times was 2d. Also published in competition were the Manx Sun , a Conservative newspaper (Saturdays, price 3d), and Mona's Herald , a Liberal paper (Wednesdays, price 2d).

James Brown died in 1881 and his son, John Archibald Brown, took over the running of the family business. The Isle of Man Times continued to be critical of authority, but became one of the dominant newspapers on the island. It remained in the hands of the family until 1958, and continued publication until 1987.