College Street was named after Sydney Grammar School that was located in the street.
The Sydney Public Free Grammar School opened in 1825 with Laurence Hynes Halloran as Head Master. Halloran had operated a private school in England, but had fled England in 1796 due to debts and after being accused of immorality. It subsequently emerged that his degrees were self-awarded.
Halloran eventually returned to Britain but was arrested for forgery and transported to the penal colony of New South Wales, arriving there in 1819. He was immediately granted a ticket-of-leave.
In 1830, Sydney College was founded. Sir Francis Forbes, Chief Justice, became President of the College and laid the foundation stone of the present building in College Street on 26 January 1830. In 1835, Sydney College opened in this building with W.T. Cape as Head Master. In 1842 he resigned and was succeeded by T.H. Braim. In 1850 Sydney College was closed.