The Kapitan Keling Mosque was first erected by the troops of the East India Company who were some of Penang's first Muslim settlers, in the late 1700s. Several years later, the growing Indian Muslim community required a more permanent mosque. In 1801, the community, led by their leader, Cauder Mydin Merican or also referred to as 'Kapitan Keling', was granted with an 18-acre lot. Made of brick, the original mosque was a single-storey structure, but over the years, the mosque has seen plenty of renovation works.

Surrounded by a low wall, this white-washed mosque is adorned with yellow Moghul-style domes and turrets. A madrassah (a place where religious classes are held) can also be found within the mosque's compound. The tall minaret, once used by the muezzin to deliver his call to prayers, is located on the corner of Buckingham and Pitt Street.