This church once belonged to the former convent of San Hermenegildo, established in 1586 by Friar Nicolás de Jesús y María, which was known as the Carmen Descalzo Convent since it was home to members of said religious order.

The original building, whose church was built in 1605, was demolished in the early 18th century, and in 1730 Pedro de Ribera commissioned the construction of the contemporary building, which was completed in 1748 by José de Arredondo and Fausto Manso. The construction has a typical church layout designed following Madrid Baroque aesthetics: a Latin cross plan with a central nave and two lateral ones.

The façade, built in a Ribera style, has a central niche with an image of Nuestra Señora del Carmen created by French sculptor Roberto Michel. The original façade was altered in 1912 by architect Juan Moya e Idígoras to extend it outwards and upwards and adapt it to the proportions of the adjacent “Casa del párroco” (Parish priest house), also built around this period to celebrate the opening of Gran Vía.

In 1836, after the Mendizábal confiscation, the Carmelites were expelled and both the convent and the temple were left empty and given no other use. The convent housed the Military Administration Direction for several years but was knocked down to make way for the Teatro Apolo and then the Banco de Vizcaya building. The church, however, became the site for the Parish of San José, and continues to function as such at present.

A plaque on the façade indicates that the playwright, Lope de Vega, gave his first mass after being ordained as a priest in this temple, when it was part of the Saint Hermenegildo Convent.