Human face recognition is disrupted by the reversal of luminance contrast polarity (ie photo negatives—see Galper 1970 Psychonomic Science 19 207–208; Johnston et al 1992 Perception 21 365–375), while recognition of other objects is less impacted (Nederhouser et al 2007 Vision Research 47 2134–2142; Subramaniam and Biederman 1997 Investigative Ophthalmology & Visual Science 38 998). This suggests that correct patterns of luminance contrast are important for facial coding. Here we investigate this further by minimising luminance contrast. We contrast peoples’ ability to categorise cars and faces when images vary in luminance and when images are altered to predominantly contain differences in colour (equiluminance). Eliminating luminance contrast had a greater adverse impact on facial classifications relative to car categorisations. This was true even though precautions were taken to equate visibility, and despite equal levels of performance when images contained luminance contrast. These results were not due to images containing markedly different spectra, as the effect persisted for facial images altered to match car images in this regard, and performance in both tasks dropped off proportionally with increasing levels of image blur. Finally, consistent with previous observations, we show that facial coding is not only adversely impacted at equiluminance but becomes even worse when the polarity of luminance contrast is reversed. Our data show that the correct pattern of luminance contrast is very important for facial coding. We suggest that this is related to the role of luminance contrast in signalling 3-D shape from shading.