One of the key issues for local appearance based face recognition methods is that how to find the most discriminative facial areas. Most of the existing methods take the assumption that anatomical facial components, such as the eyes, nose, and mouth, are the most useful areas for recognition. Other more elaborate methods locate the most salient parts within the face according to a pre-specified criterion. In this paper, a novel method is proposed to identify the discriminative facial areas for face recognition. Unlike the existing methods that only analyze the given face, the proposed method identifies the distinctive areas of each individual’s face by its comparison to the general population. In particular, non-negative matrix factorization (NMF) is extended to learn a localized non-overlapping subspace representation of the facial patterns from a generic face image database. In the learned subspace, the degree of distinctiveness for any facial area is measured depends on the probability of this area is belong to a general face. For evaluation, the proposed method is tested on exaggerated face images and applied in exiting face recognition systems. Experimental results demonstrate the efficiency of the proposed method.