Recently, sparse inverse covariance estimation (SICE) technique has been employed to model functional brain connectivity. The inverse covariance matrix (SICE matrix in short) estimated for each subject is used as a representation of brain connectivity to discriminate Alzheimers disease from normal controls. However, we observed that direct use of the SICE matrix does not necessarily give satisfying discrimination, due to its high dimensionality and the scarcity of training subjects. Looking into this problem, we argue that the intrinsic dimensionality of these SICE matrices shall be much lower, considering i) an SICE matrix resides on a Riemannian manifold of symmetric positive definiteness (SPD) matrices, and ii) human brains share common patterns of connectivity across subjects. Therefore, we propose to employ manifold-based similarity measures and kernel-based PCA to extract principal connectivity components as a compact representation of brain network. Moreover, to cater for the requirement of both discrimination and interpretation in neuroimage analysis, we develop a novel pre-image estimation algorithm to make the obtained connectivity components anatomically interpretable. To verify the efficacy of our method and gain insights into SICE based brain networks, we conduct extensive experimental study on synthetic data and real rs-fMRI data from the ADNI data set. Our method outperforms the comparable methods and improves the classification accuracy significantly.