Quantitative correlation between intrinsic coercivity and grain boundaries in three dimensions is critical to further improve the performance of sintered Nd-Fe-B permanent magnets. Here, we quantitatively reveal the local composition variation across and especially along grain boundaries using the powerful atomic-scale analysis technique known as atom probe tomography. We also estimate the saturation magnetization, magnetocrystalline anisotropy constant, and exchange stiffness of the grain boundaries on the basis of the experimentally determined structure and composition. Finally, using micromagnetic simulations, we quantify the intrinsic coercivity degradation caused by inhomogeneous grain boundaries. This approach can be applied to other magnetic materials for the analysis and optimization of magnetic properties.