It is well known that when coarse-grained metals undergo severe plastic deformation to be transformed into nano-grained metals, their ductility is reduced. However, there are no ductile fracture criteria developed based on grain refinement. In this paper, we propose a new relationship between ductile fracture and grain refinement during deformation, considering factors besides void nucleation and growth. Ultrafine-grained Al-Mg alloy sheets were fabricated using different rolling techniques at room and cryogenic temperatures. It is proposed for the first time that features of the microstructure near the fracture surface can be used to explain the ductile fracture post necking directly. We found that as grains are refined to a nano size which approaches the theoretical minimum achievable value, the material becomes brittle at the shear band zone. This may explain the tendency for ductile fracture in metals under plastic deformation.