The magnesium alloy AZ31, which has undergone high-pressure torsion processing, was subjected to in situ annealing microbeam synchrotron high-energy X-ray diffraction and compared to the as-received rolled sheet material that was investigated through in situ neutron diffraction. While the latter only exhibits thermal expansion and minor recovery, the nanostructured specimen displays a complex evolution, including recovery, strong recrystallization, phase transformations, and various regimes of grain growth. Nanometer-scale grain sizes, determined using Williamson–Hall analysis, exhibit seamless growth, aligning with the transition to larger grains, as assessed through the occupancy of single-grain reflections on the diffraction rings. The study uncovers strain anomalies resulting from thermal expansion, segregation of Al atoms, and the kinetics of vacancy creation and annihilation. Notably, a substantial number of excess vacancies were generated through high-pressure torsion and maintained for driving the recrystallization and forming highly activated volumes for diffusion and phase precipitation during heating. The unsystematic scatter observed in the Williamson–Hall plot indicates high dislocation densities following severe plastic deformation, which significantly decrease during recrystallization. Subsequently, dislocations reappear during grain growth, likely in response to torque gradients in larger grains. It is worth noting that the characteristics of unsystematic scatter differ for dislocations created at high and low temperatures, underscoring the strong temperature dependence of slip system activation. Graphical Abstract: [Figure not available: see fulltext.]