Investigations of microstructural and hardness gradients in the heat-affected zone (HAZ) of quenched and tempered (Q&T) steels have indicated that peak hardness does not occur in the grain-coarsened heat-affected zone (GCHAZ) adjacent to the fusion boundary, which is typical of ferritic steels, but corresponds more closely to the grain-refined region (GRHAZ). This phenomenon, the displaced hardness peak (DHP) effect, is considered to arise when the hardenability of the steel is high enough to result in the same microstructure in the GC and GR heat-affected zones, except for significant refinement of the microstructure of the GRHAZ, which increases the hardness and strength above those of the GCHAZ. The current paper concentrates on the effect of grain size on hardness in the HAZ of a boron-containing low-carbon martensitic steel subjected to bead-on-plate welding. Thermal simulation experiments were used to clarify the relationship between prior austenite grain size and the hardness gradients in the actual HAZ. The simulation work demonstrated that peak hardness in simulation samples occurred in regions of lower austenite grain size, supporting the proposed origin of the DHP effect in actual welds. Implications regarding hydrogen-induced cold cracking (HICC) susceptibility of the GRHAZ are discussed.