High pressure torsion (HPT) is an efficient technique of producing ultrafine grained materials with exceptional small grain size. In this study, a crystal plasticity finite element method (CPFEM) model has been developed to investigate the plastic deformation behavior of pure aluminum single crystal during the HPT process. The simulation results show that, the distribution and evolution of the macroscopic plastic strain and the accumulative shear strain are similar. The value increases with the increase of the distance from the center as well as the number of revolution. The simulation is capable of reflecting the anisotropic characteristics of HPT deformation, a non-homogenous deformation along the circumference of the sample could be observed. At the early stage of HPT deformation, the critical resolved shear stress (CRSS) along the radial direction presents a rapid increase, followed by a moderate increase and then reaches the near-saturate state. As the HPT deformation proceeds, there is a relatively weak increase in the quasi-saturate value and the near-steady region expands gradually towards the sample center. The orientation changes during the HPT process with increasing applied strain predicted by the developed CPFEM model are also presented.