The prestressed concrete sleepers (or railroad ties), which are installed in railway track systems as the crosstie beam support, are designed to carry and transfer the wheel loads from the rails to the ground. It is well known that railway tracks are subject to impact loading conditions, which are attributable to the train operations with either wheel or rail abnormalities such as flat wheels, dipped rails, etc. These loads are of very high magnitude but short duration. In addition, there exists the potential of repeated load experience during the design life of prestressed concrete sleepers. Prestressed concrete has played a significant role in maintaining the high endurance of sleepers subjected to low to moderate repeated impact loads. In spite of the common use of prestressed concrete sleepers in railway tracks, their impact response and behaviour under repetitions of severe impact loads are not deeply appreciated nor taken into consideration in design. This experimental investigation was aimed at understanding the residual capacity of prestressed concrete sleepers in railway track structures under ultimate impact loading, in order to develop state of the art limit states design concepts for prestressed concrete sleepers. A high-capacity drop weight impact testing machine was constructed at the University of Wollongong to achieve this purpose. A series of severe impact tests on in-situ prestressed concrete sleepers was carried out, ranging from low to high impact magnitudes. The impact energy was evaluated in relation to the drop heights. The impact-damaged sleepers were re-tested under static conditions in order to evaluate the residual fracture toughness in accordance with the Australian Standard. It was found that a concrete sleeper damaged by an impact load could possess significant reserve capacity sufficient for resisting about 1.05 to 1.10 times the design axle loads. The applied impact energy and residual fracture toughness under different magnitudes of impacts are highlighted in this paper. The effects of track environment including soft and hard tracks are also presented together with a design guidance related to the serviceability and ultimate limit states design.