Young's modulus (denoted by E) is an intrinsic and valuable quantity for appropriate estimation of material performance. E can be obtained from the resonant frequency from the vibration of the tested material sample excited by the external forces. In this paper, we present a system design for determining the parameter by using an optical feedback self-mixing interferometry (OFSMI) system. An OFSMI system consists of a laser diode (LD), a micro-lens and an external target. The material sample to be tested is used as the external target. The vibration of the sample causes the variation of the length of the external cavity, and then causes a modulated laser power of the LD. The modulation contains the vibration information of the tested samples. The OFSMI system can achieve high measurement accuracy with an extremely simple and inexpensive set-up, thus can be thought as a good candidate for the evaluation of material properties. The system design in this work includes the mechanical part for holding and exciting the tested samples and the optical part for picking up the vibration information from the samples. In order to accurately determinate E, the exclusive supporting system is used for holding the material sample of specific dimension. The complex waveform of the LD output power is studied and simulated by using our proposed system model. The proposed method is verified by simulation and experiments, and satisfied accuracy of the experiments is achieved.