When a part of light emitted by a laser is back-reflected or back-scattered from an external target and re-enters the laser cavity, both the laser intensity and its wavelength can be modulated. This is so called Self-Mixing Effect (SME). An SME based displacement sensor is featured with compact physical setup and interferometric measurement resolution. However, when a target has a surface with very low reflectivity, the sensing signal is very weak and noisy. The system may even lose its sensing capability. It is desired to develop a method for improving the sensing performance in this case. We propose a configuration of dual external cavities for the SME based sensing system, where one of the cavities is used to provide pre-feedback for improving the sensing sensitivity and the other is related to the target to be measured. Firstly, a sensing model is derived for the proposed configuration and shows that the magnitude of the sensing signal can be enhanced by setting the pre-feedback cavity with suitable location and high optical feedback strength. Then experimental investigations are conducted to demonstrate the feasibility of the proposed approach. This paper provides an important guidance for designing an SME based sensor with high sensitivity.