Eye state recognition is a key step in fatigue detection method. However, factors such as occlusion of different types of glasses and changes in lighting conditions may have some impact on eye state recognition. In order to solve these problems, a driver's eye state recognition method based on deep learning is proposed. Firstly, the driver's face images are acquired using an infrared acquisition device. Secondly the multi-task cascaded convolution neural networks are used to detect the face bounding box and feature points of the driver's face image, and then the eye regions are extracted. Finally the Convolution Neural Network (CNN) is adopted to identify the open and closed state of the eyes. Experimental result shows that the proposed method can accurately identify the state of eyes and help to calculate the fatigue parameters of drivers.