Existing video-based smoke detection methods often rely on the visual features extracted directly from the original frames. In the case of light smoke, the background is still visible and it deteriorates the quality of the features. This paper presents an approach to separating the smoke component from the background such that visual features can be extracted from the smoke component for reliable smoke detection. Specifically, an image is assumed to be a linear blending of a smoke component and a background image. Given a video frame and its background, the estimation of the blending parameter and the actual smoke component can be formulated as an optimization problem. Three methods based on different models for the smoke component are proposed to solve the optimization problem. Experimental results on synthesized and real video data have shown that the proposed approach can effectively separate the smoke component and the smoke detection performance is significantly improved by using the visual features extracted from the smoke component.