Cognitive deficits have been regarded as one of the most significant clinical symptoms of depressive disorder. Accumulating evidence has shown that apolipoprotein B (ApoB) levels, which are responsible for inducing neurodegeneration, may be involved in cognitive deficits. This study examines cognitive deficits, and the correlation of serum ApoB levels with cognitive deficits of depressive disorder. 90 depressive patients and 90 healthy controls with matched age and gender were recruited. Cognition was assessed using the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS). Serum ApoB levels in depressive patients were measured by immunoturbidimetric method. Our results showed that depressive patients had lower scores of cognition including RBANS total score and subscales of language and delayed memory (all, p < 0.001) than healthy controls after controlling for the variables. The differences in cognitive functions also passed Bonferroni corrections. Serum ApoB levels were negatively correlated with delayed memory score in depressive patients (r = −0.30, p = 0.01). Furthermore, stepwise multivariate regression analysis indicated that serum ApoB levels independently contributed to delayed memory in depressive patients (t = −2.68, p = 0.01). Our findings support that serum ApoB levels may be involved in delayed memory decline in depressive patients. Depressive patients also experience greater cognitive deficits, especially in delayed memory and language than healthy controls.