We demonstrate amplification of picosecond laser pulses to 40 mJ at a 2 kHz pulse repetition frequency (PRF) from a two-stage cryogenic chirped-pulse Yb:YAG amplifier, composed of a regenerative amplifier (RGA) and a two-pass booster amplifier. The RGA produces 8.2 mJ of energy at 2 kHz PRF and 13.2 mJ at 1 kHz PRF with excellent energy stability (~0.3% rms) and beam quality ( M2<1.1 ). Pulse stretching and compression are achieved by using a chirped fiber Bragg grating and a multilayer dielectric grating pair, respectively. Compressed 15 ps pulses from the RGA are obtained with a throughput efficiency of ~80% (~6.5 mJ for 2 kHz ). The booster amplifier further amplifies the pulses to 40 mJ at 2 kHz PRF, and ~32 mJ, ~15 ps pulses are expected after compression. The amplifier chain seeded from a femtosecond Yb-fiber laser enables the optical self-synchronization between signal and pump in optical parametric chirped-pulse amplifier applications.