Polycrystalline (PC) and single crystalline (SC) samples of La0.7Ca0.3CoO3 (LCCO) with the perovskite structure were synthesized by conventional solid-state reaction and the floating-zone growth method. We conducted isothermal magnetization measurements of the PC and SC samples at temperatures from 2.8 K to 140 K, and evaluated the magnetic entropy change (−ΔSM) under zero field cooling (ZFC) and field cooling (FC) to 2.8 K. An interesting result has been obtained, where the −ΔSM–T curves in the low temperature range show totally different features between the ZFC and FC cooling procedures. The −ΔSM shows a large inverse irreversibility value for the ZFC process, while the −ΔSM also shows a normal positive value, but one that is slightly larger at 2.8 K for the FC process for both samples. We also present the results of a comprehensive investigation of the magnetic properties of the LCCO system. Systematic measurements have been conducted on DC magnetization, AC susceptibility, and exchange-bias. These findings suggest that complex structural phases, including ferromagnetic and spin-glass/cluster-spin-glass (SG/CSG)) states and their transitions, exist in PC samples, while there is a much simpler magnetic phase regime in SC samples. It was also of interest to discover that the CSG induced a magnetic field memory effect and an exchange-bias-like effect.