This paper proposes a novel modulation technique and a new balancing control strategy for a single-phase five-level flying-capacitor (FC)-based active-neutral-point-clamped (ANPC) converter. The proposed modulator can control the FC voltage to follow the requested reference value and simultaneously generate the required ac output voltage regardless of the values of the dc capacitor voltages of the converter. By implementing this method, smaller values of the dc-link capacitor and FC can be used even in applications that could experience ripple or transient in the capacitor voltage. In a single-phase five-level ANPC converter applications, where the capacitors can experience pulsation power and dc-link balancing issues, such as grid-connected photovoltaic system, the selection of the reference voltage value for the FC can play an important role to balance the average values of the dc-link capacitor voltage. The proposed new control strategy uses a new reference voltage for the FC to be applied by the new modulator to have an average balanced dc-link voltages as well as an ac output voltage with good power quality. Simulation studies and experimental results demonstrate the effectiveness of the proposed modulation technique and control strategy even with relatively small dc capacitors to produce high-quality output voltage and current waveforms while maintaining an average balanced dc-link voltages.