Multi-component biological machines, comprising individual proteins with specialized functions, perform a variety of essential processes in cells. Once assembled, most such complexes are considered very stable, retaining individual constituents as long as required. However, rapid and frequent exchange of individual factors in a range of critical cellular assemblies, including DNA replication machineries, DNA transcription regulators and flagellar motors, has recently been observed. The high stability of a multi-protein complex may appear mutually exclusive with rapid subunit exchange. Here, we describe a multisite competitive exchange mechanism, based on simultaneous binding of a protein to multiple low-affinity sites. It explains how a component can be stably integrated into a complex in the absence of competing factors, while able to rapidly exchange in the presence of competing proteins. We provide a mathematical model for the mechanism and give analytical expressions for the stability of a pre-formed complex, in the absence and presence of competitors. Using typical binding kinetic parameters, we show that the mechanism is operational under physically realistic conditions. Thus, high stability and rapid exchange within a complex can be reconciled and this framework can be used to rationalize previous observations, qualitatively as well as quantitatively.