The notions of identity-based multi-proxy signature, proxy multi-signature and multi-proxy multisignature have been proposed to facilitate public key certificate management of these kinds of signatures by merely employing signer’s identities in place of the public keys and their certificates. In the literature, most identity-based multi-proxy signature, proxy multi-signature and multi-proxy multi-signature schemes are based on bilinear pairings. Without incorporating bilinear pairings, Tiwari and Padhye proposed an identity-based proxy multi-signature scheme in 2011. Subsequently, an identity-based multi-proxy multi-signature scheme was proposed byTiwari et al. in 2012. First, we review identity-based (multi)-proxy multi-signature schemes without bilinear pairings and show that unfortunately, they are insecure in their security models. Secondly, we propose an identity-based multi-proxy multi-signature scheme without bilinear pairings, where identity-based multi-proxy signature and proxy multi-signature schemes are its special cases. Then, we prove that they are secure under Rivest, Shamir and Adleman (RSA) assumption in the random oracle model by presenting a new Forking Lemma. The proposal and its special cases are the first identity-based multi-proxy signature, proxy multi-signature and multi-proxy multi-signature from RSA assumption.