This article describes the responsive evaluation component of an educational leadership capacity-building initiative developed at one Australian university and implemented by three others. The project aimed to develop, implement and disseminate an innovative framework to address the national strategic goal to increase the pool of qualified educational leaders. The framework reflected principles of distributive leadership, featured individual action-learning plans and fostered engagement in a supportive, scholarly community. Evaluation was challenging on many fronts, which the qualitative and responsive approach of design-based research was used to address. An external evaluator joined the project team and adjustments based on feedback were implemented throughout the process. The leadership capacity development framework is described, and design-based research endorsed as a suitable methodology to evaluate innovative academic development programs.