The design and execution of effective and informative experiments in comparative studies on water treatment is challenging due to their complexity and multidisciplinarity. Often, environmental engineers and researchers carefully set up their experiments based on literature information, available equipment and time, analytical methods and experimental operations. However, because of time constraints but mainly missing insight, they overlook the value of preliminary experiments, as well as statistical and modeling techniques in experimental design. In this paper, the crucial roles of these overlooked techniques are highlighted in a practical protocol with a focus on comparative studies on water treatment optimization. By integrating a detailed experimental design, lab experiment execution, and advanced data analysis, more relevant conclusions and recommendations are likely to be delivered, hence, we can maximize the outputs of these precious and numerous experiments. The protocol underlines the crucial role of three key steps, including preliminary study, predictive modeling, and statistical analysis, which are strongly recommended to avoid suboptimal designs and even the failure of experiments, leading to wasted resources and disappointing results. The applicability and relevance of this protocol is demonstrated in a case study comparing the performance of conventional activated sludge and waste stabilization ponds in a shock load scenario. From that, it is advised that in the experimental design, the aim is to make best possible use of the statistical and modeling tools but not lose sight of a scientific understanding of the water treatment processes and practical feasibility.