After implementation of a web-based application in a tertiary education environment, an expert review of this application was conducted. An analysis of the processes and results of the expert review of the web-based application is presented. The owners of the system implemented the recommendations of this expert review to resolve system issues. Through two rounds of expert evaluations, system modifications were made based on recommendations from the initial expert review. The concern that usability evaluations are used as a ‘cure-all solution’ to demonstrate to users that system owners are trying to resolve issues with the system is discussed. It should be understood by all those involved in the development and implementation of applications that usability evaluations are unlikely to provide the claimed benefits when issues are beyond the scope of the web-based application. Rather, in these cases a review of the overall business process in order to have a genuine positive impact is required.