In an ongoing research project focusing on improving learning in first year engineering mechanics, a framework for engineering mechanics knowledge has been identified. The framework has been applied to break down and categorise common mistakes made by students at four separate institutions to find out where students are struggling in their efforts to learn statics and dynamics. The framework separates knowledge into factual, procedural, conceptual, and principle areas in a semi hierarchical manner. In using this framework, it has become clearly evident that the marks students are awarded for their work tend to be biased towards procedural knowledge, rather than conceptual knowledge as one might expect for an introductory course. The implication here is that students make most of their mistakes in the problem solving procedures for which most marks are awarded. We propose in our efforts to encourage a deep conceptual understanding needed for further study in engineering mechanics, we may be inadvertently encouraging surface procedural knowledge.