Background Evidence is growing on the benefit of physical activity to improve well-being following a cancer diagnosis. This study examined changes in physical activity from pre to post diagnosis and explored this relationship with quality of life and depression. Methods Participants were recruited by posters and by letter of invitation. The questionnaire was completed by 59 prostate and 32 breast cancer survivors. Results Physical activity decreased by 72 minutes per week from pre to post diagnosis, although 20.9% reported having increased activity post diagnosis. Over 30% were considered depressed. Breast cancer participants who increased physical activity post diagnosis reported higher scores for Physical Wellbeing subscale (26 versus 21; F[1,29] = 5.19, p < .03), Emotional Wellbeing subscale (22 versus 19; F[1,30] = 4.57, p < .04) and Functional Wellbeing subscale (26 versus 19; F[1,30] = 9.03, p < .001). A greater proportion of participants taking part in no physical activity were depressed (55.6%; χ2 = 6.83, p < .04). Conclusion Over 25% of participants identified with emotional and/or well being problems, and more than half reported insufficient physical activity to yield benefits. Future research needs to gain a better understanding of why cancer survivors decrease their physical activity following a cancer diagnosis and what is necessary for them in order to retain or increase their physical activity.