Purpose Interobserver variability in target and OAR delineation can impact patient survival and toxicity incidence. Although this is widely understood, interobserver contouring variations are not regularly assessed in routine clinical practice due to the time burden on clinicians of conventional dummy-run studies. This work demonstrates how retrospective information from past treatment plans can guide improvements in interobserver contouring agreement through a time-conscious, evidence-based intervention. Methods Structure volumes, lengths (in craniocaudal dimension), and DVHs of 492 prostate cancer patients prescribed 60 Gy in 20 fractions were extracted from the Eclipse treatment planning system (Varian Medical Systems) using the system’s scripting API. Both moderate-risk (prostate only) and high-risk (prostate + pelvic lymph-nodes (PLN)) patients were included and stratified into separate groups for analysis. Interobserver variations between four treating oncologists and their dosimetric consequences were assessed using Student’s 2-sample t-tests. Significant variations were presented at an intervention meeting and used to inform new consensus contour definitions. The impact of the intervention on said variations was assessed one-year post-intervention, by repeating the analysis on 152 new plans. Results Pre-intervention, significant interobserver variations in prostate and PLN target delineation existed between oncologists, leading to significant variations in dose to nearby OARs. Inter and intraobserver variations also existed for rectum and penile bulb structures. Post-intervention, the interobserver concordance increased for all previously discordant structures. No significant interobserver variations persisted for penile bulb, rectum, or prostate structures. Dosimetric variations reduced post-intervention as well. PLN contouring variations also decreased dramatically, though some interobserver variations still existed for one oncologist relative to their peers. Conclusion Interobserver contouring agreement between oncologists increased following an intervention based on evidence gathered from a retrospective analysis of clinical data. Semi-regular review with this type of approach may aid improvements in intra and inter-institutional practice standardization and improve quality of care. This work was supported by the Fonds de Recherche du Quebec - Sante (FRQS) (Doctoral Training Award: H. Patrick).