Glenoid baseplate orientation in reverse shoulder arthroplasty (RSA) influences clinical outcomes, complications, and failure rates. Novel technologies have been produced to decrease performance heterogeneity of low and high-volume surgeons. This study aimed to determine novice and experienced shoulder surgeon's ability to accurately characterise glenoid component orientation in an intra-operative scenario. Glenoid baseplates were implanted in eight fresh frozen cadavers by novice surgical trainees. Glenoid baseplate version, inclination, augment rotation, and superior-inferior centre of rotation (COR) offset were then measured using in-person visual assessments by novice and experienced shoulder surgeons immediately after implantation. Glenoid orientation parameters were then measured using 3D CT scans with digitally reconstructed radiographs (DRRs) by two independent observers. Bland-Altman plots were produced to determine the accuracy of glenoid orientation using standard intraoperative assessment compared to postoperative 3D CT scan results. Visual assessment of glenoid baseplate orientation showed “poor” to “fair” correlation to 3D CT DRR measurements for both novice and experienced surgeon groups for all measured parameters. There was a clinically relevant, large discrepancy between intra-operative visual assessments and 3D CT DRR measurements for all parameters. Errors in visual assessment of up to 19.2 degrees of inclination and 8mm supero-inferior COR offset occurred. Experienced surgeons had greater measurement error than novices for all measured parameters. Intra-operative measurement errors in glenoid placement may reach unacceptable clinical limits. Kinesthetic input during implantation likely improves orientation understanding and has implications for hands-on learning.