In this study, 120–144 commercial varieties and breeding lines were assessed for grain size attributes including plump grain (>2.8 mm) and retention (>2.5 mm+>2.8 mm). Grain samples were produced from replicated trials at 25 sites across four years. Climatic conditions varied between years as well as between sites. Several of the trial sites were irrigated while the remaining were produced under dryland conditions. A number of the dryland sites suffered severe drought stress. The grain size data was analysed for genetic (G), environmental (E) and genotype by environment (G×E) interactions. All analyses included maturity as a covariate. The genetic effect on grain size was greater than environmental or maturity effects despite some sites suffering terminal moisture stress. The model was used to calculate heritability values for each site used in the study. These values ranged from 89 to 98% for plump grain and 88 to 96% for retention. The results demonstrated that removing the sources of non-heritable variation, such as maturity and field effects, can improve genetic estimates of the retention and plump grain fractions. By partitioning all variance components, and thereby having more robust estimates of genetic differences, plant breeders can have greater confidence in selecting barley genotypes which maintain large, stable grain size across a range of environments.