Deep learning methods for image processing are rapidly advancing and imaging techniques have become a standard for classification and quantification in agriculture. However, agricultural datasets assembled by domain experts are still comparably small. The global wheat consortium brings together this domain knowledge to assemble a large training set. We aim is to define a balanced dataset for training and validation containing the most relevant features observable for field-grown wheat. Image information will be enhanced by metadata, such as the developmental stage, genotype or agricultural treatment.