The WheatPheno dataset is a large-scale deep learning dataset constructed for the phenological monitoring of wheat. *** A two-year study was conducted at Baima Experimental Station (119◦18′71′′E, 31◦62′00′′N) of Nanjing Agricultural University, China. In 2020, a total of 960 experiment plots were cultivated consisting of 160 wheat cultivars, three nitrogen (N) fertilizer levels (N deficiency group, 0 kg/ha; medium group, 180 kg/ha; control group, 240 kg/ha), and two replications according to the split-plot design principle (Fig. 4(a)). Each plot was 1.5 m × 1.5 m in size, containing six rows of wheat with a row spacing of 0.25 m and 175 seeds in each row. The micro plots were spaced 0.5 m apart. In 2021, 1260 plots were planted under the same experimental setting as that in 2020, except each plot was 1.25 m × 1.5 m in size and covered five rows. The plots included 210 varieties, two N fertilizer levels (N deficiency group, 0 kg/ha; control group, 240 kg/ha), and three replications. *** A total of 240 RGB cameras (Reolink K2, Reolink Digital Technology Co., Ltd, China) were installed to continuously monitor the winter wheat phenophase. Each camera was mounted on an iron pole at 2 m above the ground, and the shooting angle was 45◦ from the horizon (Fig. 4(b)). Each camera automatically monitored four plots’ areas every hour from March 6 to May 25, 2021, and March 1 to May 26, 2022. All camera images were sent to a server in real-time through a wireless transmission module.