Soil moisture is highly variable in space and time, and deficits (i.e., droughts) play an important role in modulating crop yields. Limited hydroclimate and yield data, however, hamper drought impact monitoring and assessment at the farm field scale. This study demonstrates the potential of using field-scale soil moisture simulations to support high-resolution agricultural yield prediction and drought monitoring at the smallholder farm field scale. We present a multiscale modeling approach that combines HydroBlocks – a physically based hyper-resolution land surface model (LSM) – with machine learning. We used HydroBlocks to simulate root zone soil moisture and soil temperature in Zambia at 3 h 30 m resolution. These simulations, along with remotely sensed vegetation indices, meteorological data, and descriptors of the physical landscape (related to topography, land cover, and soils) were combined with district-level maize data to train a random forest (RF) model to predict maize yields at district and field scales (250 m). Our model predicted yields with an average testing coefficient of determination (R2) of 0.57 and mean absolute error (MAE) of 310 kg ha−1 using year-based cross-validation. Our predicted maize losses due to the 2015–2016 El Niño drought agreed well with losses reported by the Food and Agriculture Organization (FAO). Our results reveal that soil moisture is the strongest and most reliable predictor of maize yield, driving its spatial and temporal variability. Soil moisture was also a more effective indicator of drought impacts on crops than precipitation, soil and air temperatures, and remotely sensed normalized difference vegetation index (NDVI)-based drought indices. This study demonstrates how field-scale modeling can help bridge the spatial-scale gap between drought monitoring and agricultural impacts.