The lower Yellow River irrigates more than three million hectares of cropland and supplies water for municipalities and major industries along the river downstream. The vast irrigated areas along the lower river constitute the central piece of the Huang-Huai-Hai plain, the breadbasket of China, producing more than 20 million metric tons of grain annually as well as cotton and oil crops. Despite its obvious socioeconomic importance, water supply shortages have been rather common for the lower Yellow River, due to natural hydroclimatic variability, overuse in the upstream reaches, and imperfect management of water infrastructures. The last two storage reservoirs on the main stem of the Yellow River, Sanmenxia and Xiaolangdi, play a key role in supplying water to the downstream, especially during the dry season. Effective operation of these two reservoirs is essential for ensuring timely and reliable water supply to the downstream irrigated areas and non-agricultural users. The reservoirs also undertake flood control and ice-flood control tasks, facilitating sediment-flushing in summer and forming of river ice cap with sufficient flow capacity underneath it in winter. These functions pose additional complexity in modeling irrigation and water supply management in the downstream with these two reservoirs. We formulate the water management problem using a dynamic programming model. The Discrete Differential Dynamic Programming algorithm is applied to refine model solutions and speed up model solving. The results indicate that water shortages occur frequently however at relatively small amount in most years. Nevertheless, much greater shortages can occur in major drought years, implying dramatic damages for irrigated agricultural and industries. Conjunctive use of surface and ground waters in the lower Yellow River irrigated areas and improved flexibility in operating major storage reservoirs in the upper reaches can improve water supply reliability in the lower Yellow River. Demand side management such as water-saving irrigation technologies and improved industrial water use efficiency may greatly enhance water use sustainability.