In this paper, we address the problem of planning optimal zero moment point (ZMP) trajectories for the double support phase in bipedal gaits that alternate between single and double support. This is achieved by allowing pre- and post-actuation during the single support phases. Thus, we solve two coupled problems: exact tracking of a given desired ZMP trajectory in the pre- and post-phases (single support), and determination of the desired ZMP during the transition phase (double support). Both are solved while minimizing the overall control energy. We also provide a formal method to assess how the choice of desired ZMP trajectory during the single support phases impacts the overall energy expended during the footstep cycle. Although the obtained solution may not be physically feasible in general, it represents a benchmark to which alternative feasible solutions may be compared. Our approach generalizes previous results that consider only constant output in the preand post-phases e.g., allowing pre- and post-phase output from a family of polynomial splines. We evaluate the approach via simulations.