We present mapping of the vibration modes of an atomic force microscope cantilever with quantitative phase-resolved confocal microscopy. To this end, we combine synthetic optical holography with fast light detection at MHz frequencies to produce three-dimensional data sets, from which we reconstruct time-resolved amplitude and phase maps. Excitation at a single frequency and subsequent demodulation of the data yields the mode structure of the selected (vertical) oscillation mode. We also present a spectroscopic imaging modality, where broadband pulses simultaneously excite a variety of vibration modes from 40 kHz to 1 MHz that are all mapped in a single image acquisition in five minutes time. Our method is technically simple and fast because special modulated light sources as employed in stroboscopic methods for vibrational mapping are not needed, rather the maximum resolvable frequency is determined by the response time of the photodetector. We thus envision our technique for the measurement of mechanical vibration modes of micromechanical systems at up to GHz frequencies.