Spectroscopy is a fundamental diagnostic technique in physical sciences with widespread application. Multi-order slitless imaging spectroscopy has been recently proposed to overcome the limitations of traditional spectrographs, in particular their small instantaneous field of view. Since an inversion is required to infer the physical parameters of interest from slitless spectroscopic measurements, a rigorous theory is essential for quantitative characterization of their performance. In this paper we develop such a theory using the Cramer-Rao lower bounds for the physical parameters of interest, which are derived in terms of important instrument design considerations including the spectral orders to measure, dispersion scale, signal-to-noise ratio, and number of pixels. Our treatment provides a framework for exploring the optimal choices of these design considerations. We illustrate these concepts for an application in EUV solar spectroscopy.