We present a physics-based interpretation of slow-flow dynamics derived by empirical mode decomposition (EMD), which possesses equivalence or close correspondence with the analytical slow-flow model (e.g., by means of complexificationaveraging technique). Based on this observation, we develop a nonlinear system identification (NSI) method for direct analysis of measured time series, which is capable of analyzing strongly nonlinear, complex, multi-component systems. Nonlinear modal interactions can be described by means of intrinsic mode oscillators (IMOs), which are typically expressed as a set of linear, damped oscillators with nonhomogeneous terms that carry the nonlinear modal interactions at the different time scales of the dynamics. Both analytical and empirical slow flows are utilized to calculate the nonlinear modal interactions and validated by comparing the IMO solutions and the corresponding intrinsic mode functions obtained from EMD analysis. A main advantage of our proposed technique is that it is nonparametric, eliminating the necessity for a priori assumption of functional forms for stiffness and damping nonlinearities, which might restrict system identification. Hence, it is applicable to a broad range of linear as well as nonlinear dynamical systems, including systems with smooth or non-smooth nonlinearities (such as clearances, vibroimpacts, and dry friction), and strong (even nonlinearizable) or weak nonlinear effects.