The discrete element method (DEM) was developed about 35 years ago to simulate particulate interaction of granular materials. However, DEM is a computationally expensive method despite significant algorithmic developments that have been made to enhance the computational performance. In this study, a new simulation approach is proposed that runs nearly as fast as the simulation methods used in the computer graphics area while keeping the simulation fidelity required for engineering applications. The proposed impulse-based DEM (iDEM) uses collision impulse instead of contact force, and directly handles velocity while bypassing integration at the acceleration level. Contact force is not an integral part of simulation, but retrieved via an original proposed formulation. The method is robust, numerically stable and results in significant speed-up of almost two orders of magnitude relative to conventional DEM, which makes iDEM as a suitable tool for large scale simulation of granular materials.