In the discontinuous Galerkin time-domain (DGTD) simulation of a transient electromagnetic (EM) problem, the polynomial orders in the solution domain can be adjusted locally and dynamically in real time of the simulation to achieve a similar spatial resolution with a reduced computational cost compared to the simulation with uniformly high-order polynomials. The resulting dynamic p-adaptation technique has been shown to be very flexible and efficient. However, the time step size of an explicit time integration method is limited by the element with the smallest size and the highest polynomial order, which can be unnecessarily small for the elements with larger sizes or lower polynomial orders. To alleviate the global constraint on the time step size, a multi rate time integration technique, which permits different time step sizes in different elements, is adopted in this work to enhance the efficiency of the p-adaptive DG TD method. An EM scattering problem is given as an example to demonstrate the flexibility and efficiency of the proposed method.