Model-based feedforward-feedback tracking control has been shown to be one of the most effective methods for real-time hybrid simulation (RTHS). This approach is derived based on a linear time-invariant model of the closed-loop servo-hydraulic system. However, the closed-loop system is intrinsically nonlinear and time-varying when nonlinear experimental components are tested such as magnetorheological dampers. In this paper, an adaptive control scheme of a model-based feedforward-feedback controller is proposed to further improve the tracking performance of the actuator. This adaptive strategy is used to estimate the system parameters for the feedforward controller online during a test. The robust stability of this adaptive controller is provided by introducing Routh’s stability criteria and applying the parameter projection algorithm. The proposed control scheme is shown to attain better tracking performance. Finally, RTHS of a nine-story shear building controlled by a full-scale MR damper is carried out to verify the efficacy of the proposed control method. The adaptive feedforward-feedback control scheme is demonstrated to be effective for structural performance assessment using RTHS.