Unmanned aerial vehicles (UAVs) suffer from sensor drifts in GPS denied environments, which can lead to potentially dangerous situations. To avoid intolerable sensor drifts in the presence of GPS spoofing attacks, we propose a safety constrained control framework that adapts the UAV at a path re-planning level to support resilient state estimation against GPS spoofing attacks. The attack detector is used to detect GPS spoofing attacks and provides a switching criterion between the robust control mode and emergency control mode. An attacker location tracker (ALT) is developed to track the attacker's location and estimate the spoofing device's output power by the unscented Kalman filter (UKF) with sliding window outputs. Using the estimates from ALT, we design an escape controller (ESC) based on the model predictive controller (MPC) such that the UAV escapes from the effective range of the spoofing device within the escape time.