WiFi localization and tracking of indoor moving objects is an important problem in many contexts of ubiquitous buildings, first responder environments, and others. Previous approaches in WiFi-based indoor localization and tracking either assume prior knowledge of indoor environment or assume many data samples from location-fixed WiFi sources (i.e. anchor points). However, such assumptions are not always true, especially in emergency scenarios. This paper explores the possibility of real-time indoor localization and tracking without any knowledge of indoor environment and with real-time data samples from only few anchor points outside the building. By using a small set of synchronized directional antennas as outdoor anchor points to actively scan in various directions, a moving device inside the building can be localized and tracked from the received signal in real-time manner. The paper proposes an angle-of-arrival estimator for accurate localization and adaptive per-antenna angular scheduling for real-time indoor tracking. The validation results from real experiment and simulation yield effectiveness and accuracy of the proposed schemes.