Seismic site response analysis is commonly used to predict ground response due to local soil effects. An increasing number of downhole arrays are deployed to measure motions at the ground surface and within the soil profile and to provide a check on the accuracy of site response analysis models. Ad-hoc approaches and more recently system identification and optimization techniques are often adopted to adjust soil model properties to match field observations. However, these approaches are not always successful and are limited by the specific cyclic soil constitutive model used in the analysis. A novel inverse analysis framework, SelfSim (Self learning engineering simulations), to integrate site response analysis and field measurement is introduced. This framework uses downhole array measurements to extract the underlying soil behavior and develops a constitutive model of the soil. The resulting soil model, used in a site response analysis, provides correct ground response and can be used in the forward prediction of future earthquake events. The soil model can continuously evolve using additional field information. The performance of the algorithm is successfully demonstrated using two synthetic arrays and is currently being applied to field recordings.