This paper presents adaptive robust quadratic program (QP) based control using control Lyapunov and barrier functions for nonlinear systems subject to time-varying and state-dependent uncertainties. An adaptive estimation law is proposed to estimate the pointwise value of the uncertainties with pre-computable estimation error bounds. The estimated uncertainty and the error bounds are then used to formulate a robust QP, which ensures that the actual uncertain system will not violate the safety constraints defined by the control barrier function. Additionally, the accuracy of the uncertainty estimation can be systematically improved by reducing the estimation sampling time, leading subsequently to reduced conservatism of the formulated robust QP. The proposed approach is validated in simulations on an adaptive cruise control problem and through comparisons with existing approaches.