Fault tolerant adaptive filters (FTAF) have previously been enabled by exploiting the inherent learning capability of an adaptive process to recover from transient and fixed error conditions. Two drawbacks of this method are the long recovery time during which the system does not produce any useful outputs and for some implementations, the necessity of performing computations in the transform domain which limit applicability. In this work we use idea of combinations of adaptive filters and propose FTAF's which can guarantee a minimum level of overall system performance under error conditions. We investigate a combination scheme with respect to its overall mean square error (MSE) behavior. The fault tolerance capability of the proposed method may be useful in systems implemented in highly scaled CMOS process technologies where reliability is a concern.