The effect of different cultural lenses on reliability and validity in observational data: The example of Chinese immigrant parent-toddler dinner interactions

This study used a mixed methodology to investigate reliability, validity, and analysis level with Chinese immigrant observational data. European-American and Chinese coders quantitatively rated 755 minutes of Chinese immigrant parent-toddler dinner interactions on parental sensitivity, intrusiveness, detachment, negative affect, positive affect, and parents' confidence. They also gave qualitative coding rationales for their quantitative ratings. Analysis of quantitative ratings indicated systematic differences associated with coder ethnicity. Such differences could threaten the reliability between coders with different ethnicities. Analysis of qualitative rationales revealed different cultural links between concepts and behavior cues, which could challenge the cross-cultural validity of a macro-level coding system. When a coding system is incongruous with coders' cultural schemata, even after intensive training, the effect of culture cannot be eliminated. Implications of these findings for observational cultural research are discussed.