While consumer adoption of smart home devices continues to grow, privacy concerns reportedly remain a roadblock to mass adoption. However, it is unclear exactly how the interplay between privacy and other factors affect consumers' purchase decisions, and whether such considerations are held only by certain consumer groups but not others. In order to unpack the decision-making process of smart home device adoption, we conducted a mixed-method analysis using online survey data collected from 631 US participants. Our analysis uncovered motivators and blockers of purchase decisions, along with their relative importance. We found that consumers can be segmented based on their considerations into three clusters: affordability-oriented, privacy-oriented, and reliability-oriented. We present an in-depth quantification of consumer considerations on smart home device adoption along with desired privacy and security features consumers wish to use to protect their privacy in the smart home.