Ethical and Legal Challenges Posed by the Implementation of AI Applications in the Healthcare Setting

Healthcare institutions are implementing artificial intelligence (AI) at a rapid pace. The hope is that AI will improve the quality of care and reduce costs in the long run. However, the deployment of AI in healthcare settings also presents new ethical and legal challenges. For example, AI can reproduce health disparities and pose a risk to patients if human factors, like implicit or explicit bias, are present in the training data set or it lacks representation from population subgroups. AI also raises challenges for regulators like the U.S. Food and Drug Administration (FDA). How should regulators like the FDA deal with so-called “adaptive” AI algorithms that continuously learn or opaque (“black-box”) algorithms? Regulators need to update the regulatory framework for AI-based medical devices as soon as possible to ensure that AI-based medical devices are reasonably safe and effective when introduced on the market and stay safe and effective throughout their entire life cycle. A new regulatory framework is particularly important given that the FDA has already permitted the marketing of more than 340 AI-based medical devices, and shortcomings in the framework can compromise patient safety and shake public trust. There are also several questions about data privacy and security. AI relies on a large amount of data, and its use and sharing can, in some cases, compromise patient privacy. The Dinerstein v. Google lawsuit shows the likelihood that patients will struggle to successfully sue hospitals for sharing their data with technology companies like Google because they cannot demonstrate damages. This case also raises the issue of “data triangulation,” emphasizing the risk that even a de-identified data set may be re-identified if technology companies use other data sets at hand. Lastly, AI raises liability questions. For example, who is liable when a physician follows an inaccurate AI recommendation and the patient is injured as a result? This collection is a starting point for readers who wish to become familiar with bias, privacy, liability, and other ethical and legal challenges of implementing AI applications in the healthcare setting.