December 6, 2023 at 10:01AM
The FDA and ONC are creating guidelines for AI in healthcare, focusing on transparency, data quality, and model validation. The ONC’s new rules aim to certify EHRs with fair and safe decision support tools. Healthcare companies are advised that AI tools must be safe, effective, explainable, and secure.
**Clear Takeaways from Meeting Notes:**
1. **FDA’s Responsibility and Public Feedback:** The FDA is collecting feedback to inform a regulatory framework for AI in healthcare, focusing on ensuring that healthcare products, including those employing AI and machine learning, are safe and effective.
2. **FDA’s AI Transparency and Governance:**
– Advocates for human-led governance, accountability, and transparency in AI model development and deployment.
– Emphasizes the importance of high-quality, reliable, and representative data for training AI models.
– Stresses on model development, performance, ongoing monitoring, and validation, considering the influences and consequences of AI recommendations.
3. **ONC’s New Rules for DSIs:**
– A proposed rule, ONC HTI-1, seeks to modernize the agency’s approach to EHRs and healthcare data management.
– Although the rule does not directly address AI/ML, it has important implications for decision support interventions (DSIs), pushing for the criteria of fair, appropriate, valid, effective, and safe (FAVES).
– Increased transparency and public disclosure around risk management for DSI models.
4. **Equity and Fairness in DSI Models:**
– The ONC emphasizes the need to avoid model bias and ensure fairness in training data, pertinent to the intended audience.
– There is a push for clarity and transparency regarding the inputs used by a model in making recommendations.
5. **Implications for Healthcare Companies:**
– AI tools in healthcare must prioritize safety, effectiveness, and transparency.
– Security practices are fundamental to ensure that AI-driven systems in healthcare remain safe and effective, as mentioned in the FDA’s rule 524B and related guidance.
6. **Overall Message:**
– Both the FDA and ONC are working towards safeguarding patient safety and treatment efficacy in the face of emerging AI and ML technologies in healthcare.
– Healthcare organizations need to stay informed and compliant with the evolving regulatory environment shaped by both the FDA and ONC, ensuring that the use of AI/ML tools is transparent, reliable, explainable, and secured against risks.
(Readers are encouraged to consult Partner Perspectives from Google Cloud for further insights.)