Guidance

Predetermined Change Control Plans for Machine Learning-Enabled Medical Devices: Guiding Principles

Published 24 October 2023

In 2021, the U.S. Food and Drug Administration (FDA), Health Canada, and the U.K.’s Medicines and Healthcare products Regulatory Agency (MHRA) jointly identified 10 guiding principles that can inform the development of Good Machine Learning Practice (GMLP). GMLP supports the development of safe, effective, and high-quality artificial intelligence/machine learning technologies that can learn from real-world use and, in some cases, improve device performance.

In this document, FDA, Health Canada, and MHRA jointly identified 5 guiding principles for predetermined change control plans. These principles draw upon the overarching GMLP guiding principles, in particular principle 10, which states that deployed models are monitored for performance and re-training risks are managed.

Advancements in digital health technologies include artificial intelligence/machine learning-enabled medical devices (MLMD). Regulatory expectations that are aligned with best practices for development and change management, such as those described in the GMLP Guiding Principles, can help to support the quality of such devices. Ultimately, this can lead to patient benefits such as earlier access to innovative technologies or more accurate diagnoses.

The change management process helps to ensure the ongoing safety and effectiveness of devices in the face of change throughout the device’s total product lifecycle (TPLC). However, certain changes to MLMDs, such as changes to a model or algorithm, may be substantive or significant. For this reason, they can require regulatory oversight, such as additional premarket review. Such regulatory expectations may not always coincide with the rapid pace of MLMD development.

Internationally, the medical device community is discussing the use of predetermined change control plans (PCCPs) as a way of managing certain device changes where regulatory authorization before marketing is typically required. PCCPs can be used to help:

• align regulatory processes with the rapid and ongoing approach to change management in MLMDs
• manage risks in a timely and ongoing fashion through monitoring, maintenance, and/or improving device performance
• uphold high regulatory standards to ensure device safety and effectiveness.

For this document, the term PCCP describes a plan, proposed by a manufacturer, that specifies:

• certain planned modifications to a device
• the protocol for implementing and controlling those modifications and
• the assessment of impacts from modifications.

PCCPs may be developed and implemented in different ways in different regulatory jurisdictions.

One key objective of the 5 Guiding Principles for PCCPs for MLMD is to provide foundational considerations that highlight the characteristics of robust PCCPs. Another objective of this document is to facilitate and foster ongoing engagement and collaboration among stakeholders on the PCCP concept for MLMD. As with the GMLP Guiding Principles, this document intends to lay a foundation for PCCPs and encourages international harmonization.

International harmonization and stakeholder consensus on the core concepts of PCCPs will help support the advancement of responsible innovations in the digital health space.

We welcome your continued feedback through the FDA public docket (FDA-2019-N-1185) at Regulations.gov, and we look forward to engaging with you on these efforts. This work is being spearheaded by the Digital Health Center of Excellence for the FDA, the Medical Devices Directorate Digital Health Division at Health Canada and the software and AI team at the MHRA. Contact us directly at [email protected], [email protected], and [email protected].

Guiding Principles

1. Focused and Bounded

A PCCP describes specific changes that a manufacturer intends to implement. Such changes are limited to modifications within the intended use or intended purpose of the original MLMD. This characterization can include:

• the extent of planned changes and scope of the MLMD with changes implemented
• plans in place to safely modify the device within the bounds of the PCCP, including methods for verifying and validating the changes and mechanisms to detect and revert or stop implementation of a change that fails to meet specified performance criteria
• the impacts of the planned changes

2. Risk-based

The value and reliability of a PCCP is strengthened when the intent, design, and implementation of a PCCP are driven by a risk-based approach that adheres to the principles of risk management. This risk-informed perspective is relevant:

• throughout the TPLC, from inception, through implementation and to use
• to ensure that individual and cumulative changes remain appropriate over time for the device and its use environment

3. Evidence-Based

Evidence generated throughout the TPLC of the device is important to:

• ensure the ongoing safety and effectiveness of the device with a PCCP
• demonstrate that the benefits outweigh the associated risks and
• establish that the risks are adequately managed and controlled

Considerations for evidence supporting a PCCP include:

• methods and metrics used to measure device performance are scientifically and clinically justified, in proportion to the risk and consistent with other evidence gathered throughout the TPLC
• methods to generate evidence are specified that demonstrate the benefits and risks of the device before and after changes specified in the PCCP are implemented

4. Transparent

For PCCPs, the best practice is to provide clear and appropriate information and detailed plans for ongoing transparency to users and other stakeholders. This helps ensure that stakeholders stay aware of the device’s performance and use before and after changes are implemented. Consider, for example:

• characterization of data used in development and modifications, demonstrated to reflect the intended population
• comprehensive testing for planned changes
• characterization of the device before and after implementation of changes
• monitoring, detection, and response to deviations in device performance

5. Total Product Lifecycle (TPLC) Perspective

Creating and using a PCCP from a TPLC perspective can:

• elevate the quality and integrity of a PCCP by continually considering the perspectives of all stakeholders as well as risk management practices throughout the TPLC
• use and support existing regulatory, quality, and risk management measures throughout the TPLC to ensure device safety by monitoring, reporting and responding to safety concerns.