How to achieve and maintain transparency in artificial intelligence-enabled medical devices

Transparency in artificial intelligence in medical devices is achievable and necessary, according to new guidance promoted jointly by the FDA, Health Canada and the UK Medicines and Healthcare products Regulatory Agency.

Released on June 13, the document updates and builds on the principles set by the three bodies in 2021.

The guidance defines “MLMD” transparency – for machine learning-enabled medical devices – as “the degree to which relevant information about MLMD… is clearly communicated to appropriate audiences.”

The guidance asks healthcare AI developers and vendors to consider the full range of implications of their technology for patient safety and, by extension, population health. It contains six questions to facilitate pre-implementation surveys.

Q. For whom is MLMD transparency important?

AND. As the new guidance states, transparency is important for people using the device, such as healthcare workers, patients and carers, and for people receiving healthcare through the device, such as patients. More:

It is also important for additional parties, including those who make decisions about the patient outcomes device, such as support staff, administrators, payers, and governing bodies.

Q. Why is transparency essential for patient-centered care and device safety and effectiveness?

AND. Because presenting information clearly and consistently, while taking into account known information gaps, can bring many benefits. For starters, transparency “builds fluency and efficiency in using MLMD.”

Moreover, transparency can increase trust in technology. Encourages adoption and access to beneficial technologies.

Q. What information should be shared?

AND. The new guidelines allow that the type of information deemed appropriate to share will vary depending on MLMD regulations. Appropriateness will depend on the benefits and risks of each MLMD and the needs of the intended users.

It is good practice to provide information that facilitates understanding of the device and its purpose. A clear and accurate description of a device typically includes information about its intended use and medical function, the diseases or conditions it is intended to address, and the intended users, environments of use, and target populations.

Q. Where is the best place to put this information?

AND. Optimally through the user interface. This should include “all parts of the device that the user can see, hear and touch.” For example, training, physical inspections, display elements, packaging, labeling, and alarms are part of the user interface.

It is good practice to optimize the use of the software’s user interface so that the information it provides responds to the user’s needs. User needs can be met in a variety of ways, including audio, video, on-screen text, alerts, diagrams, software security, and document libraries.

Question: At what point in MLMD implementation should communication be directed?

AND. “Taking into account information needs at every stage of the entire product lifecycle can ensure effective transparency,” the document says. “It may also be helpful to provide timely notifications when a device is updated or modified or new device information is detected.”

It may be appropriate to provide targeted information, such as on-screen instructions or warnings, at a specific stage in the workflow, e.g., during high-risk stages, and for specific triggers, e.g., the presence of certain input or output features.

Q. What is the best way to ensure long-term transparency of any MLMD system?

AND. Communicating information about MLMD requires a holistic understanding of users, environments, and workflows. The guidelines maintain that this can be addressed by applying human-centered design principles.

Transparency information is optimally accessible and useful when it provides the appropriate level of detail for the target audience. Simple language may be adequate for understanding and usability in some cases. In other cases, technical language may be appropriate for specialized clinical users.

Read the entire document.