Diane de Saint-Affrique is the author of “IA et médecine : quand les avancées technologiques redéfinissent les règles d’une IA responsable dans le domaine de la santé” in the Médecine & Droit journal (ed. Elsevier).
In many respects, the invention of artificial intelligence (AI) is comparable to that of the printing press, insofar as it embodies a revolution that transcends all spheres of society, from the economy to politics and medicine. While it is undeniable that AI now plays a central role, a debate is emerging regarding its use and the limits of its application in sensitive sectors such as healthcare. AI holds great promise for medical practice and innovation, but it also gives rise to serious concerns about potential misuse. The ethical issues and challenges it raises are therefore considerable.
The Benefits of AI in Medecine
There is no denying that AI is bringing about major advances in medical practice. It enhances efficiency, notably by taking over certain repetitive tasks from medical staff. In certain fields where image analysis is required, such as diagnostic radiology, dermatology or ophthalmology, AI can even deliver diagnoses faster and more reliably than a human clinician. Consequently, allowing medical professionals to spend more time with patients. A study conducted by Capio reveals that in Sweden, AI-enabled medical assistants reduce doctors’ administrative workload by 30 per cent1. Beyond the time savings it offers, AI also contributes to improving predictive medicine, particularly through its ability to sort and identify relevant information rapidly to aid decision-making and advance scientific knowledge in specialties such as oncology. AI is also enhancing precision medicine by supporting doctors through computer-assisted surgery. In 2024, for example, Dr Richard Gaston performed remote surgery on a patient in Beijing, 11,000 km away, from the Clinique Saint-Augustin in Bordeaux. Finally, AI is helping to accelerate biomedical research and drug discovery.
The Ethical Challenges of Using AI in Medicine
While AI offers unprecedented opportunities for medical progress, it is imperative to remain mindful of the dangers it poses so as to address them effectively. In this respect, Montesquieu’s proverb, “the best is the enemy of the good”, perfectly captures the mindset with which this issue should be approached.
A first limitation concerns the so-called “black-box” nature of certain AI systems. In other words, how can a disease prediction generated by AI be verified? In certain areas, the results produced by AI will not be verifiable, such as predictions relating to the future onset of a disease or genomic analysis. In such cases, the doctor will have no choice but to rely blindly on this new technology. This raises a number of concerns, particularly regarding the reliability of the data, but also regarding the risk of error generated by an AI system that may prove fallible when confronted with the complexity of the real world. The challenge here lies in validating “black-box” outputs. Some algorithms make it virtually impossible for human beings to understand how the machine arrived at its conclusion. This raises issues of security, transparency and accountability. Moreover, with the growing prominence of this technology, the balance established between doctor, patient and medical decision-making may be weakened, potentially altering the trust-based relationship between doctor and patient.
A second limitation concerns patient consent. To improve, AI systems rely on vast quantities of data, including patient data. Consequently, the quality and selection of this data are at the core of the ethical debate: who will own this data? Who will be entitled to use it? For example, in 2019 Google gained access to the data of 50 million patients in the United States without explicit individual consent, triggering controversy and lawsuits2.
A third limitation relates to the doctor’s responsibility in the age of AI. The sanctity of the doctor–patient relationship could be altered if the practitioner is no longer perceived as the sole decision-maker, but rather as a joint decision-maker alongside a machine. This calls into question the primacy of the doctor–patient relationship and, in parallel, the issue of responsibility. Who will be held responsible in the event of a medical error: the algorithm’s designer, the software developer, or the doctor who used it?
Solutions: Towards a Responsible Use of AI in Medicine
The primacy of the doctor–patient relationship and the doctor’s ultimate authority in the decision-making process must remain fundamental to ensure the development of ethical and responsible AI. This implies that doctors must also develop resistance to economic pressures, since these should not, under any circumstances, drive the development and use of AI. Consequently, it seems to us that securing data and ensuring algorithmic transparency must be prioritised in the development of responsible AI. In practice, this means verifying the selection and quality of data at the collection stage, auditing algorithms throughout their lifecycle, and ensuring that their outputs can be explained.
Finally, harmonising international legislation to control AI developments is essential. Global competition encourages ever greater speed of development, at the risk of producing defective or dangerous systems. To prevent this, it appears essential to establish international regulatory bodies. At the national level, close collaboration between public and private actors is crucial. Finally, creating an independent oversight body, modelled on the French Comité Consultatif National d’Éthique (National Consultative Ethics Committee) and dedicated specifically to AI issues, would help to harmonise practices and ensure that AI is used ethically in healthcare.
The opinion is available below:
- Frapin, A. (2025). Les assistants médicaux dotés d’IA réduisent de 30 % le temps administratif des médecins (study). Le Quotidien du Médecin. https://www.lequotidiendumedecin.fr/sante-societe/e-sante/les-assistants-medicaux-dotes-dia-reduisent-de-30-le-temps-administratif-des-medecins-etude ↩︎
- Le Point.fr. (2019). « Nightingale » : quand Google fait la chasse aux données médicales. Le Point.fr. https://www.lepoint.fr/high-tech-internet/nightingale-quand-google-fait-la-chasse-aux-donnees-medicales-12-11-2019-2346666_47.php ↩︎
