– Reflections of a clinical microbiology scientist with extensive experience in diagnostic laboratory medicine –
The development of Artificial Intelligence (AI) in a variety of areas in medicine can help to expand knowledge and can have significant advantages (particularly in time) over some older technologies and methodologies to provide data to clinicians in the management of care for their patients.
The methodology of AI, as this viewer understands it, is to compile large amounts of data from various sources- from literature, from data banks, and from sources provided by laboratories or vendors (e.g. diagnostic testing devices, pharmaceutical industry, etc.) to answer medical issues posed by the medical field about potential improvement to medical care.
In my field of laboratory medicine there is currently at the International Standards Organization (ISO) level, the beginnings of those processes in anatomical pathology. It is called Digital Pathology, whereby results of manual reading of pathology slides with specific parameters of certain malignancies can be examined by automated microscopic readers to measure whether the AI readings perform better than the expertise and naked eye of the pathologist.
Based on my experience in medical microbiology as a consultant, my role as Associate Chair of a Medical Ethics Board, and my involvement in evaluating the potential of AI to improve patient care, I will present several examples that highlight key issues requiring careful consideration.
My first example occurred during the COVID pandemic. Nucleic acid methodologies were being developed by those who wanted into the game. Rapid testing kits were rife. Validation was minimal at best. So a result was a result with little scientific backing. At ISO, in collaboration with an expert group that also investigated pre-examination sample collection processes (and with data collected from a CMPT quality assurance programme), the results showed that in some instances there were as many as 30% false positive or false negative test results. In one such instance that I was involved with, the quality of the nucleic acid testing depended on the limits of detection which only were resolved when they were compared with the international standard criteria that we developed (see ISO/TS 5798:2022 In vitro diagnostic test systems — Requirements and recommendations for detection of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) by nucleic acid amplification methods). Many test results prior to that were considered positive but were not associated with SARS-CoV-2. The summary of this experience related to possible AI diagnostics was – sound, standardized methodology including pre-examination requirements, reference to clinical conditions, quality assurance, extensive validation (not just a few samples as some have suggested) and extensive comparison with current standards. Examination and comparison with the “literature” is fraught with difficulties. Many recent publications are open access (if you pay for publication, it is much easier and faster to get studies in print) which may or may not have extensive review. It is unknown if AI can sort through these considerations and discard those which are not of superior quality.
A second example in my own field of medical microbiology, involves the assessment of antimicrobial susceptibility by determination of whether a bacterial or fungal species carries genes that may be associated with a resistance determinant. I have observed this applied to both individual test results and to more global population indications of antimicrobial resistance.
As you will be well aware, the pharmaceutical industry is constantly trying to keep up with the problem of antibiotic resistance. We now have well developed and standardized methodologies to test for phenotypic resistance (that is measurable by standard laboratory methods.). Genotypic antimicrobial resistance is another matter entirely. Examination for resistance determinants by determining if a clinical isolate has a gene measurable resistance determinant (.e.g for mecA – methicillin/penicillin resistance in staphylococci) or for isoniazid resistance in Mycobacterium tuberculosis – MTB) is now available. For these examples, particularly for MTB, resistance rapid determination of resistance is valuable since the classical methodology can take too long for effective early treatment of tuberculosis. For mecA the rapid determination may be of greater use for infection control purposes. In both cases, a culture isolate is required at this time.
Whole genome sequencing (WGS) and related methods for gene determination in micro-organisms is also being developed. Examples are extended spectrum beta-lactamase (ESBL) production, carbapenemases, etc. It is well-known that many of these are mediated on plasmids, which regularly migrate in and out of the bacterial species, or where the nucleic acid determinant is not routinely expressed (covered in the cell by a protein that is necessary to be removed in situ for resistance expression. Without standard phenotypic demonstrations of those resistances for the particular clinical isolate, it is only a guess for the clinician as to the efficacy or lack thereof of directed antimicrobial therapy. Further, at the present time, there is insufficient evidence regarding the degree of resistance that the isolate imparts based only on sequencing recognition.
I harken back to a statement made by a colleague a few years ago where it was indicated that the micro-organisms have been here for billions of years, long before we entered the picture, and will be here a lot longer thereafter. They are just trying to survive in their own environments. I discovered some years ago, from reading a publication that showed that an apparent pan-susceptible isolate of Escherichia coli (phenotypic methods) that had been recovered in the late 1940s, had resistance determinants to sulphonamides tetracycline and beta-lactams. So, this phenomenon is not new. It is essential that any nucleic acid resistance determinants recognized in the diagnostic laboratory are confirmed by phenotype before being reported. AI that only examines nucleic acid test results may provide some degree of likelihood of resistance but that examination requires careful and extensive quality investigation before diagnostic microbiology laboratories and particularly the automated test systems being utilized now can be relied on for patient management.
I believe we will need to await further developments in antimicrobial phenotypic test measurements such as rapid determination of specific resistance proteins that can then be quantified in a culture (as we do now with standardized susceptibility testing methods) to clearly establish that the clinical isolate has the capability to attack and destroy the antimicrobial agent.
In the meantime. It is my belief that laboratories and their associated industries should be cautious about test results that are reported by AI or other mechanisms that are not properly validated and for which quality assurance cannot be accurately determined.
Robert P. Rennie PhD, FCCM, D(ABMM)
Professor Emeritus
Laboratory Medicine and Pathology
Faculty of Medicine and Dentistry, University of Alberta,
Edmonton Alberta
