Journal:Hierarchical AI enables global interpretation of culture plates in the era of digital microbiology

Full article title	Hierarchical AI enables global interpretation of culture plates in the era of digital microbiology
Journal	Nature Communications
Author(s)	Signoroni, Alberto; Ferrari, Alessandro; Lombardi, Stefano; Savardi, Mattia; Fontana, Stefania; Culbreath, Karissa
Author affiliation(s)	University of Brescia, Copan WASP, Tricore Laboratories
Primary contact	Email: alberto dot signoroni at unibs dot it
Year published	2023
Volume and issue	14
Article #	6874
DOI	10.1038/s41467-023-42563-1
ISSN	2041-1723
Distribution license	Creative Commons Attribution 4.0 International
Website	https://www.nature.com/articles/s41467-023-42563-1
Download	https://www.nature.com/articles/s41467-023-42563-1.pdf (PDF)

Abstract

Full laboratory automation is revolutionizing work habits in an increasing number of clinical microbiology facilities worldwide, generating huge streams of digital images for interpretation. Contextually, deep learning (DL) architectures are leading to paradigm shifts in the way computers can assist with difficult visual interpretation tasks in several domains. At the crossroads of these epochal trends, we present a system able to tackle a core task in clinical microbiology, namely the global interpretation of diagnostic bacterial culture plates, including presumptive pathogen identification. This is achieved by decomposing the problem into a hierarchy of complex subtasks and addressing them with a multi-network architecture we call DeepColony. Working on a large stream of clinical data and a complete set of 32 pathogens, the proposed system is capable of effectively assisting plate interpretation with a surprising degree of accuracy in the widespread and demanding framework of urinary tract infections (UTIs). Moreover, thanks to the rich species-related generated information, DeepColony can be used for developing trustworthy clinical decision support services in laboratory automation ecosystems from local to global scale.

Keywords: bacteriology, clinical microbiology, computer science, urinary tract infection, laboratory automation, deep learning

Introduction

Microbiology is faced with tremendous questions, with bacterial , viral, and parasitic infections representing major threats to human health [1], in conjunction with concerns such as new species discovered annually [2], re-emerging pathogens [3,4], zoonotic infections [5], and antimicrobial resistance. [6] Correct and timely identification of pathogens is essential to effectively fight infections and the interpretation of bacterial cultures from human collected samples is a pivotal step in the clinical process. Over the last 30 years, molecular biology-based techniques (from polymerase chain reaction [PCR] to next-generation sequencing [NGS] [7,8]), vibrational spectroscopy [9], and mass spectrometry (MS) diagnostic tools such as MALDI-ToF [10,11] have also emerged as critical tools in the clinical microbiology laboratory (CML) for identification of pathogens. However, routine bacterial culture continues to be the mainstay for diagnosis of bacterial infectious diseases. [12] The continued use of culture in the CML [13] is due to its role in the recovery of viable organisms, the availability of a given organism for antimicrobial susceptibility testing (AST), the detection of unusual or unexpected pathogens, and the lower costs associated with culture-based methods compared with culture-independent methods.

Notwithstanding the benefits, culture interpretation is often a challenging undertaking even for the skilled microbiologist. Other specialties in clinical laboratory medicine that require subtle interpretation and complex visual diagnostic tasks have already shifted into a digital imaging ecosystem (e.g., urinalysis, hematology, and cytology). However, bacterial culture and culture plate interpretation have remained mostly unchanged from its origins in the nineteenth century. Thus, the entire culture process is still accompanied by the susceptibility to human factors and is still restricted by staff shift-availability, while the skilled workforce available to perform and read the cultures has decreased in the past few years. [14] In this resource-critical context, the recent advent and deployment of full laboratory automation (FLA) systems [15,16,17] has led to levels of automation and standardization of the physical culture steps (e.g., specimen processing, plate streaking, and incubation) along with a step of shooting the culture plates, which makes the plate images available for the application of digital analysis tools (Fig. 1a).

Figure 1. a The full laboratory automation (FLA) workflow and the role of the proposed solutions. b DeepColony hierarchical architecture of the operational and decision-making processes. c Example of “enumeration map” (level 0 output) where the cardinality of colony aggregates is indicated with colors and numbers. An “X” appears on colonies discarded because they are located in color distortion areas. d Example (on the same plate) of selected “good colonies” (level 1 output) with their presumptive ID (level 2 output). An “X” appears on colonies discarded because they are not single or too close to the plate border or to confluence areas. Detected species Si are indicated for each colony with colors and corresponding legend. e Outputs of level 3 and 4 are shown on the same plate. Gi refers to species ID updated after similarity-driven contextual grouping, while subsequent clinical relevance interpretation indicates a significant mixed flora plate, where best colonies eligible for picking are indicated with arrows. f Convolutional neural network (CNN) architecture for single colony presumptive ID (red box at level 2). g “Siamese CNN” scheme (red box at level 3) for contextual ID.

FLAs have already demonstrated clinical improvements through early growth detection, better culture quality, shorter turnaround times, and significant cost savings, with highly positive impacts on complementary downstream tasks (e.g., MALDI-ToF identification, AST, genome sequencing). [18,19,20,21] Despite this, considerable challenges still remain open within the digitized laboratory workflow [22,23] regarding the interpretation of culture images. Image analysis solutions in bacterial culture have been developed in specific contexts [18,24,25,26,27] and especially for the use of selective and differential media, including chromogenic agar and other differential media. However, algorithms developed using these methods already benefit from the biochemical properties of media and do not require broad identification of the many types of organisms seen in culture. Moreover, to implement such a tool, laboratories would be required to adopt a new media for their culture process, potentially incurring additional costs.

Few studies have demonstrated the development of an image analysis algorithm using a non-selective, non-differential media commonly used in the laboratory, such as sheep blood agar. Additionally, consistent and replicable culture interpretation remains a clinical challenge requiring significant expertise. [28] Therefore, there is a need for an additional clinical decision support tool that informs the technologist of the expected action based on observation of morphologies of grown organisms to ensure accurate and consistent culture interpretation on a large scale. Such an automated whole-plate interpretation has still not been tackled in its full complexity, and nothing even close to it has been prefigured until now. [16,17,23,29]

We approach these challenging objectives by leveraging the recent digital revolution of clinical microbiology culture and the strengths of deep learning (DL) for the solution of complex tasks. [30,31,32,33] In particular, compared to the most common solutions based on single convolutional neural networks (CNN) [34], multi-network architectures are attractive in our case because of their ability to fit into contexts where decision-making processes are stratified into a complex structure. [35] The system must be designed to generate useful and easily interpretable information and to support expert decisions according to safety-by-design and human-in-the-loop (HITL) policies, aiming at achieving cost-effectiveness and skill-empowerment respectively. This requires an advanced ability to combine species-level identification and quantitation across a possibly wide range of clinically relevant pathogens; a skillful combination of multiple visual, cognitive, and procedural tasks; and a computational architecture capable of reproducing this complex environment involving both single-colony and whole-plate analyses.

Notes

This presentation is faithful to the original, with only a few minor changes to presentation, grammar, and punctuation. In some cases important information was missing from the references, and that information was added.