Difference between revisions of "Journal:Evidence-based design and evaluation of a whole genome sequencing clinical report for the reference microbiology laboratory"
Shawndouglas (talk | contribs) (Saving and adding more.) |
Shawndouglas (talk | contribs) (Saving and adding more.) |
||
| Line 38: | Line 38: | ||
Whole genome sequencing (WGS) is quickly moving from proof-of-concept research into routine clinical and public health use. WGS can diagnose infections at least as accurately as current protocols<ref name="FukuiMeta15">{{cite journal |title=Metagenomic analysis for detecting pathogens in culture-negative infective endocarditis |journal=Journal of Infection and Chemotherapy |author=Fukui, Y.; Aoki, K.; Okuma, S. et al. |volume=21 |issue=12 |pages=882–4 |doi=10.1016/j.jiac.2015.08.007 |pmid=26360016}}</ref><ref name="LomanACulture13">{{cite journal |title=A culture-independent sequence-based metagenomics approach to the investigation of an outbreak of Shiga-toxigenic Escherichia coli O104:H4 |journal=JAMA |author=Loman, N.J.; Constantinidou, C.; Christner, M. et al. |volume=309 |issue=14 |pages=1502-10 |doi=10.1001/jama.2013.3231 |pmid=23571589}}</ref>, can predict antimicrobial resistance phenotypes for certain drugs<ref name="BradleyRapid15">{{cite journal |title=Rapid antibiotic-resistance predictions from genome sequence data for Staphylococcus aureus and Mycobacterium tuberculosis |journal=Nature Communications |author=Bradley, P.; Gordon, N.C.; Walker, T.M. et al. |volume=6 |pages=10063 |doi=10.1038/ncomms10063 |pmid=26686880 |pmc=PMC4703848}}</ref><ref name="PankhurstRapid16">{{cite journal |title=Rapid, comprehensive, and affordable mycobacterial diagnosis with whole-genome sequencing: A prospective study |journal=The Lancet Respiratory Medicine |author=Pankhurst, L.J.; Del Ojo Elias, C.; Votintseva, A.A. et al. |volume=4 |issue=1 |pages=49-58 |doi=10.1016/S2213-2600(15)00466-X |pmid=26669893 |pmc=PMC4698465}}</ref><ref name="WalkerWhole15">{{cite journal |title=Whole-genome sequencing for prediction of Mycobacterium tuberculosis drug susceptibility and resistance: A retrospective cohort study |journal=The Lancet Infectious Diseases |author=Walker, T.M.; Kohl, T.A.; Omar, S.V. et al. |volume=15 |issue=10 |pages=1193–1202 |doi=10.1016/S1473-3099(15)00062-6 |pmid=26116186 |pmc=PMC4579482}}</ref> with high concordance to culture-based testing methods, and can be used in outbreak surveillance to resolve transmission clusters at a resolution not possible with existing genomic or epidemiological methods.<ref name="NikolayevskyyWhole16">{{cite journal |title=Whole genome sequencing of Mycobacterium tuberculosis for detection of recent transmission and tracing outbreaks: A systematic review |journal=Tuberculosis |author=Nikolayevskyy, V.; Kranzer, K.; Niemann, S. et al. |volume=98 |pages=77-85 |doi=10.1016/j.tube.2016.02.009 |pmid=27156621}}</ref> Importantly, WGS offers faster turnaround times compared to many culture-based tests, particularly for antimicrobial resistance testing in slow-growing bacteria. | Whole genome sequencing (WGS) is quickly moving from proof-of-concept research into routine clinical and public health use. WGS can diagnose infections at least as accurately as current protocols<ref name="FukuiMeta15">{{cite journal |title=Metagenomic analysis for detecting pathogens in culture-negative infective endocarditis |journal=Journal of Infection and Chemotherapy |author=Fukui, Y.; Aoki, K.; Okuma, S. et al. |volume=21 |issue=12 |pages=882–4 |doi=10.1016/j.jiac.2015.08.007 |pmid=26360016}}</ref><ref name="LomanACulture13">{{cite journal |title=A culture-independent sequence-based metagenomics approach to the investigation of an outbreak of Shiga-toxigenic Escherichia coli O104:H4 |journal=JAMA |author=Loman, N.J.; Constantinidou, C.; Christner, M. et al. |volume=309 |issue=14 |pages=1502-10 |doi=10.1001/jama.2013.3231 |pmid=23571589}}</ref>, can predict antimicrobial resistance phenotypes for certain drugs<ref name="BradleyRapid15">{{cite journal |title=Rapid antibiotic-resistance predictions from genome sequence data for Staphylococcus aureus and Mycobacterium tuberculosis |journal=Nature Communications |author=Bradley, P.; Gordon, N.C.; Walker, T.M. et al. |volume=6 |pages=10063 |doi=10.1038/ncomms10063 |pmid=26686880 |pmc=PMC4703848}}</ref><ref name="PankhurstRapid16">{{cite journal |title=Rapid, comprehensive, and affordable mycobacterial diagnosis with whole-genome sequencing: A prospective study |journal=The Lancet Respiratory Medicine |author=Pankhurst, L.J.; Del Ojo Elias, C.; Votintseva, A.A. et al. |volume=4 |issue=1 |pages=49-58 |doi=10.1016/S2213-2600(15)00466-X |pmid=26669893 |pmc=PMC4698465}}</ref><ref name="WalkerWhole15">{{cite journal |title=Whole-genome sequencing for prediction of Mycobacterium tuberculosis drug susceptibility and resistance: A retrospective cohort study |journal=The Lancet Infectious Diseases |author=Walker, T.M.; Kohl, T.A.; Omar, S.V. et al. |volume=15 |issue=10 |pages=1193–1202 |doi=10.1016/S1473-3099(15)00062-6 |pmid=26116186 |pmc=PMC4579482}}</ref> with high concordance to culture-based testing methods, and can be used in outbreak surveillance to resolve transmission clusters at a resolution not possible with existing genomic or epidemiological methods.<ref name="NikolayevskyyWhole16">{{cite journal |title=Whole genome sequencing of Mycobacterium tuberculosis for detection of recent transmission and tracing outbreaks: A systematic review |journal=Tuberculosis |author=Nikolayevskyy, V.; Kranzer, K.; Niemann, S. et al. |volume=98 |pages=77-85 |doi=10.1016/j.tube.2016.02.009 |pmid=27156621}}</ref> Importantly, WGS offers faster turnaround times compared to many culture-based tests, particularly for antimicrobial resistance testing in slow-growing bacteria. | ||
As reference microbiology laboratories move towards accreditation of WGS for routine clinical use, the community is turning its attention toward standardization, developing standard operating procedures for reproducible sample handling, sequencing, and downstream bioinformatics analysis.<ref name="BudowleValid14">{{cite journal |title=Validation of high throughput sequencing and microbial forensics applications |journal=Investigative Genetics |author=Budowle, B.; Connell, N.D.; Bielecka-Oder, A. et al. |volume=5 |pages=9 |doi=10.1186/2041-2223-5-9 |pmid=25101166 |pmc=PMC4123828}}</ref><ref name="GargisAssur16">{{cite journal |title=Assuring the Quality of Next-Generation Sequencing in Clinical Microbiology and Public Health Laboratories |journal=Journal of Clinical Microbiology |author=Gargis, A.S.; Kalman, L.; Lubin, I.M. |volume=54 |issue=12 |pages=2857-2865 |doi=10.1128/JCM.00949-16 |pmid=27510831 |pmc=PMC5121372}}</ref> Reporting genomic microbiology test results in a way that is interpretable by clinicians, nurses, laboratory staff, researchers, and surveillance experts and that meets regulatory requirements is equally important; however, relatively little effort has been directed toward this area. WGS clinical reports are often produced in-house on an ''ad hoc'', project-by-project basis, with the resulting product not necessarily meeting the needs of the many stakeholders using the report in their clinical and surveillance workflows. | |||
===Human-centered design in the clinical laboratory=== | |||
The information visualization, human–computer interaction, and usability engineering fields offer techniques and design guidelines that have informed bioinformatics tools, including Disease View<ref name="DriscollInteg11">{{cite journal |title=Integration and visualization of host-pathogen data related to infectious diseases |journal=Bioinformatics |author=Driscoll, T.; Gabbard, J.L.; Mao, C. et al. |volume=27 |issue=16 |pages=2279-87 |doi=10.1093/bioinformatics/btr391 |pmid=21712250 |pmc=PMC3150046}}</ref> for exploring host-pathogen interaction data and Microreact<ref name="ArgimónMicro16">{{cite journal |title=Microreact: Visualizing and sharing data for genomic epidemiology and phylogeography |journal=Microbial Genomics |author=Argimón, S.; Abudahab, K.; Goater, R.J. et al. |volume=2 |issue=11 |pages=e000093 |doi=10.1099/mgen.0.000093 |pmid=28348833 |pmc=PMC5320705}}</ref> for visualizing phylogenetic trees in the context of epidemiological or clinical data. Although the public health community is beginning to recognize the potential role of visualization and analytics in daily laboratory workflows<ref name="CarrollVisual14">{{cite journal |title=Visualization and analytics tools for infectious disease epidemiology: A systematic review |journal=Journal of Biomedical Informatics |author=Carroll, L.N.; Au, A.P.; Detwiler, L.T. et al. |volume=51 |pages=287-98 |doi=10.1016/j.jbi.2014.04.006 |pmid=24747356 |pmc=PMC5734643 }}</ref> these techniques have not yet been applied to routine reporting of microbiological test results. However, work from the human health domain — particularly the formatting and display of pathology reports, where standardization is critical<ref name="LeslieStandard94">{{cite journal |title=Standardization of the surgical pathology report: Formats, templates, and synoptic reports |journal=Seminars in Diagnostic Pathology |author=Leslie, K.O.; Rosai, J. |volume=11 |issue=4 |pages=253-7 |pmid=7878300}}</ref> — sheds light on the complex task of clinical report design. | |||
==References== | ==References== | ||
Revision as of 16:19, 15 February 2018
| Full article title | Evidence-based design and evaluation of a whole genome sequencing clinical report for the reference microbiology laboratory |
|---|---|
| Journal | PeerJ |
| Author(s) | Crisan, Anamaria; McKee, Geoffrey; Munzner, Tamara; Gardy, Jennifer L. |
| Author affiliation(s) | University of British Columbia, British Columbia Centre for Disease Control |
| Primary contact | Email: jennifer dot gardy at bccdc dot ca |
| Editors | Smidt, H. |
| Year published | 2018 |
| Volume and issue | 6 |
| Page(s) | e4218 |
| DOI | 10.7717/peerj.4218 |
| ISSN | 2167-8359 |
| Distribution license | Creative Commons Attribution 4.0 International |
| Website | https://peerj.com/articles/4218/ |
| Download | https://peerj.com/articles/4218.pdf (PDF) |
 |
This article should not be considered complete until this message box has been removed. This is a work in progress. |
Abstract
Background: Microbial genome sequencing is now being routinely used in many clinical and public health laboratories. Understanding how to report complex genomic test results to stakeholders who may have varying familiarity with genomics — including clinicians, laboratorians, epidemiologists, and researchers — is critical to the successful and sustainable implementation of this new technology; however, there are no evidence-based guidelines for designing such a report in the pathogen genomics domain. Here, we describe an iterative, human-centered approach to creating a report template for communicating tuberculosis (TB) genomic test results.
Methods: We used design study methodology — a human centered approach drawn from the information visualization domain — to redesign an existing clinical report. We used expert consults and an online questionnaire to discover various stakeholders’ needs around the types of data and tasks related to TB that they encounter in their daily workflow. We also evaluated their perceptions of and familiarity with genomic data, as well as its utility at various clinical decision points. These data shaped the design of multiple prototype reports that were compared against the existing report through a second online survey, with the resulting qualitative and quantitative data informing the final, redesigned, report.
Results: We recruited 78 participants, 65 of whom were clinicians, nurses, laboratorians, researchers, and epidemiologists involved in TB diagnosis, treatment, and/or surveillance. Our first survey indicated that participants were largely enthusiastic about genomic data, with the majority agreeing on its utility for certain TB diagnosis and treatment tasks and many reporting some confidence in their ability to interpret this type of data (between 58.8% and 94.1%, depending on the specific data type). When we compared our four prototype reports against the existing design, we found that for the majority (86.7%) of design comparisons, participants preferred the alternative prototype designs over the existing version, and that both clinicians and non-clinicians expressed similar design preferences. Participants showed clearer design preferences when asked to compare individual design elements versus entire reports. Both the quantitative and qualitative data informed the design of a revised report, available online as a LaTeX template.
Conclusions: We show how a human-centered design approach integrating quantitative and qualitative feedback can be used to design an alternative report for representing complex microbial genomic data. We suggest experimental and design guidelines to inform future design studies in the bioinformatics and microbial genomics domains. We also suggest that this type of mixed-methods study is important to facilitate the successful translation of pathogen genomics in the clinic, not only for clinical reports but also more complex bioinformatics data visualization software.
Keywords: human-centered design, next-generation sequencing, report, tuberculosis, genome
Introduction
Whole genome sequencing (WGS) is quickly moving from proof-of-concept research into routine clinical and public health use. WGS can diagnose infections at least as accurately as current protocols[1][2], can predict antimicrobial resistance phenotypes for certain drugs[3][4][5] with high concordance to culture-based testing methods, and can be used in outbreak surveillance to resolve transmission clusters at a resolution not possible with existing genomic or epidemiological methods.[6] Importantly, WGS offers faster turnaround times compared to many culture-based tests, particularly for antimicrobial resistance testing in slow-growing bacteria.
As reference microbiology laboratories move towards accreditation of WGS for routine clinical use, the community is turning its attention toward standardization, developing standard operating procedures for reproducible sample handling, sequencing, and downstream bioinformatics analysis.[7][8] Reporting genomic microbiology test results in a way that is interpretable by clinicians, nurses, laboratory staff, researchers, and surveillance experts and that meets regulatory requirements is equally important; however, relatively little effort has been directed toward this area. WGS clinical reports are often produced in-house on an ad hoc, project-by-project basis, with the resulting product not necessarily meeting the needs of the many stakeholders using the report in their clinical and surveillance workflows.
Human-centered design in the clinical laboratory
The information visualization, human–computer interaction, and usability engineering fields offer techniques and design guidelines that have informed bioinformatics tools, including Disease View[9] for exploring host-pathogen interaction data and Microreact[10] for visualizing phylogenetic trees in the context of epidemiological or clinical data. Although the public health community is beginning to recognize the potential role of visualization and analytics in daily laboratory workflows[11] these techniques have not yet been applied to routine reporting of microbiological test results. However, work from the human health domain — particularly the formatting and display of pathology reports, where standardization is critical[12] — sheds light on the complex task of clinical report design.
References
- ↑ Fukui, Y.; Aoki, K.; Okuma, S. et al.. "Metagenomic analysis for detecting pathogens in culture-negative infective endocarditis". Journal of Infection and Chemotherapy 21 (12): 882–4. doi:10.1016/j.jiac.2015.08.007. PMID 26360016.
- ↑ Loman, N.J.; Constantinidou, C.; Christner, M. et al.. "A culture-independent sequence-based metagenomics approach to the investigation of an outbreak of Shiga-toxigenic Escherichia coli O104:H4". JAMA 309 (14): 1502-10. doi:10.1001/jama.2013.3231. PMID 23571589.
- ↑ Bradley, P.; Gordon, N.C.; Walker, T.M. et al.. "Rapid antibiotic-resistance predictions from genome sequence data for Staphylococcus aureus and Mycobacterium tuberculosis". Nature Communications 6: 10063. doi:10.1038/ncomms10063. PMC PMC4703848. PMID 26686880. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4703848.
- ↑ Pankhurst, L.J.; Del Ojo Elias, C.; Votintseva, A.A. et al.. "Rapid, comprehensive, and affordable mycobacterial diagnosis with whole-genome sequencing: A prospective study". The Lancet Respiratory Medicine 4 (1): 49-58. doi:10.1016/S2213-2600(15)00466-X. PMC PMC4698465. PMID 26669893. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4698465.
- ↑ Walker, T.M.; Kohl, T.A.; Omar, S.V. et al.. "Whole-genome sequencing for prediction of Mycobacterium tuberculosis drug susceptibility and resistance: A retrospective cohort study". The Lancet Infectious Diseases 15 (10): 1193–1202. doi:10.1016/S1473-3099(15)00062-6. PMC PMC4579482. PMID 26116186. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4579482.
- ↑ Nikolayevskyy, V.; Kranzer, K.; Niemann, S. et al.. "Whole genome sequencing of Mycobacterium tuberculosis for detection of recent transmission and tracing outbreaks: A systematic review". Tuberculosis 98: 77-85. doi:10.1016/j.tube.2016.02.009. PMID 27156621.
- ↑ Budowle, B.; Connell, N.D.; Bielecka-Oder, A. et al.. "Validation of high throughput sequencing and microbial forensics applications". Investigative Genetics 5: 9. doi:10.1186/2041-2223-5-9. PMC PMC4123828. PMID 25101166. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4123828.
- ↑ Gargis, A.S.; Kalman, L.; Lubin, I.M.. "Assuring the Quality of Next-Generation Sequencing in Clinical Microbiology and Public Health Laboratories". Journal of Clinical Microbiology 54 (12): 2857-2865. doi:10.1128/JCM.00949-16. PMC PMC5121372. PMID 27510831. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5121372.
- ↑ Driscoll, T.; Gabbard, J.L.; Mao, C. et al.. "Integration and visualization of host-pathogen data related to infectious diseases". Bioinformatics 27 (16): 2279-87. doi:10.1093/bioinformatics/btr391. PMC PMC3150046. PMID 21712250. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3150046.
- ↑ Argimón, S.; Abudahab, K.; Goater, R.J. et al.. "Microreact: Visualizing and sharing data for genomic epidemiology and phylogeography". Microbial Genomics 2 (11): e000093. doi:10.1099/mgen.0.000093. PMC PMC5320705. PMID 28348833. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5320705.
- ↑ Carroll, L.N.; Au, A.P.; Detwiler, L.T. et al.. "Visualization and analytics tools for infectious disease epidemiology: A systematic review". Journal of Biomedical Informatics 51: 287-98. doi:10.1016/j.jbi.2014.04.006. PMC PMC5734643. PMID 24747356. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5734643.
- ↑ Leslie, K.O.; Rosai, J.. "Standardization of the surgical pathology report: Formats, templates, and synoptic reports". Seminars in Diagnostic Pathology 11 (4): 253-7. PMID 7878300.
Notes
This presentation is faithful to the original, with only a few minor changes to presentation. In several cases the PubMed ID was missing and was added to make the reference more useful. The original article lists references alphabetically, but this version — by design — lists them in order of appearance.
