Difference between revisions of "Journal:PathEdEx – Uncovering high-explanatory visual diagnostics heuristics using digital pathology and multiscale gaze data"

From LIMSWiki
Jump to navigationJump to search
(Saving and adding more.)
(Saving and adding more.)
Line 39: Line 39:
Pathology diagnosis is a highly complex process where multiple clinical and diagnostic factors have to be taken into account in an iterative fashion to produce a plausible conclusion that most accurately explains these factors from a biological standpoint.<ref name="ShinAPath12">{{cite journal |title=A pathologist-in-the-loop IHC antibody test selection using the entropy-based probabilistic method |journal=Journal of Pathology Informatics |author=Shin, D.; Arthur, G.; Caldwell, C. et al. |volume=3 |pages=1 |year=2012 |doi=10.4103/2153-3539.93393 |pmid=22439121 |pmc=PMC3307231}}</ref> [[Information]] from patient clinical history; morphological findings from microscopic evaluation of biopsies, aspirates, and smears; as well as data from flow cytometry, immunohistochemistry (IHC), and "omics" modalities such as comparative hybridization arrays and next-generation sequencing are used in the diagnostic process, which currently can be described more like a subjective exercise than a well-defined protocol. As such, it can frequently lead to diagnostic pitfalls which may harmfully impact patient case with a wrong diagnosis. The diagnostic pitfalls are most often encountered in the case of complex diseases such as cancer<ref name="HigginsApplication08">{{cite journal |title=Application of immunohistochemistry in the diagnosis of non-Hodgkin and Hodgkin lymphoma |journal=Archives of Pathology & Laboratory Medicine |author=Higgins, R.A.; Blankenship, J.E.; Kinney, M.C. |volume=132 |issue=3 |pages=441-61 |year=2008 |doi=10.1043/1543-2165(2008)132[441:AOIITD]2.0.CO;2 |pmid=18318586}}</ref>, where multiple fine-grained clinical phenotypes may require a better understanding of genomic variations across patient populations and therefore require better protocols for pathology diagnosis. It is especially important in light of realizing precision medicine ideas.<ref name="TenenbaumAnInformatics16">{{cite journal |title=An informatics research agenda to support precision medicine: Seven key areas |journal=JAMIA |author=Tenenbaum, J.D.; Avillach, P.; Benham-Hutchins, M. et al. |volume=23 |issue=4 |pages=791-5 |year=2016 |doi=10.1093/jamia/ocv213 |pmid=27107452 |pmc=PMC4926738}}</ref> A better understanding of pathology diagnosis, especially of heuristics of visual reasoning over microscopic slides, is crucial to develop means for new genomic-enabled precision diagnostics methods.
Pathology diagnosis is a highly complex process where multiple clinical and diagnostic factors have to be taken into account in an iterative fashion to produce a plausible conclusion that most accurately explains these factors from a biological standpoint.<ref name="ShinAPath12">{{cite journal |title=A pathologist-in-the-loop IHC antibody test selection using the entropy-based probabilistic method |journal=Journal of Pathology Informatics |author=Shin, D.; Arthur, G.; Caldwell, C. et al. |volume=3 |pages=1 |year=2012 |doi=10.4103/2153-3539.93393 |pmid=22439121 |pmc=PMC3307231}}</ref> [[Information]] from patient clinical history; morphological findings from microscopic evaluation of biopsies, aspirates, and smears; as well as data from flow cytometry, immunohistochemistry (IHC), and "omics" modalities such as comparative hybridization arrays and next-generation sequencing are used in the diagnostic process, which currently can be described more like a subjective exercise than a well-defined protocol. As such, it can frequently lead to diagnostic pitfalls which may harmfully impact patient case with a wrong diagnosis. The diagnostic pitfalls are most often encountered in the case of complex diseases such as cancer<ref name="HigginsApplication08">{{cite journal |title=Application of immunohistochemistry in the diagnosis of non-Hodgkin and Hodgkin lymphoma |journal=Archives of Pathology & Laboratory Medicine |author=Higgins, R.A.; Blankenship, J.E.; Kinney, M.C. |volume=132 |issue=3 |pages=441-61 |year=2008 |doi=10.1043/1543-2165(2008)132[441:AOIITD]2.0.CO;2 |pmid=18318586}}</ref>, where multiple fine-grained clinical phenotypes may require a better understanding of genomic variations across patient populations and therefore require better protocols for pathology diagnosis. It is especially important in light of realizing precision medicine ideas.<ref name="TenenbaumAnInformatics16">{{cite journal |title=An informatics research agenda to support precision medicine: Seven key areas |journal=JAMIA |author=Tenenbaum, J.D.; Avillach, P.; Benham-Hutchins, M. et al. |volume=23 |issue=4 |pages=791-5 |year=2016 |doi=10.1093/jamia/ocv213 |pmid=27107452 |pmc=PMC4926738}}</ref> A better understanding of pathology diagnosis, especially of heuristics of visual reasoning over microscopic slides, is crucial to develop means for new genomic-enabled precision diagnostics methods.


Over the last decade, digital pathology (DP) and whole slide imaging (WSI) have become a mature technology that allows reproduction of the histopathologic glass slide in its entirety.<ref name="PantanowitzDigital10">{{cite journal |title=Digital images and the future of digital pathology |journal=Journal of Pathology Informatics |author=Pantanowitz, L. |volume=1 |pages=15 |year=2010 |doi=10.4103/2153-3539.68332 |pmid=20922032 |pmc=PMC2941968}}</ref> For the first time in the history of microscopy, diagnostic-quality digital images can be stored electronically and analyzed using computer algorithms to assist primary diagnosis and streamline research in biomedical [[imaging informatics]].<ref name="ChenHistological14">{{cite journal |title=Histological quantitation of brain injury using whole slide imaging: a pilot validation study in mice |journal=PLOS One |author=Chen, Z.; Shin, D.; Chen, S. et al. |volume=9 |issue=3 |pages=e92133 |year=2014 |doi=10.1371/journal.pone.0092133 |pmid=24637518 |pmc=PMC3956884}}</ref> This was not possible before the DP era, when cropped image areas from pathology slides could be used only to seek second opinions or share the diagnostic details with clinicians. At the same time, gaze-capturing devices have undergone transformation from bulky systems to portable, mobile trackers that can be installed on laptops and used to seamlessly collect a user's gaze.
Over the last decade, digital pathology (DP) and whole slide imaging (WSI) have become a mature technology that allows reproduction of the histopathologic glass slide in its entirety.<ref name="PantanowitzDigital10">{{cite journal |title=Digital images and the future of digital pathology |journal=Journal of Pathology Informatics |author=Pantanowitz, L. |volume=1 |pages=15 |year=2010 |doi=10.4103/2153-3539.68332 |pmid=20922032 |pmc=PMC2941968}}</ref> For the first time in the history of microscopy, diagnostic-quality digital images can be stored electronically and analyzed using computer algorithms to assist primary diagnosis and streamline research in biomedical [[imaging informatics]].<ref name="ChenHistological14">{{cite journal |title=Histological quantitation of brain injury using whole slide imaging: A pilot validation study in mice |journal=PLOS One |author=Chen, Z.; Shin, D.; Chen, S. et al. |volume=9 |issue=3 |pages=e92133 |year=2014 |doi=10.1371/journal.pone.0092133 |pmid=24637518 |pmc=PMC3956884}}</ref> This was not possible before the DP era, when cropped image areas from pathology slides could be used only to seek second opinions or share the diagnostic details with clinicians. At the same time, gaze-capturing devices have undergone transformation from bulky systems to portable, mobile trackers that can be installed on laptops and used to seamlessly collect a user's gaze.


There have been a number of research studies where WSI was used to analyze visual patterns of regions of interests (ROIs) annotated without [6],[7],[8],[9] and with gaze-tracking technology.[10] In the former case, ROIs were identified either manually or using a viewport analysis, and in the latter case, gaze fixation points were used for the same purpose. Gaze tracking along with mouse movement has also been used to study pathologists' attention while viewing WSI during pathology diagnosis [11] as well as to study visual and cognitive aspects of pathology expertise.[12] The underlying idea in majority of these studies was to analyze captured gaze data with respect to the ROIs marked in the pathology images, gaze time spent within the ROIs, and total number of fixations within the ROIs as measures of diagnostically relevant viewing behavior. However, it is challenging to use ROIs to articulate underlying biology to understand visual heuristics of specific diagnostic decisions. It is not trivial to encode morphological patterns of an ROI using narrative language. As such, ROI might not be an effective means to translate best practices findings from gaze-tracking studies into the pathology education and pave a road toward precision diagnostics.
There have been a number of research studies where WSI was used to analyze visual patterns of regions of interests (ROIs) annotated without<ref name="RederNDER16">{{cite journal |title=NDER: A novel web application using annotated whole slide images for rapid improvements in human pattern recognition |journal=Journal of Pathology Informatics |author=Reder, N.P.; Glasser, D.; Dintzis, S.M. et al. |volume=7 |pages=31 |year=2016 |doi=10.4103/2153-3539.186913 |pmid=27563490 |pmc=PMC4977980}}</ref><ref name="MercanLocalization16">{{cite journal |title=Localization of Diagnostically Relevant Regions of Interest in Whole Slide Images: A Comparative Study |journal=Journal of Digital Imaging |author=Mercan, E.; Aksoy, S.; Shapiro, L.G. et al. |volume=29 |issue=4 |pages=496-506 |year=2016 |doi=10.1007/s10278-016-9873-1 |pmid=26961982 |pmc=PMC4942394}}</ref><ref name="MercanMulti16">{{cite journal |title=Multi-instance multi-label learning for whole slide breast histopathology |journal=SPIE Proceedings |author=Mercan, C.; Mercan, E.; Aksoy, S. et al. |volume=9791 |pages=979108 |year=2016 |doi=10.1117/12.2216458}}</ref><ref name="Roa-PeñaAmExperi10">{{cite journal |title=An experimental study of pathologist's navigation patterns in virtual microscopy |journal=Diagnostic Pathology |author=Roa-Peña, L.; Gómez, F.; Romero, E. |volume=5 |pages=71 |year=2010 |doi=10.1186/1746-1596-5-71 |pmid=21087502 |pmc=PMC3001424}}</ref> and with gaze-tracking technology.<ref name="BrunyéEye14">{{cite journal |title=Eye movements as an index of pathologist visual expertise: A pilot study |journal=PLOS One |author=Brunyé,T.T.; Carney, P.A.; Allison, K.H. et al. |volume=9 |issue=8 |pages=e103447 |year=2014 |doi=10.1371/journal.pone.0103447 |pmid=25084012 |pmc=PMC4118873}}</ref> In the former case, ROIs were identified either manually or using a viewport analysis, and in the latter case, gaze fixation points were used for the same purpose. Gaze tracking, along with mouse movement, has also been used to study pathologists' attention while viewing WSI during pathology diagnosis<ref name="RaghunathMouse12">{{cite journal |title=Mouse cursor movement and eye tracking data as an indicator of pathologists' attention when viewing digital whole slide images |journal=Journal of Pathology Informatics |author=Raghunath, V.; Braxton, M.O.; Gagnon, S.A. et al. |volume=3 |pages=43 |year=2012 |doi=10.4103/2153-3539.104905 |pmid=23372984 |pmc=PMC3551530}}</ref> as well as to study visual and cognitive aspects of pathology expertise.<ref name="JaarsmaExpertise15">{{cite journal |title=Expertise in clinical pathology: combining the visual and cognitive perspective |journal=Journal of Pathology Informatics |author=Jaarsma, T; Jarodzka, H.; Nap, M. et al. |volume=20 |issue=4 |pages=1089-106 |year=2015 |doi=10.1007/s10459-015-9589-x |pmid=25677013 |pmc=PMC4564442}}</ref> The underlying idea in the majority of these studies was to analyze captured gaze data with respect to the ROIs marked in the pathology images, gaze time spent within the ROIs, and total number of fixations within the ROIs as measures of diagnostically relevant viewing behavior. However, it is challenging to use ROIs to articulate underlying biology for understanding the visual heuristics of specific diagnostic decisions. It is not trivial to encode morphological patterns of an ROI using narrative language. As such, ROI might not be an effective means to translate best practices findings from gaze-tracking studies into pathology education and pave a road toward precision diagnostics.


==References==
==References==

Revision as of 22:24, 26 July 2017

Full article title PathEdEx – Uncovering high-explanatory visual diagnostics heuristics using digital pathology and multiscale gaze data
Journal Journal of Pathology Informatics
Author(s) Shin, Dmitriy; Kovalenko, Mikhail; Ersoy, Ilker; Li, Yu; Doll, Donald; Shyu, Chi-Ren; Hammer, Richard
Author affiliation(s) University of Missouri – Columbia
Primary contact Email: Available w/ login
Year published 2017
Volume and issue 8
Page(s) 29
DOI 10.4103/jpi.jpi_29_17
ISSN 2153-3539
Distribution license http://www.jpathinformatics.org
Download http://www.jpathinformatics.org/temp/JPatholInform8129-6283361_172713.pdf (PDF)
This article should not be considered complete until this message box has been removed. This is a work in progress.

Abstract

Background: Visual heuristics of pathology diagnosis is a largely unexplored area where reported studies only provided a qualitative insight into the subject. Uncovering and quantifying pathology visual and non-visual diagnostic patterns have great potential to improve clinical outcomes and avoid diagnostic pitfalls.

Methods: Here, we present PathEdEx, an informatics computational framework that incorporates whole-slide digital pathology imaging with multiscale gaze-tracking technology to create web-based interactive pathology educational atlases and to datamine visual and non-visual diagnostic heuristics.

Results: We demonstrate the capabilities of PathEdEx for mining visual and non-visual diagnostic heuristics using the first PathEdEx volume of a hematopathology atlas. We conducted a quantitative study on the time dynamics of zooming and panning operations utilized by experts and novices to come to the correct diagnosis. We then performed association rule mining to determine sets of diagnostic factors that consistently result in a correct diagnosis, and studied differences in diagnostic strategies across different levels of pathology expertise using Markov chain (MC) modeling and MC Monte Carlo simulations. To perform these studies, we translated raw gaze points to high-explanatory semantic labels that represent pathology diagnostic clues. Therefore, the outcome of these studies is readily transformed into narrative descriptors for direct use in pathology education and practice.

Conclusion: The PathEdEx framework can be used to capture best practices of pathology visual and non-visual diagnostic heuristics that can be passed over to the next generation of pathologists and have potential to streamline implementation of precision diagnostics in precision medicine settings.

Keywords: Digital pathology, eye tracking, gaze tracking, pathology diagnosis, visual heuristics, visual knowledge, whole slide images

Introduction

Pathology diagnosis is a highly complex process where multiple clinical and diagnostic factors have to be taken into account in an iterative fashion to produce a plausible conclusion that most accurately explains these factors from a biological standpoint.[1] Information from patient clinical history; morphological findings from microscopic evaluation of biopsies, aspirates, and smears; as well as data from flow cytometry, immunohistochemistry (IHC), and "omics" modalities such as comparative hybridization arrays and next-generation sequencing are used in the diagnostic process, which currently can be described more like a subjective exercise than a well-defined protocol. As such, it can frequently lead to diagnostic pitfalls which may harmfully impact patient case with a wrong diagnosis. The diagnostic pitfalls are most often encountered in the case of complex diseases such as cancer[2], where multiple fine-grained clinical phenotypes may require a better understanding of genomic variations across patient populations and therefore require better protocols for pathology diagnosis. It is especially important in light of realizing precision medicine ideas.[3] A better understanding of pathology diagnosis, especially of heuristics of visual reasoning over microscopic slides, is crucial to develop means for new genomic-enabled precision diagnostics methods.

Over the last decade, digital pathology (DP) and whole slide imaging (WSI) have become a mature technology that allows reproduction of the histopathologic glass slide in its entirety.[4] For the first time in the history of microscopy, diagnostic-quality digital images can be stored electronically and analyzed using computer algorithms to assist primary diagnosis and streamline research in biomedical imaging informatics.[5] This was not possible before the DP era, when cropped image areas from pathology slides could be used only to seek second opinions or share the diagnostic details with clinicians. At the same time, gaze-capturing devices have undergone transformation from bulky systems to portable, mobile trackers that can be installed on laptops and used to seamlessly collect a user's gaze.

There have been a number of research studies where WSI was used to analyze visual patterns of regions of interests (ROIs) annotated without[6][7][8][9] and with gaze-tracking technology.[10] In the former case, ROIs were identified either manually or using a viewport analysis, and in the latter case, gaze fixation points were used for the same purpose. Gaze tracking, along with mouse movement, has also been used to study pathologists' attention while viewing WSI during pathology diagnosis[11] as well as to study visual and cognitive aspects of pathology expertise.[12] The underlying idea in the majority of these studies was to analyze captured gaze data with respect to the ROIs marked in the pathology images, gaze time spent within the ROIs, and total number of fixations within the ROIs as measures of diagnostically relevant viewing behavior. However, it is challenging to use ROIs to articulate underlying biology for understanding the visual heuristics of specific diagnostic decisions. It is not trivial to encode morphological patterns of an ROI using narrative language. As such, ROI might not be an effective means to translate best practices findings from gaze-tracking studies into pathology education and pave a road toward precision diagnostics.

References

  1. ↑ Shin, D.; Arthur, G.; Caldwell, C. et al. (2012). "A pathologist-in-the-loop IHC antibody test selection using the entropy-based probabilistic method". Journal of Pathology Informatics 3: 1. doi:10.4103/2153-3539.93393. PMC PMC3307231. PMID 22439121. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3307231. 
  2. ↑ Higgins, R.A.; Blankenship, J.E.; Kinney, M.C. (2008). "Application of immunohistochemistry in the diagnosis of non-Hodgkin and Hodgkin lymphoma". Archives of Pathology & Laboratory Medicine 132 (3): 441-61. doi:10.1043/1543-2165(2008)132[441:AOIITD]2.0.CO;2. PMID 18318586. 
  3. ↑ Tenenbaum, J.D.; Avillach, P.; Benham-Hutchins, M. et al. (2016). "An informatics research agenda to support precision medicine: Seven key areas". JAMIA 23 (4): 791-5. doi:10.1093/jamia/ocv213. PMC PMC4926738. PMID 27107452. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4926738. 
  4. ↑ Pantanowitz, L. (2010). "Digital images and the future of digital pathology". Journal of Pathology Informatics 1: 15. doi:10.4103/2153-3539.68332. PMC PMC2941968. PMID 20922032. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2941968. 
  5. ↑ Chen, Z.; Shin, D.; Chen, S. et al. (2014). "Histological quantitation of brain injury using whole slide imaging: A pilot validation study in mice". PLOS One 9 (3): e92133. doi:10.1371/journal.pone.0092133. PMC PMC3956884. PMID 24637518. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3956884. 
  6. ↑ Reder, N.P.; Glasser, D.; Dintzis, S.M. et al. (2016). "NDER: A novel web application using annotated whole slide images for rapid improvements in human pattern recognition". Journal of Pathology Informatics 7: 31. doi:10.4103/2153-3539.186913. PMC PMC4977980. PMID 27563490. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4977980. 
  7. ↑ Mercan, E.; Aksoy, S.; Shapiro, L.G. et al. (2016). "Localization of Diagnostically Relevant Regions of Interest in Whole Slide Images: A Comparative Study". Journal of Digital Imaging 29 (4): 496-506. doi:10.1007/s10278-016-9873-1. PMC PMC4942394. PMID 26961982. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4942394. 
  8. ↑ Mercan, C.; Mercan, E.; Aksoy, S. et al. (2016). "Multi-instance multi-label learning for whole slide breast histopathology". SPIE Proceedings 9791: 979108. doi:10.1117/12.2216458. 
  9. ↑ Roa-Peña, L.; Gómez, F.; Romero, E. (2010). "An experimental study of pathologist's navigation patterns in virtual microscopy". Diagnostic Pathology 5: 71. doi:10.1186/1746-1596-5-71. PMC PMC3001424. PMID 21087502. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3001424. 
  10. ↑ Brunyé,T.T.; Carney, P.A.; Allison, K.H. et al. (2014). "Eye movements as an index of pathologist visual expertise: A pilot study". PLOS One 9 (8): e103447. doi:10.1371/journal.pone.0103447. PMC PMC4118873. PMID 25084012. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4118873. 
  11. ↑ Raghunath, V.; Braxton, M.O.; Gagnon, S.A. et al. (2012). "Mouse cursor movement and eye tracking data as an indicator of pathologists' attention when viewing digital whole slide images". Journal of Pathology Informatics 3: 43. doi:10.4103/2153-3539.104905. PMC PMC3551530. PMID 23372984. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3551530. 
  12. ↑ Jaarsma, T; Jarodzka, H.; Nap, M. et al. (2015). "Expertise in clinical pathology: combining the visual and cognitive perspective". Journal of Pathology Informatics 20 (4): 1089-106. doi:10.1007/s10459-015-9589-x. PMC PMC4564442. PMID 25677013. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4564442. 

Notes

This presentation is faithful to the original, with only a few minor changes to presentation and updates to spelling and grammar. In some cases important information was missing from the references, and that information was added.

Retrieved from "https://www.limswiki.org/index.php?title=Journal:PathEdEx_–_Uncovering_high-explanatory_visual_diagnostics_heuristics_using_digital_pathology_and_multiscale_gaze_data&oldid=31199"