Pacis, D. M. M., Subido Jr, E. D. & Bugtai, N. T. Trends in telemedicine utilizing artificial intelligence. In International Symposium of Biomedical Engineering, Vol. 1933, 040009 (AIP Publishing LLC, 2018).
Kuziemsky, C. et al. Role of artificial intelligence within the telehealth domain. Yearb. Med. Inform. 28, 035–040 (2019).
Google Scholar
Webster, P. Virtual health care in the era of covid-19. Lancet 395, 1180–1181 (2020).
Google Scholar
Esteva, A. et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature 542, 115–118 (2017).
Google Scholar
Tschandl, P. et al. Human–computer collaboration for skin cancer recognition. Nat. Med. 26, 1229–1234 (2020).
Google Scholar
Ghunaim, H., Weekes, T. R. & Eskridge, T. C. Designing an AI assistant for student telehealth: a case study using human-centered design. In International Conference on Human–Machine Systems (ICHMS), 1–6 (IEEE, 2022).
Soberanis-Mukul, R. et al. A novel method to identify streptococcus pharyngitis using smart-phone images and artificial intelligence. In American Academy of Pediatrics National Conference (2022).
Haggenmüller, V. et al. Smartphone-based point-of-care anemia screening in rural Bihar in India. Commun. Med. 3, 38 (2023).
Google Scholar
Lebovitz, S., Lifshitz-Assaf, H. & Levina, N. To engage or not to engage with AI for critical judgments: how professionals deal with opacity when using AI for medical diagnosis. Organ. Sci. 33, 126–148 (2022).
Google Scholar
Markus, A. F., Kors, J. A. & Rijnbeek, P. R. The role of explainability in creating trustworthy artificial intelligence for health care: a comprehensive survey of the terminology, design choices, and evaluation strategies. J. Biomed. Inform. 113, 103655 (2021).
Google Scholar
Amann, J., Blasimme, A., Vayena, E., Frey, D. & Madai, V. I. Explainability for artificial intelligence in healthcare: a multidisciplinary perspective. BMC Med. Inform. Decision Mak. 20, 1–9 (2020).
Vellido, A. The importance of interpretability and visualization in machine learning for applications in medicine and health care. Neural Comput. Appl. 32, 18069–18083 (2020).
Google Scholar
Sarkaria, S., Rebiffe, S., Gupta, U., Maliapen, M. & Whitehead, P. Image processing of streptococcal infection in pharyngitis subjects. US Patent No. 11,602,312 B2 (2022).
Canares, T. Machine learning systems and related aspects for the detection of disease states. US Patent No. 63,375,978 (2022).
Ebell, M. H., Smith, M. A., Barry, H. C., Ives, K. & Carey, M. Does this patient have strep throat? JAMA 284, 2912–2918 (2000).
Google Scholar
Fine, A. M., Nizet, V. & Mandl, K. D. Large-scale validation of the centor and McIsaac scores to predict group a streptococcal pharyngitis. Arch. Intern. Med. 172, 847–852 (2012).
Google Scholar
Centor, R. M., Witherspoon, J. M., Dalton, H. P., Brody, C. E. & Link, K. The diagnosis of strep throat in adults in the emergency room. Med. Decision Mak. 1, 239–246 (1981).
Google Scholar
Seeley, A. et al. Diagnostic accuracy of FEVER-PAIN and Centor criteria for bacterial throat infection in adults with sore throat: a secondary analysis of a randomised controlled trial. BJGP Open 5, no. 6 (2021).
Hjortdahl, P., Lærum, E. & Mowinckel, P. Clinical assessment of pharyngitis in general practice. Scand. J. Prim. Health Care 6, 219–223 (1988).
Google Scholar
Mohseni, S., Zarei, N. & Ragan, E. D. A multidisciplinary survey and framework for design and evaluation of explainable AI systems. ACM Trans. Interact. Intell. Syst. 11, 1–45 (2021).
Google Scholar
Tonekaboni, S., Joshi, S., McCradden, M. D. & Goldenberg, A. What clinicians want: contextualizing explainable machine learning for clinical end use. In Machine Learning for Healthcare Conference 359–380 (Proceedings of Machine Learning Research, 2019).
Xie, Y., Chen, M., Kao, D., Gao, G. & Chen, X. Chexplain: enabling physicians to explore and understand data-driven, ai-enabled medical imaging analysis. In Proc. 2020 CHI Conference on Human Factors in Computing Systems 1–13 (Association for Computing Machinery, 2020).
Jacobs, M. et al. Designing AI for trust and collaboration in time-constrained medical decisions: a sociotechnical lens. In Proc. 2021 CHI Conference on Human Factors in Computing Systems 1–14 (Association for Computing Machinery, 2021).
Cai, C. J. et al. Human-centered tools for coping with imperfect algorithms during medical decision-making. In Proc. 2019 CHI Conference on Human Factors in Computing Systems 1–14 (Association for Computing Machinery, 2019).
Thieme, A. et al. Designing human-centered AI for mental health: Developing clinically relevant applications for online CBT treatment. ACM Trans. Comput.–Hum. Interact. 30, 1–50 (2022).
Google Scholar
Reyes, M. et al. On the interpretability of artificial intelligence in radiology: challenges and opportunities. Radiology: Artif. Intell. 2, e190043 (2020).
Tjoa, E. & Guan, C. A survey on explainable artificial intelligence (XAI): toward medical XAI. IEEE Trans. Neural Netw. Learn. Syst. 32, 4793–4813 (2020).
Google Scholar
Van der Velden, B. H., Kuijf, H. J., Gilhuijs, K. G. & Viergever, M. A. Explainable artificial intelligence (XAI) in deep learning-based medical image analysis. Med. Image Anal. 79, 102470 (2022).
Google Scholar
Antoniadi, A. M. et al. Current challenges and future opportunities for XAI in machine learning-based clinical decision support systems: a systematic review. Appl. Sci. 11, 5088 (2021).
Google Scholar
Nazar, M., Alam, M. M., Yafi, E. & Su’ud, M. M. A systematic review of human–computer interaction and explainable artificial intelligence in healthcare with artificial intelligence techniques. IEEE Access 9, 153316–153348 (2021).
Google Scholar
Chen, H., Gomez, C., Huang, C.-M. & Unberath, M. Explainable medical imaging ai needs human-centered design: guidelines and evidence from a systematic review. npj Digit. Med. 5, 1–15 (2022).
Google Scholar
Schoonderwoerd, T. A., Jorritsma, W., Neerincx, M. A. & Van Den Bosch, K. Human-centered XAI: developing design patterns for explanations of clinical decision support systems. Int. J. Hum.–Comput. Stud. 154, 102684 (2021).
Google Scholar
Calisto, F. M., Santiago, C., Nunes, N. & Nascimento, J. C. Breastscreening-AI: evaluating medical intelligent agents for human–AI interactions. Artif. Intell. Med. 127, 102285 (2022).
Google Scholar
Gaube, S. et al. Non-task expert physicians benefit from correct explainable AI advice when reviewing x-rays. Sci. Rep. 13, 1383 (2023).
Google Scholar
Wang, D., Yang, Q., Abdul, A. & Lim, B. Y. Designing theory-driven user-centric explainable AI. In Proc. 2019 CHI Conference on Human Factors in Computing Systems 1–15 (Association for Computing Machinery, 2019).
Lee, M. H., Siewiorek, D. P., Smailagic, A., Bernardino, A. & Bermúdez i Badia, S. A human–AI collaborative approach for clinical decision making on rehabilitation assessment. In Proc. 2021 CHI Conference on Human Factors in Computing Systems 1–14 (Association for Computing Machinery, 2021).
Panigutti, C., Perotti, A. & Pedreschi, D. Doctor XAI: an ontology-based approach to black-box sequential data classification explanations. In Proc. 2020 Conference on Fairness, Accountability, and Transparency 629–639 (2020).
Jacobs, M. et al. How machine-learning recommendations influence clinician treatment selections: the example of antidepressant selection. Transl. Psychiatry 11, 108 (2021).
Google Scholar
Tsai, C.-H., You, Y., Gui, X., Kou, Y. & Carroll, J. M. Exploring and promoting diagnostic transparency and explainability in online symptom checkers. In Proc. 2021 CHI Conference on Human Factors in Computing Systems 1–17 (Association for Computing Machinery, 2021).
Du, Y., Antoniadi, A. M., McNestry, C., McAuliffe, F. M. & Mooney, C. The role of xai in advice-taking from a clinical decision support system: a comparative user study of feature contribution-based and example-based explanations. Appl. Sci. 12, 10323 (2022).
Google Scholar
Suresh, H., Lewis, K. M., Guttag, J. & Satyanarayan, A. Intuitively assessing ML model reliability through example-based explanations and editing model inputs. In 27th International Conference on Intelligent User Interfaces 767–781 (Association for Computing Machinery, 2022).
Bhattacharya, A., Ooge, J., Stiglic, G. & Verbert, K. Directive explanations for monitoring the risk of diabetes onset: introducing directive data-centric explanations and combinations to support what-if explorations. In Proc. 28th International Conference on Intelligent User Interfaces 204–219 (Association for Computing Machinery, 2023).
Wang, R., Fu, G., Li, J. & Pei, Y. Diagnosis after zooming in: a multilabel classification model by imitating doctor reading habits to diagnose brain diseases. Med. Phys. 49, 7054–7070 (2022).
Google Scholar
Jungmann, F. et al. Algorithmic transparency and interpretability measures improve radiologists’ performance in BI-RADS 4 classification. Eur. Radiol. 33, 1844–1851 (2023).
Google Scholar
Naiseh, M., Al-Thani, D., Jiang, N. & Ali, R. How the different explanation classes impact trust calibration: the case of clinical decision support systems. Int. J. Hum.–Comput. Stud. 169, 102941 (2023).
Google Scholar
Hwang, J., Lee, T., Lee, H. & Byun, S. A clinical decision support system for sleep staging tasks with explanations from artificial intelligence: user-centered design and evaluation study. J. Med. Internet Res. 24, e28659 (2022).
Google Scholar
Cabitza, F. et al. Painting the black box white: experimental findings from applying xai to an ECG reading setting. Mach. Learn. Knowl. Extr. 5, 269–286 (2023).
Google Scholar
Hah, H. & Goldin, D. Moving toward ai-assisted decision-making: observation on clinicians’ management of multimedia patient information in synchronous and asynchronous telehealth contexts. Health Inform. J. 28, 14604582221077049 (2022).
Google Scholar
Jiang, J., Kahai, S. & Yang, M. Who needs explanation and when? Juggling explainable AI and user epistemic uncertainty. Int. J. Hum.–Comput. Stud. 165, 102839 (2022).
Google Scholar
Yang, Q., Steinfeld, A., Rosé, C. & Zimmerman, J. Re-examining whether, why, and how human–AI interaction is uniquely difficult to design. In Proc. 2020 CHI Conference on Human Factors in Computing Systems 1–13 (Association for Computing Machinery, 2020).
Panigutti, C., Beretta, A., Giannotti, F. & Pedreschi, D. Understanding the impact of explanations on advice-taking: a user study for ai-based clinical decision support systems. In CHI Conference on Human Factors in Computing Systems 1–9 (Association for Computing Machinery, 2022).
McIsaac, W. J., Kellner, J. D., Aufricht, P., Vanjaka, A. & Low, D. E. Empirical validation of guidelines for the management of pharyngitis in children and adults. JAMA 291, 1587–1595 (2004).
Google Scholar
Scott, I., Carter, S. & Coiera, E. Clinician checklist for assessing suitability of machine learning applications in healthcare. BMJ Health & Care Informatics. 28, e100251 (2021).
Yin, M., Wortman Vaughan, J. & Wallach, H. Understanding the effect of accuracy on trust in machine learning models. In Proc. 2019 CHI Conference on Human Factors in Computing Systems 1–12 (Association for Computing Machinery, 2019).
Molnar, C. Interpretable Machine Learning (Lulu. com, 2020).
Lundberg, S. M. & Lee, S.-I. A unified approach to interpreting model predictions. Advances in Neural Information Processing Systems, 30, 4768–4777 (2017).
Cohen, J. F., Bertille, N., Cohen, R. & Chalumeau, M. Rapid antigen detection test for group a streptococcus in children with pharyngitis. Cochrane Database of Systematic Reviews (2016).
Harris, P. A. et al. Research electronic data capture (redcap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J. Biomed. Inform. 42, 377–381 (2009).
Google Scholar
Harris, P. A. et al. The redcap consortium: building an international community of software platform partners. J. Biomed. Inform. 95, 103208 (2019).
Google Scholar
Vodrahalli, K., Daneshjou, R., Gerstenberg, T. & Zou, J. Do humans trust advice more if it comes from AI? An analysis of human-ai interactions. In Proc. 2022 AAAI/ACM Conference on AI, Ethics, and Society 763–777 (Association for Computing Machinery, 2022).
Lundberg, S. M. et al. Explainable machine-learning predictions for the prevention of hypoxaemia during surgery. Nat. Biomed. Eng. 2, 749–760 (2018).
Google Scholar
Dratsch, T. et al. Automation bias in mammography: the impact of artificial intelligence BI-RADS suggestions on reader performance. Radiology 307, e222176 (2023).
Google Scholar
Bussone, A., Stumpf, S. & O’Sullivan, D. The role of explanations on trust and reliance in clinical decision support systems. In 2015 International Conference on Healthcare Informatics 160–169 (IEEE, 2015).
Levy, A., Agrawal, M., Satyanarayan, A. & Sontag, D. Assessing the impact of automated suggestions on decision making: domain experts mediate model errors but take less initiative. In Proc. 2021 CHI Conference on Human Factors in Computing Systems 1–13 (Association for Computing Machinery, 2021).
Gomez, C., Unberath, M. & Huang, C.-M. Mitigating knowledge imbalance in ai-advised decision-making through collaborative user involvement. Int. J. Hum.–Comput. Stud. 172, 102977 (2023).
Google Scholar
Wang, X. & Yin, M. Are explanations helpful? A comparative study of the effects of explanations in AI-assisted decision-making. In 26th International Conference on Intelligent User Interfaces 318–328 (Association for Computing Machinery, 2021).
Adam, H., Balagopalan, A., Alsentzer, E., Christia, F. & Ghassemi, M. Mitigating the impact of biased artificial intelligence in emergency decision-making. Commun. Med. 2, 149 (2022).
Google Scholar
Zhang, Y., Liao, Q. V. & Bellamy, R. K. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proc. 2020 Conference on Fairness, Accountability, and Transparency 295–305 (2020).
Cao, S. & Huang, C.-M. Understanding user reliance on AI in assisted decision-making. Proc. ACM Hum.–Comput. Interact. 6, 1–23 (2022).
Google Scholar
Papenmeier, A., Kern, D., Englebienne, G. & Seifert, C. It’s complicated: the relationship between user trust, model accuracy and explanations in AI. ACM Trans. Comput.–Hum. Interact. 29, 1–33 (2022).
Google Scholar
Bisno, A. L. Diagnosing strep throat in the adult patient: do clinical criteria really suffice? Ann. Intern. Med. 139, 150–151 (2003).
Google Scholar
link