Lena Janousek

Lena Janousek

Master's Thesis

Machine learning based improvement of the diagnostic accuracy of an online symptom checker

Dr.-Ing. Felix Kluge, Prof. Dr. Björn Eskofier 

09/2020 – 02/2021

Inflammatory rheumatic diseases (IRD) often go along with unspecific symptoms that make the correct diagnosis difficult [1]. The European League Against Rheumatism (EULAR) passed clear recommendations that patients should be seen during 6 weeks after symptom onset [2] as early
treatment start is likely to improve the patient outcome significantly [3]. Various strategies have been identified [1, 4] to implement these recommendations, however diagnostic delays are high [5, 6]. There are different reasons for this delay, one of them is that the time until a patient is seeking medical advice is too long, due to lack of awareness of the disease [5]. Symptom checkers (SCs) foster patient-empowerment and could be used to optimise triage decisions in a personalised manner (the right appointment, for the right patient, at the right time). 90%
of patients with IRDs regularly use smartphones, 65% believe that using medical apps could be beneficial to improve their health [7] and 4% consulted the internet to check their symptoms [8] previous to a rheumatologist appointment. SCs like artificial-intelligence-driven Ada (www.ada.com) have been used to complete more than 15 million health assessments in 130 countries [9] and a recent publication suggests that the diagnostic delay could significantly be reduced even for rare diseases when using digital SCs [10]. Another SC is Rheport (www.rheport.de) which has been specifically designed for IRD identification. Current research (data not published) shows that sensitivity and specificity of Rheport regarding IRDs is limited, however the majority of patients would recommend the use of SC and their usability was well perceived.

The current Rheport algorithm is rule-based and builds on subjective expert knowledge. Machine learning (ML), a way of mathematically modelling patterns in available training data and making predictions on new data could help to improve the algorithm. There are a number of general (not specifically designed for IRD) SCs that use ML techniques. These SCs include buoy (www.buoyhealth.com) and Isabel (www.isabelhealthcare.com), among others. SCs for IRD, however, are mostly simple questionnaires as they can be found on Rheuma-Check (www.rheumacheck.rheumanet.org/questionnaire.aspx) or Arthritis (www.arthritis.ca). It has recently been shown that the use of ML can outperform conventional techniques, which has not been shown for IRD yet [11, 12].

For a classification problem like distinguishing between patients with and without IRD, a number of ML algorithms could be used including Naive Bayes Classifier, Decision Trees, Support Vector Machines or Neural Networks in order to improve sensitivity and specificity of Rheport. Based
on structure, quality and quantity of data, the best algorithm for performing the required task needs to be determined. The results will be compared to previously determined performances. As Rheport is currently being used by several German rheumatologists, the results can directly
improve the quality of life of patients with IRD. The aim of this project is to improve Rheport’s algorithm using ML based on self-reported multicentric real-world data.


[1] Benesova K, Lorenz HM, Lion V, Voigt A, Krause A, Sander O, et al.: Früh- und Screeningsprechstunden: Ein notwendigerWeg zur besseren Frühversorgung in der internistischen Rheumatologie? Zeitschrift für Rheumatologie. 2019;78(8):722-42.
[2] Combe, Bernard and Landewe, Robert and Daien, Claire I and Hua, Charlotte and Aletaha, Daniel and Álvaro-Gracia et al.: 2016 update of the EULAR recommendations for the management of early arthritis. Annals of the Rheumatic Diseases. 2017;76(6):948.
[3] Quinn MA, Emery P.: Window of opportunity in early rheumatoid arthritis: possibility of altering the disease process with early intervention. Clin Exp Rheumatol. 2003;21(0392- 856X (Print)):154-7.
[4] Villeneuve E, Nam Jl Fau – Bell MJ, Bell Mj Fau – Deighton CM, Deighton Cm Fau – Felson DT, Felson Dt Fau – Hazes JM, Hazes Jm Fau – McInnes IB, et al.: A systematic literature review of strategies promoting early referral and reducing delays in the diagnosis and management of inflammatory arthritis. Annals of the rheumatic diseases. 2012;72(1468-2060 (Electronic)):13-22.
[5] Raza K, Stack R, Kumar K, Filer A, Detert J, Bastian H, et al.: Delays in assessment of patients with rheumatoid arthritis: variations across Europe. Annals of the rheumatic diseases. 2011;70(10):1822.
[6] Stack RJ, Nightingale P, Jinks C, Shaw K, Herron-Marx S, Horne R, et al.: Delays between the onset of symptoms and first rheumatology consultation in patients with rheumatoid arthritis in the UK: an observational study. BMJ Open. 2019;9(3):e024361.
[7] Knitza J, Raab C, Lambrecht A, Simon D, Hagen M, Bayat S, et al.: The Urge for Mobile Apps in Rheumatology – a German Patient Perspective [abstract]. Arthritis Rheumatol. 2019;71 (suppl 10).
[8] Powley L, McIlroy G, Simons G, Raza K.: Are online symptoms checkers useful for patients with inflammatory arthritis? BMC musculoskeletal disorders. 2016;17(1):362.
[9] Butcher M. Ada: Health built an AI-driven startup by moving slowly and not breaking things Techcrunch; 2020 [Available from: https://techcrunch.com/2020/03/05/move-slowand-dont-break-things-how-to-build-an-ai-driven-startup/].
[10] Ronicke S, Hirsch MC, Turk E, Larionov K, Tientcheu D, Wagner AD.: Can a decision support system accelerate rare disease diagnosis? Evaluating the potential impact of Ada DX in a retrospective study. Orphanet J Rare Dis. 2019;14(1):69.
[11] Kao H, Tang K, Chang E.: Context-Aware Symptom Checking for Disease Diagnosis Using Hierarchical Reinforcement Learning. AAAI Conference on Artificial Intelligence. 2018.
[12] Akrout M, Farahmand A, Jarmain T, Abid L.: Improving Skin Condition Classification with a Visual Symptom Checker Trained Using Reinforcement Learning. Medical Image Computing and Computer Assisted Intervention. 2019.