Government-sponsored symptom checkers in both the US and UK failed to identify the symptoms of severe Covid-19, bacterial pneumonia and sepsis, advising these cases to stay at home and not seek urgent treatment.
The symptom trackers involved in the study were: the Singapore COVID-19 Symptom Checker; Japan’s Stop COVID-19 Symptom Checker; the US CDC Coronavirus Symptom Checker (US); and the 111 COVID-19 Symptom Checker from the UK.
The researchers, based at the University of Gibraltar’s Postgraduate School of Medicine, wanted to see if the symptom checkers could differentiate mild Covid-19 from severe Covid-19 and see how well they picked up diseases with similar symptoms, such as bacterial pneumonia and sepsis.
The team input 52 standardised case scenarios across the four programmes, with varying symptom severity and pre-existing risk factors.
The symptom checkers in Singapore and Japan triaged twice as many cases for direct clinical assessment as the symptom checkers in the US and UK.
Singapore’s checker had the highest overall referral rate at 88%, while the US had the lowest at 38%.
The Singapore checker will advise patients to seek medical assessment if they are aged over 65 years, have a pre-existing health condition or have had symptoms for more than four days, with any case of shortness of breath triaged directly to emergency care.
Japan’s symptom checker also performed well, but did reveal a potential delay to treatment of very severe sepsis.
The US symptom checker frequently advised those with cases of possible severe Covid-19, bacterial pneumonia and sepsis to stay at home, although it did advise simulated cases of possible neutropenic (low white cell count after chemotherapy) sepsis to seek medical attention within 24 hours.
The UK symptom checker frequently triaged severe Covid-19 and bacterial pneumonia cases to stay home with no follow-up, and was likely to delay treatment for sepsis, severe Covid-19 and severe sepsis.
None of the symptom checkers triaged simulated neutropenic sepsis to emergency care.
The researchers said: “Whilst ‘symptom checkers’ may be of use to the healthcare Covid-19 response, there is the potential for such patient-led assessment tools to worsen outcomes by delaying appropriate clinical assessment.
“Our results support the recommendation that symptom checkers should be subjected to the same level of evidence-based quality assurance as other diagnostic tests prior to implementation.”
This isn’t the first time research published in the BMJ has taken issue with symptom assessment apps.
A study published in BMJ Open in December found significant variance in the coverage, accuracy and safety of the eight most popular online symptom assessment apps.