Skip to main content
Download PDF

Safety and accuracy of digitally supported primary and secondary urgent care telephone triage in England: an observational study using routine data

BMC Medical Informatics and Decision Making volume 25, Article number: 52 (2025) Cite this article

Abstract

Background

England’s urgent care telephone triage system comprises non-clinician-led primary triage (NHS111) assessment followed, for approximately 50% patients, by clinician-led secondary triage. Digital decision support is utilised by both. We explore the system’s safety and accuracy relative to patients’ use of emergency departments (EDs) and in-patient care in the subsequent 24 h.

Methods

Descriptive analyses were used to investigate outcomes of 98,946 calls that underwent primary and secondary triage. We investigated sensitivity (safety) and specificity (efficiency/accuracy) in relation to subsequent ED attendance and in-patient hospital admission. Mixed effects regression models were used to explore potential under-estimation of clinical risk (under-triage).

Results

Sensitivity was greater in primary triage, whilst specificity was greater in secondary triage. The positive predictive value for attending ED after being assigned a triage urgency level of within 2 h was 46.0% for secondary triage compared to 20.7% for primary triage; for inpatient admission it was 18.0% and 9.2% respectively. 1.5% (n = 1468) patients triaged to same-day or less urgent care at secondary triage were subsequently admitted for in-patient care. In relation to in-patient admission within 24 h, there were greater odds of potential under-triage for calls made between midnight and 6am, and for shorter duration calls, respectively OR = 1.71; CI:1.32–2.21 and OR: 1.66, CI: 1.30–2.11. The service provider (e.g., service provider 2, OR = 5.61; CI:3.36–9.36) and individual clinician (OR covering the 95% midrange = 16.15) conducting triage were the characteristics most greatly associated with this potential under-triage; p < 0.001 for all.

Conclusions

Clinician-led urgent care triage is more accurate in identifying the likelihood of a need for ED or in-patient care than non-clinician triage. Non-clinician primary triage is risk averse, reflected in its high sensitivity but low specificity. Service and clinician characteristics associated with potential under-triage need further investigation to inform ways of improving the safety and effectiveness of urgent care telephone triage.

Clinical trial number

Not applicable.

Peer Review reports

Background

Urgent care is routinely accessed by patients via telephone helplines. Different staff types deliver these services [1]. A non-clinician (non-medially trained operator) led model triage is widely used in the UK; whilst internationally triage is typically conducted by clinicians (usually nurses) [2], examples include Australia’s HealthDirect [3] and AskMayoClinic telephone service based in the USA [4]. Although computerised decision support is used to facilitate triage, there is limited evidence about its accuracy and safety, particularly in relation to non-clinician triage [5].

Studies have found that non-clinician triage is overly risk averse [6, 7] and may increase workload for emergency/urgent care services that are already over-burdened [8,9,10]. Turnbull et al. described non-clinical call handlers deferring triage decisions to clinicians [6]. Lewis et al. similarly reported risk aversion in triage, with less than 70% of patients referred to the ambulance service or advised to attend an ED subsequently attending ED [7]. In addition, potential safety issues were identified with 11% of patients advised to self-care subsequently attending ED within 48 h; of these, 88% were assessed as having urgent needs in ED with 37% subsequently being hospitalised [7].

The unique two-step telephone triage model used in most of the UK (except Northern Ireland) provides opportunity to inform how telephone-based urgent care triage can be best delivered internationally. In England, the National Health Service (NHS) 111 telephone service provides patients access to urgent care, particularly during the period outside of general practice opening hours. The service triages over 50,000 calls daily [11]. Non-clinicians (non-medically trained call operators) conduct a primary triage based on the patient’s symptoms using the NHS Pathways algorithms [12]. Approximately 24% calls are referred directly to emergency care, 8% are triaged to a primary care service such as a general practice, dental service or pharmacy, and 15% are assessed as self-care. The remaining 50% are transferred to an urgent care provider [11], and typically are called back and re-triaged (‘secondary triage’) by a clinician, usually a nurse, to categories covering the clinical urgency and the service (e.g. attending urgent care centre or emergency department (ED), home visit, routine GP appointment) needed.

Our previous study reported that 74% of NHS111 primary triage outcomes were downgraded in urgency following clinician-led secondary triage [5]. We also found that about 12% calls were upgraded during secondary triage, raising concerns about the safety of NHS111 primary triage [13]. However, no studies have compared the accuracy of primary and secondary triage as delivered within the two-step triage model. Addressing this gap is important in identifying ways of improving the efficiency and safety of urgent care, and the consequences that triage decisions have on ED and urgent care workload and waiting times. Such evidence is needed to inform health systems regarding the use of non-clinically trained staff given the possibility that it may be seen as an solution to the increasing challenges in recruiting and retaining the clinical workforce [14, 15].

Here we compare the primary and secondary triage outcomes (urgency levels) assigned to a cohort of patients who attended an ED or received hospital in-patient care in the 24-hour period following a call to the NHS111 service. It was anticipated that patients with a primary or secondary triage outcome indicating more urgent clinical needs would be more likely to attend ED or be admitted for inpatient care than patients who had less urgent triage outcomes.

Objectives

  1. 1.

    To identify the frequency and characteristics of patients who attend ED or are admitted for inpatient care within 24 h post-secondary (clinician) telephone triage.

  2. 2.

    To describe and compare sensitivity and specificity of primary and secondary telephone triage outcomes for patients who subsequently attend ED or are admitted for inpatient care within 24 h post-secondary (clinician) triage.

  3. 3.

    To describe the characteristics associated with patients that were assigned a lower level of urgency at secondary triage and subsequently attended ED or were admitted for inpatient care within 24 h post-secondary (clinician) triage.

Methods

Study design and setting

This was an observational retrospective analysis of triage call records from three urgent care providers based in Northwest England. Call records were linked to Hospital Episode Statistics (HES) and Emergency Care Dataset (ECDS) which were provided by NHS England via data application request service (DARS: https://digital.nhs.uk/services/data-access-request-service-dars).

The participating urgent care providers each received NHS111 calls that have undergone primary triage by non-clinicians using the NHS Pathways digital triage software and been identified as having an urgent care need. They then used clinicians to conduct secondary triage supported by a different digital triage software (Odyssey); the services all had a similar skill mix in the clinicians conducting triage who are typically nurses. Further information about the digital triage software can be found in a supplementary file. We have previously described the patient population and the primary and secondary triage outcomes for this cohort [13].

Datasets and linkage

NHS patient number and date of birth of patients were used by NHS England DARS to link datasets which were supplied as a pseudonymised dataset with all identifiable data removed.

The HES data included date/time of inpatient admission, diagnoses, and discharge date; the ECDS data included date/time of ED attendance and an acuity level (triage urgency level, that is based on a face to face triage assessment that is undertaken at the time that the patient attends the ED) [16]. The ED acuity takes one of 5 levels of urgency: 1 ‘Immediate resuscitation level care’; 2 ‘very urgent care’; 3 ‘urgent care’; 4 ‘standard care’; and 5 ‘non-urgent care’. Patients assigned to standard care are considered as having non-emergency problems suitable for treatment in an urgent care centre or minor injuries unit [16].

Dataset variables are provided in a supplementary file.

Outcome measures

Key outcome measures were:

  1. 1)

    ED attendance and in-patient admission 24 h post-secondary triage.

  2. 2)

    ED acuity level [16].

  3. 3)

    Two indicators of potential under-triage (potential under-estimation of clinical risk in secondary triage) defined as: non-urgent secondary triage outcomes (care needs less urgent than “within 6 hours”) having been documented and:

Definition 1)

patients with ED acuity level indicative of a need for immediate or very urgent care.

Definition 2)

patient admitted for inpatient care 24 h post-secondary triage.

For all outcomes, a time-period of 24 h post-secondary triage was selected; during this period the patient’s use of ED or in-patient care is more likely to relate to the healthcare concern or symptoms the patient had presented with.

Secondary triage patient records

The dataset contained 98,946 patient records of primary and secondary triage outcomes for patients who made a call to NHS111 between 1 April 2019 to 1 October 2020, triaged by 253 clinicians. As previously reported [5], this included:

  • Patient, clinician and call information: call ID, anonymised patient number, anonymised clinician ID, sociodemographic characteristics, time/date of call, length of call, presenting problem.

  • Primary (non-clinician) triage recommendation and secondary triage outcome recorded by a clinician. These corresponded to one of seven urgency levels: ‘emergency’; ‘immediate care within 1 h’; ‘immediate care within 2 h’; ‘urgent care within 4–6 h’; ‘same-day care within 24 h’; ‘routine primary care appointment’; and ‘self-care/no urgency’ (including advice to contact a different service). The triage urgency level selected by the clinician undertaking secondary triage was used as a proxy for disposition (care service/timeframe recommendation, e.g. advice to attend ED now), as the dataset only included the latter for a minority of calls.

Analyses

Chi-squared tests were used to investigate differences between primary and secondary triage outcomes in relation to both the proportions of patients subsequently attending ED or being admitted and the acuity level assigned in patients that attended ED.

Safety and accuracy of primary and secondary triage outcomes were evaluated by assessing the diagnostic performance (sensitivity, specificity, positive and negative predictive values) of a given (or higher) level of triage urgency.

Two mixed effects regression models were used to explore potential under-triage (using both indicators defined in the outcomes section). Models included fixed effects: patient sex, age group, deprivation, and presenting symptom; service provider; day of week and time of day, number of calls triaged by the clinician within the full dataset (an indicator of the clinician’s familiarity with the digital triage tool) and call length. A random intercept for the clinician that conducted triage was included, enabling quantification of the variability between individual clinicians [17].

Analyses were conducted using Stata (version 18).

Results

In total, 20,745 (21.0%) patients attended ED and/or were admitted for inpatient care within 24 h post-secondary triage (Fig. 1). Of these, 12,814 (61.8%) only attended ED, 5955 (28.7%) were admitted to inpatient care from ED, and 1976 (9.5%) were directly admitted to inpatient care. For these patients, the most common problems that had been assessed at secondary triage were abdominal pain (n = 2697; 13.0%) and fever (n = 1360; 6.6%) (Table 1). Younger age groups made up a larger proportion of those attending ED and being discharged, and of those directly admitted: in children aged under 5 these were 24.2% and 38.1% of calls respectively.

Fig. 1
figure 1

Summary of patients’ use of ED and inpatient care within 24 h post-secondary triage

Full size image

In total, 59.3% (4703) patients admitted for inpatient care were discharged from hospital within one day; this comprised 3256 (54.6%) patients admitted from ED and 1447 (73.2%) patients admitted directly (without first attending ED). Characteristics of these patients are provided in supplementary file (T1).

Table 1 Characteristics of patients that attended ED and/or were admitted for inpatient care
Full size table

Primary and secondary triage urgency levels in relation to ED attendance and in-patient admission

Patients assigned a high urgency level at primary triage attended ED less frequently than those assigned equivalent urgency in secondary triage. For example, 20.8% (14616 of 70428 calls) of patients with primary triage levels indicating care needed within 2 h or less attended ED compared to 46.0% (9188 of 19984 calls) at similar urgency levels at secondary triage. Specificity (accuracy/efficiency indicator) was greater in secondary triage than primary triage at all urgency levels. For example, at care within one hour, specificity was 65.4% and 97.7%, and PPV was 24.0% and 66.7.5%, in primary and secondary triage respectively (Fig. 2, and supplementary file T2).

For in-patient admission a similar pattern was observed. For example, 9.2% (6503 of 70428 calls) of patients with primary triage levels indicating care needed within 2 h or less were admitted for inpatient care compared to 18.0% (n = 3592 of 19984 calls) at similar urgency levels in secondary triage. Specificity (efficiency indicator) was again greater in secondary triage than primary triage (Fig. 2, and supplementary file T3).

Sensitivity (safety/risk aversion) of primary triage was greater for both ED attendance and hospital admission than in secondary triage at all urgency levels. For example, at triage levels indicating care needed within 6 h, the sensitivity for ED attendance was 93.5% at primary triage compared to 80.4% at secondary triage (Fig. 2, and supplementary file T2), and 94.1% and 81.5% respectively for inpatient admission (Fig. 2, and supplementary file T3).

Fig. 2
figure 2

Sensitivity, specificity, and positive/negative predictive values of primary and secondary triage levels for patients attending ED within 24 h (see supplementary file T2 and T3 for underlying data)

Full size image

The ED-assigned acuity level was available for 86.3% (n = 16257) of those who attended (Table 2).

Except at the most urgent acuity levels, secondary triage urgency levels tended to be closer to those assigned in ED (Table 2). For example, for patients with an ED acuity of non-urgent or standard care, 24.6% (n = 1520/6182) had a secondary triage level equal to or less than same day care compared to only 7.6% (n = 470) for primary triage. However, for the 3003 patients assigned to immediate resuscitation or very urgent care in ED, only 698 (23.2%) had been assessed as needing care within 1 h at secondary triage compared to 1577 (52.5%) at primary triage.

Table 2 ED acuity assigned to patients in relation to primary and secondary triage outcomes assigned
Full size table

Exploring potential under-triage

1.7% calls (n = 1725/98946) were identified as meeting one or both definitions of potential-under-triage.

ED attendance

Of patients attending ED, 3003 (18.5%) were assigned to ‘very urgent’ or ‘immediate resuscitation’ level care on arrival in the ED, and of these 16.4% (n = 493) patients met the criteria of potential under-triage having been assigned a level less urgent than “within 6 hours” in secondary triage (definition 1, described in Methods).

Of these 493 potentially under-triaged patients, 50.9% (n = 251) were female; they were disproportionately in the youngest age groups with 36.5% (n = 180) related to children under 5; the most frequent presenting symptoms were high temperature (n = 81,15.9%), cough (n = 53,10.3%) and abdominal pain (n = 44,8.5%); and almost half (n = 246; 49.9%) were admitted for in-patient care. See supplementary tables (T4-T6) for primary and secondary outcomes assigned to these calls and patient characteristics.

Inpatient admission

Of 7931 calls where the patient was admitted for in-patient care, 1477 (18.6%) met the criteria for potential under-triage, having had a non-urgent secondary triage outcome (care needs less urgent than “within 6 hours”). 28.2% (n = 417) of these calls were about children aged under 5. The top 3 symptoms called about were abdominal pain (13.5%, n = 199), high temperature (10.0%, n = 147) and cough (6.6%, n = 97). Patient characteristics are shown in supplementary tables (T7-T9). For patients in the potentially under-triaged sub-cohort that were admitted to inpatient care, 63.4% (n = 937) were discharged within 1 day of admission; of these 7.0% (n = 66) were recorded as elective admissions.

Factors associated with potential under-triage

Patient factors

Greatest odds of potential under-triage were seen in patients with a main symptom of high temperature (model 1), OR 1.66 (1.01–2.73), p = 0.02; and diarrhoea (model 2), OR 1.81 (CI: 1.18–2.76), p < 0.001; reference symptom: abdominal pain. No other patient characteristics were associated with greater or lower odds of under-triage.

Call-related factors

Regression model 1 showed lower odds of potential under-triage (high urgency level assigned in ED, for calls that were assessed as low urgency in secondary triage) on weekend days.

Model 2 (potential under-triage, based on in-patient admission within 24 h of the call) indicated greater odds of under-triage for calls undergoing secondary triage in the early morning period (Midnight – 6am): OR = 1.71;CI:1.32–2.21 and in the shortest calls of 0–5 min in duration: OR: 1.66,CI:1.30–2.11, reference: 10–15 min duration; p < 0.001 for all.

Service and clinician level factors

There was service-specific variation associated with where patients were triaged. For example, compared to Service 3, the odds of potential under-triage at Service 2 were 7.14 (CI:3.74–13.63,p < 0.001) for definition 1 and 5.61 (CI:3.36–9.36,p < 0.001) for definition 2. Even greater variation was seen in relation to the individual clinician that conducted the triage call; the ORs covering the 95% mid-range of clinicians was 7.80 for potential under-triage definition 1, and 16.00 for potential under-triage definition 2.

Regression modelling results are shown in Table 3.

Table 3 Factors associated with potential under-triage: mixed effects regression modelling results
Full size table

Discussion

This is the first study to report on the safety and accuracy of triage assessments in a two-step system of urgent care telephone triage by comparing the urgency levels assigned to patients along the care pathway with the subsequent care that they received. In total, 21% of patients attended ED and/or were admitted to hospital following two-step triage, and almost a third of these were children aged under 5 years. However, over two-thirds of ED attenders were discharged home, and it is likely that many of them had conditions that were more appropriate to being treated in urgent/primary care. Likewise, almost two-thirds of those admitted for inpatient care were discharged within a day. The specificity and predictive value of secondary triage urgency levels for ED attendance or in-patient admission was much greater than for the equivalent levels assigned in primary triage. Greater sensitivity was seen in primary triage, reflecting risk aversion [5].

There was some indication of potential under-triage in secondary triage. Applying our definitions for ‘potential under-triage’ led to a total of 1.7% (n = 1725/98946) being identified. This included 0.5% (n = 493/98946) patients in relation to their ED acuity status and 1.5% (n = 1468/98946) in relation to in-patient care within 24 h. Of the latter 1468 patients, a large proportion (63%) were discharged within 1 day of admission; this included some who had been admitted as day-cases for elective care.

While there were some clinical conditions (e.g., abdominal pain, fever, cough), age groups (children under of 5 years) and times of day and week when potential under-triage was slightly more likely, the service provider and individual clinician that conducted triage emerged as the key factors associated with potential under-triage despite all services having similar skillmix. There was some indication that clinicians who had shorter duration calls had greater odds of potential under-triage.

Comparison to other literature

Our findings highlight the variation that exists between individual clinicians and services, even when they are all using the same decision support systems. This has also been described in relation to ED triage decision-making [18, 19]. While this may relate to differences in the patient populations served, and the effect this has on diagnostic accuracy [20], differences in staff training and supervision are recognised as being important [21]. Hence, while being an important indicator of clinical need, the acuity level assigned in ED is not an absolute measure of the clinical needs of the patient as it also reflects subjectivity and contextual factors.

Our study highlights poor specificity in primary triage that may contribute to increased demand on emergency care; other studies have similarly highlighted risk aversion in non-clinician triage [6, 7]. Low specificity was also described for a newly introduced non-clinician triage helpline in detecting patients who could be managed by a GP, based on a comparison of the triage advice provided by a non-clinician to a GP assessment for patients who were waiting to be seen in ED or at an urgent care centre [22]. However, due to differing settings, study designs, triage scales and patient populations, any comparison of sensitivity and specificity with other triage systems needs to be undertaken cautiously.

Other studies have compared triage assessment decision making between different types of clinicians. For example, there is some evidence that nurses are less likely to under-triage (i.e. have greater sensitivity) but more likely to over-triage (i.e. have lower specificity) (being less efficient in terms of patients’ healthcare service use) compared to GPs [23]. Likewise, over-triage has been reported to be greater in self-triage as compared to nurse triage [24]. These patterns seen in the shift from GP to nurses, or from nurses to self-triage in the ED appear to be reflected in this study: in English urgent care, non-clinicians are less likely to under-triage but more likely to be risk-averse.

Strengths and limitations

This is the first study to report on patients’ use of ED and in-patient admissions following two-step triage, and the first to compare the sensitivity and specificity of triage outcomes against these outcomes. It is set in England, where healthcare is free at the point of access, and this may affect the transferability of the findings to other health systems.

We only had access to complete data on the urgency level of the secondary triage, and not the referral or appointment booking associated with the clinician’s triage assessment. Hence, while it is likely that all patients triaged as having an emergency will have been referred to the ED directly, for patients triaged to lower levels of urgency most will have been offered in-person consultation appointments with the urgent care service or with their own general practice. The pathway to attending ED or to inpatient admission is likely to have been varied; in many instances, it may have been appropriate for the patient to be first be seen for further assessment and physical examination by the urgent care provider before being advised to attend the hospital. In other instances, the patient’s condition may have worsened in the hours following secondary triage, and the decision to attend ED may have been in keeping with the advice given by the clinician about what to do should this occur. In addition, some patients may have attended ED or received inpatient care for clinical needs that were entirely separate to the problem that they had presented to urgent care.

The extent to which the differences in the sensitivity and specificity of primary and secondary triage outcomes relate to the specific decision support systems being deployed by care providers is important to consider. Other decision support systems may differ in accuracy and safety compared to the systems we investigated. However, a strength was that the dataset allowed comparison between the outcomes for three urgent care providers and for individual clinicians working for these services which allowed recognition of how service and user factors may affect the triage of patients. The variability identified highlights how contextual factors (e.g. differences in workforce, training and supervision, workload) may influence decision-making and the way that digital triage tools are used and hence the safety and accuracy or urgent care triage. The triage outcome urgency level recommended at the end of a secondary triage assessment is dependent on the extent to which clinicians have completed the symptom questions included in the digital triage system; it was noteworthy that in short duration calls (where question prompts may have been left incomplete) there was a higher risk of potential under-triage. It is also possible that the urgency level recorded within the digital triage software does not always reflect the actual advice that the patient was given [25].

In relation to the concept of ‘potential under-triage’ it is important to recognise the uncertainty that exists in many acute presentations; it cannot be assumed that the condition of the patient when they attended ED or were admitted for inpatient care was the same as when they first contacted urgent care or underwent secondary triage. Patients’ conditions worsen or improve in unpredictable ways over time. Additionally, the urgency assessment when the patient arrives at the ED is not standardised, and it may under- or over-estimate the urgency of the patient’s needs and is recognised as varying between individuals and EDs undertaking such assessments [18, 19].

Much of the data collected spanned the Covid-19 pandemic, and so is likely to have included greater numbers of patients presenting with symptoms associated with Covid (e.g. cough, fever, breathlessness). During the Covid-19 pandemic there were high workload pressures on health services [26, 27], and in England, media messaging to reduce demand and “protect” the NHS [28]. This may have impacted on triage decision making, influenced by both patient and professional factors [29, 30]. Finally, our definition of ‘potential under-triage’ was based on a 24-hour time interval following secondary triage; this may have excluded some patients who attended ED or were admitted to inpatient care at a later point.

Implications for research and practice

While the overall performance of the two-step urgent care triage model deployed in England has a high degree of safety and accuracy, our findings provide new evidence that the specificity of non-clinician primary triage is limited. Alone, the primary triage is likely to lead to significant avoidable workload for urgent care providers, EDs and ambulance services owing to its poor specificity and cannot be recommended for practice as a stand-alone service. Secondary clinician triage performs an essential role in re-classifying the urgency of patients’ needs. However, our findings indicate a small risk of potential under-triage in secondary triage, associated with contextual factors such as the place, person and time at which the triage is being undertaken. The reasons for differences between services, times of the week and differences between staff undertaking secondary triage should be further investigated to understand how such differences can best be addressed through training, supervision, service design or development of the triage digital systems. These factors could also be monitored by providers, which may help inform improved operation of services.

The risk of potential under-triage is particularly present with abdominal pain, high temperature, respiratory and other common symptoms. It is also present in young children. There is a need to explore with those producing digital decision support software the scope for enhancing the accuracy of their systems in relation to these presentations, and for services to focus on the communication and IT skills essential to clinically safe triage practice. This includes the importance of clinicians giving clear, symptom-specific safety netting advice so that patients are alert to the signs of deterioration and know when to seek further urgent/emergency care. The effective management of uncertainty in acute presentations is recognised as being a crucial element of clinically safe care [31]. However, there is little empirical evidence about how safety netting advice is being implemented, including how patients experience and follow such advice. There is a need for further research to inform the training and practice of those undertaking triage and how decision support systems can support this.

Conclusions

Our findings demonstrate the accuracy and safety of secondary triage in the context of the two-step model for urgent care triage used in England. We have previously shown that secondary triage downgrades the triage urgency in 74% of calls [5], and here we have found that this is generally achieved without risking patient safety. However, areas of potential under-triage were identified which although infrequent were mainly associated with the service and individual clinician conducting triage rather than the characteristics of the patient. Further research is needed to understand how this should be addressed.

Data availability

Data was disseminated by NHS England for this study; it is not possible to share this individual-level dataset as per the NHS England standard data sharing agreement.

References

  1. Bunn F, Byrne G, Kendall S. Telephone consultation and triage: effects on health care use and patient satisfaction. Cochrane Database Syst Reviews. 2004(3).

  2. Baier N, Geissler A, Bech M, Bernstein D, Cowling TE, Jackson T, et al. Emergency and urgent care systems in Australia, Denmark, England, France, Germany and the Netherlands – analyzing organization, payment and reforms. Health Policy. 2019;123(1):1–10.

    Article  PubMed  Google Scholar 

  3. McKenzie R, Williamson M, Roberts R. Who uses the ‘after hours GP Helpline’? A profile of users of an after-hours primary care helpline. Aus Fam Physician. 2016;45:313–8.

    Google Scholar 

  4. North F, Varkey P, Laing B, Cha SS, Tulledge-Scheitel S. Are e-health web users looking for different symptom information than callers to triage centers? Telemedicine J E Health: Official J Am Telemedicine Association. 2011;17(1):19–24.

    Article  Google Scholar 

  5. Sexton V, Dale J, Bryce C, Barry J, Sellers E, Atherton H. Service use, clinical outcomes and user experience associated with urgent care services that use telephone-based digital triage: a systematic review. BMJ Open. 2022;12(1):e051569.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Turnbull J, Prichard J, Pope C, Brook S, Rowsell A. Risk work in NHS 111: the everyday work of managing risk in telephone assessment using a computer decision support system. Health Risk Soc. 2017;19(3–4):189–208.

    Article  Google Scholar 

  7. Lewis J, Stone T, Simpson R, Jacques R, O’Keeffe C, Croft S, Mason S. Patient compliance with NHS 111 advice: analysis of adult call and ED attendance data 2013–2017. PLoS ONE. 2021;16(5):e0251362–e.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  8. Anderson A, Roland M. Potential for advice from doctors to reduce the number of patients referred to emergency departments by NHS 111 call handlers: observational study. BMJ Open. 2015;5(11):e009444.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Phillips JS. Paramedics’ perceptions and experiences of NHS 111 in the south west of England. J Paramedic Pract. 2020;12(6):227–34.

    Article  Google Scholar 

  10. Turner J, O’Cathain A, Knowles E, Nicholl J. Impact of the urgent care telephone service NHS 111 pilot sites: a controlled before and after study. BMJ Open. 2013;3(11):e003451.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  11. England NHS. NHS 111 Minimum Data Set 2018–19.

  12. NHS-Digital. NHS Pathways [Available from: https://digital.nhs.uk/services/nhs-pathways

  13. Sexton V, Atherton H, Dale J, Abel G. Clinician-led secondary triage in England’s urgent care delivery: a cross-sectional study. Br J Gen Pract. 2023;73(731):e427.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Buerhaus PI, Skinner LE, Auerbach DI, Staiger DO. Four challenges facing the nursing workforce in the United States. J Nurs Regul. 2017;8(2):40–6.

    Article  Google Scholar 

  15. Mullan F, Frehywot S, Jolley LJ, Aging. Primary care, and Self-Sufficiency: Health Care Workforce challenges ahead. J Law Med Ethics. 2008;36(4):703–8.

    Article  PubMed  Google Scholar 

  16. NHS-Digital. Emergency Care Data Set (ECDS) [Available from: https://digital.nhs.uk/data-and-information/data-collections-and-data-sets/data-sets/emergency-care-data-set-ecds

  17. Abel G, Elliott MN. Identifying and quantifying variation between healthcare organisations and geographical regions: using mixed-effects models. BMJ Quality & Safety. 2019;28(12):1032.

  18. Zachariasse JM, Seiger N, Rood PP, Alves CF, Freitas P, Smit FJ, et al. Validity of the Manchester Triage System in emergency care: a prospective observational study. PLoS ONE [Electronic Resource]. 2017;12(2):e0170811.

    Article  PubMed  Google Scholar 

  19. Zachariasse JM, van der Hagen V, Seiger N, Mackway-Jones K, van Veen M, Moll HA. Performance of triage systems in emergency care: a systematic review and meta-analysis. BMJ Open. 2019;9(5):e026471.

    Article  PubMed  PubMed Central  Google Scholar 

  20. Whiting P, Rutjes AW, Reitsma JB, Glas AS, Bossuyt PM, Kleijnen J. Sources of variation and bias in studies of diagnostic accuracy: a systematic review. Ann Intern Med. 2004;140(3):189–202.

    Article  PubMed  Google Scholar 

  21. Tam HL, Chung SF, Lou CK. A review of triage accuracy and future direction. BMC Emerg Med. 2018;18(1):58.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Morreel S, Colliers A, Remmen R, Verhoeven V, Philips H. How accurate is telephone triage in out-of-hours care? An observational trial in real patients. Acta Clin Belg. 2022;77(2):301–6.

    Article  PubMed  Google Scholar 

  23. Graversen DS, Christensen MB, Pedersen AF, Carlsen AH, Bro F, Christensen HC, et al. Safety, efficiency and health-related quality of telephone triage conducted by general practitioners, nurses, or physicians in out-of-hours primary care: a quasi-experimental study using the Assessment of Quality in Telephone Triage (AQTT) to assess audio-recorded telephone calls. BMC Fam Pract. 2020;21(1):84.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  24. Dickson SJ, Dewar C, Richardson A, Hunter A, Searle S, Hodgson LE. Agreement and validity of electronic patient self-triage (eTriage) with nurse triage in two UK emergency departments: a retrospective study. Eur J Emerg Med. 2022;29(1).

  25. Erkelens DC, Rutten FH, Wouters LT, Dolmans LS, de Groot E, Damoiseaux RA, Zwart DL. Accuracy of telephone triage in patients suspected of transient ischaemic attack or stroke: a cross-sectional study. BMC Fam Pract. 2020;21(1):256.

    Article  PubMed  PubMed Central  Google Scholar 

  26. Matenge S, Sturgiss E, Desborough J, Hall Dykgraaf S, Dut G, Kidd M. Ensuring the continuation of routine primary care during the COVID-19 pandemic: a review of the international literature. Fam Pract. 2022;39(4):747–61.

    Article  PubMed  Google Scholar 

  27. Huibers L, Bech BH, Kirk UB, Kallestrup P, Vestergaard CH, Christensen MB. Contacts in general practice during the COVID-19 pandemic: a register-based study. Br J Gen Pract. 2022;72(724):e799–808.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Use NHS. 111 online service to protect busy A&Es, says health chief: The Guardian; [Available from: https://www.theguardian.com/world/2021/nov/01/use-nhs-111-online-service-to-protect-busy-aes-says-health-chief

  29. Lu P, Kong D, Shelley M, Risk, Perception. Preventive behavior, and Medical Care Avoidance among American older adults during the COVID-19 pandemic. J Aging Health. 2021;33(7–8):577–84.

    Article  PubMed  Google Scholar 

  30. Nicholson E, McDonnell T, Conlon C, Barrett M, Cummins F, Hensey C, McAuliffe E. Parental hesitancy and concerns around accessing paediatric unscheduled Healthcare during COVID-19: a cross-sectional survey. Int J Environ Res Public Health. 2020;17(24).

  31. Jones D, Dunn L, Watt I, Macleod U. Safety netting for primary care: evidence from a literature review. Br J Gen Pract. 2019;69(678):e70.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

The authors would like to thank the urgent care providers who participated and NHS England for providing the dataset used in this study.

Funding

This study formed part of a PhD, supervised by HA, GA, and JD, which was funded through the University of Warwick in collaboration with an industrial partner: OneAdvanced. OneAdvanced provided data for the study but played no role in the study design, analysis, interpretation of data, or the writing of the manuscript, which were conducted independently by the research team at the University of Warwick.

Author information

Authors and Affiliations

  1. Warwick Medical School, University of Warwick, Coventry, UK

    Vanashree Sexton, Catherine Grimley & Jeremy Dale

  2. Primary Care Research Centre, University of Southampton, Southampton, UK

    Helen Atherton

  3. Exeter Collaboration for Academic Primary Care, University of Exeter, Exeter, UK

    Gary Abel

Authors
  1. Vanashree Sexton
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Catherine Grimley
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Jeremy Dale
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Helen Atherton
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Gary Abel
    View author publications

    You can also search for this author inPubMed Google Scholar

Contributions

VS designed this study with the support of GA, JD and HA. VS and CG analysed the data, with support from GA. VS wrote the manuscript first draft; GA, JD and HA reviewed and revised the manuscript.

Corresponding author

Correspondence to Vanashree Sexton.

Ethics declarations

Ethical approval and consent to participate

This study was approved by the London - Camden & Kings Cross Research Ethics Committee (REC reference:21/LO/0184) and NHS confidentiality advisory group (CAG) for data linkage involving the use of identifiable data without consent.

Consent for publication

Not applicable.

Competing interests

In addition to his university role, Jeremy Dale is Director of the Clinical Knowledge Unit at Advanced Health and Care. Jeremy Dale played no part in the retrieval or analysis of the data. The other authors have declared no competing interests.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sexton, V., Grimley, C., Dale, J. et al. Safety and accuracy of digitally supported primary and secondary urgent care telephone triage in England: an observational study using routine data. BMC Med Inform Decis Mak 25, 52 (2025). https://doiorg.publicaciones.saludcastillayleon.es/10.1186/s12911-025-02888-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doiorg.publicaciones.saludcastillayleon.es/10.1186/s12911-025-02888-x

Keywords

BMC Medical Informatics and Decision Making

ISSN: 1472-6947

Contact us