Mapping the landscape of machine learning models used for predicting transfusions in surgical procedures: a scoping review

Abstract

Massive transfusion of blood products poses challenges in determining the need for transfusion and the appropriate volume of blood products. This review explores the use of machine learning (ML) models to predict transfusion risk during surgical procedure, focusing on the methodology, variables, and software employed to predict transfusion. This scoping review investigates the development and current state of machine learning models for predicting transfusion risk during surgical procedure, aiming to inform physicians about the field’s progress and potential directions.

The review was conducted using the databases Cochrane, Embase, and PubMed. The search included keywords related to blood transfusion, statistical models, and surgical procedures. Peer-reviewed articles were included, while literature reviews, case reports, and non-human studies were excluded.

A total of 40 studies met the inclusion criteria. The most frequently studied biological variables included haemoglobin, platelet count, international normalized ratio (INR), activated partial thromboplastin time (aPTT), fibrinogen, creatinine, white blood cells, and albumin. Clinical variables of importance included age, sex, surgery type, blood pressure, weight, surgery duration, american society of anesthesiology (ASA) status, blood loss, and body mass index (BMI). The software employed varied, with Python, R, SPSS, and SAS being the most commonly used. Logistic regression was the predominant methodology used in 20 studies.

Our scoping review highlights the need for improved reporting and transparency in methodology, variables, and software used. Future research should focus on providing detailed descriptions and open access to codes of respective models, promoting reproducibility, and enhancing the clinical relevance of transfusion risk prediction models.

Introduction

Significance

Massive transfusion of blood products presents several challenges, such as identifying the need for transfusion as early as possible, and administering the most appropriate volume of blood products [1]. Clinical, paraclinical and biological criteria are used to assess the need for transfusion. In patients with major haemorrhage, haemodynamic, biological and haemostatic status changes rapidly and over a wide range [2], especially during surgery. This is the major reason why recommendations advocate transfusion ratios that allow for rapid action [3,4,5,6].

Background

During surgical procedures, standard haemostasis parameters are often inappropriate in massive transfusion situations due to the time required for blood sample transport and analysis. In addition, the results of these analyses primarily characterise disorders of the endogenous or exogenous coagulation pathways but don’t allow the identification of specific factor deficits for targeted treatment. The advent of off-site monitoring tools such as thromboelastography (TEG^®, Haemonetics Corporation ^®, Boston, USA) or thromboelastometry (ROTEM^®, Werfen ^®, Barcelona, Spain) represents a significant improvement in perioperative bleeding management [7]. However, the longer turnaround times (10–20 min) and extensive training required for analysis and interpretation of results may limit their use in case of massive haemorrhage [8]. Both standard tests and off-site monitoring provide only a snapshot of the patient’s haemostatic status at a given time and in the case of massive haemorrhage, making interpretation of test results potentially obsolete by the time results are available. The development of machine learning (ML) or deep learning may provide a solution for improved guidance in the practice of massive transfusion [9, 10].

The term “machine learning” describes the algorithms used to find patterns in large amounts of data and to learn from these data [11]. However, the implementation of ML also faces several challenges, such as data retrieval volume, data reliability, clinical relevance, and staff training in ML and deep learning techniques [12].

Rationale

Numerous studies have attempted to model clinician intuition in transfusion practice to determine the optimal time to initiate blood transfusion [1, 9]. These studies often use ML to build models, but only a small proportion of them provide sufficient detail of their models to allow easy replication.

Aim and objectives

This scoping review examines the development and current state of machine learning models for predicting transfusion risk during surgery, with the aim of informing clinicians about the progress and potential directions of the fields. To achieve this aim, a comprehensive review of currently published models was undertaken, describing the variables, software and methodology used.

Materials and methods

The protocol of this review is available on request from the corresponding author. The literature review and research protocol were submitted to the figshare^® register and approved under the reference “Skye”. Prisma and Moose checklists were followed [13, 14].

Sources of information

The databases analysed included the cochrane central register of controlled trials (CENTRAL), Cochrane systematic reviews, Embase/Ovid, and PubMed. The time period covered was from inception to 1 April 2022. There were no langage restriction. Articles in languages other than French or English were translated with DeepL Translator^® (DeepL^® SE, Cologne, Germany).

Search strategy

Data extraction was conducted using the following keywords:

“Blood Transfusion” OR blood product transfusion* OR red blood cell transfusion* OR red blood cells transfusion* OR RBC transfusion* OR RBCs transfusion* OR blood cell transfusion* OR blood cells transfusion* OR packed cell transfusion* OR packed cells transfusion* or packed red blood cell* OR packed red cell* OR erythrocyte concentrate*.

OR fresh frozen plasma transfusion* OR plasma transfusion* OR FFP transfusion* OR lyophilized plasma transfusion* OR “Plasma” OR fresh frozen plasma*

OR thrombocyte transfusion* OR platelet transfusion* OR fibrinogen OR “Fibrinogen”

OR massive transfusion*

AND

“Models, Statistical” OR “Artificial Intelligence” OR artificial intelligence OR machine learning

AND

“Cardiac Surgical Procedures” OR cardiac surgery OR heart surgery

OR “Surgical Procedures, Operative” OR “Traumatology” OR trauma surgery

OR surgery

Eligibility criteria

Only peer-reviewed articles were included. All articles matching the specified keywords (transfusion prediction during surgery) were included. Literature reviews, case reports, case series and studies on non-human subjects were excluded. After a thorough full-text review, articles that did not describe the design of a mathematical model but only defined variables for transfusion risk assessment were also excluded.

Selection process

Two reviewers independently screened the retrieved abstracts using the Agency for Healthcare Research and Quality systematic review data repository (SRDR+) software.

The authors screened the titles and abstracts of the search results. Two authors selected manuscripts based on their relevance to the study topic and eliminated articles with mismatched keywords. These two authors were blinded to each other’s results. In the event of disagreement after the blind was lifted, a third, more experienced author made the final selection. During the data collection process, the two authors concerned excluded articles whose full text did not meet the study objectives.

Data collection process

Two additional authors, also blinded to each other, analysed all articles, and collected data using the same software. Due to the specific purpose of the studyand the nature of the studies reviewed, suggested checklists such as CARE 2013 could not be used [15]. Therefore, prior to article analysis, all authors identified a list of 10 items to be completed with data from the articles. After each reviewer had analysed two articles, a meeting was held to determine the relevance of the items. At this stage, three items were added, resulting in a total of 13 items to be completed for each retrieved and included reference.

Data collected

In each article the following information was collected by the two authors.

• Country.

• Conducted in academic hospital

• Multicentre authors

• Study design

• Number of patients

• Inclusion and exclusion criteria

• Variables collected (biological and clinical)

• Software and statistics used

• Model description

• Validation (description and methodology)

• Code description.

Assessment of risk of bias in the review process

To minimize bias, the recommendations of the Joanna Briggs Institute were followed [16]. The three reviewers performed unbiased analyses according to the objectives described in the checklist aboveand were completely blinded to each other.

Software used: description

Currently, there are only a few software tools for ML in healthcare. However, a distinction must be made between a programming language (used by several software programs) and a software program (owned by a single company with proprietary functionality). The most important software and programming languages are described below.

Python® (Python Software Foundation)

Programming language (mainly used by the Anaconda^® software), often used with packages (programming overlays that allow certain functions to be used directly without manual programming; this implies that all parameters have default values that can be modified if necessary).

R® (R Core Team)

Programming language (mainly used with R Studio^® software), all actions must be manually coded.

SPSS® (IBM® Corp, Arming, NY, USA)

Software that allows direct modelling by simply specifying modalities, no coding required.

SAS® (SAS Institute)

Programming language and associated software. Allows modelling to be done directly by simply specifying modalities, with minimal coding required.

Model building: description

The development of a ML model is based on statistical concepts, the most important of which are described in Table 1.

Table 1 Model construction description

Results

Study selection

The search identified 1,329 records, with 154 duplicates found, resulting in 1,175 articles for review. After title and abstract screening, 109 articles met the eligibility criteria and underwent full analysis and data extraction by reviewers. After detailed reading of the full articles, a further 68 manuscripts were excluded because they did not meet the objective of the predefined search, but only described variables to be studied to determine transfusion risk. A flowchart of the search results is shown in Fig. 1.

Fig. 1

Flow of studies through the scoping review

Study characteristics

Forty studies were finally included [17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56] (Table 2). Supplementary Table 1 describes the number of patients included per study, the inclusion periods, and the inclusion and exclusion criteria of each study. The type of surgery at inclusion is described in Table 3. Only 5 studies clearly defined their research outcomes [37, 42, 43, 49, 54] (Supplementary Table 2), 28 performed risk estimation and calculation [17, 18, 20, 22, 26, 27, 29, 30, 32, 34, 36, 38,39,40,41, 43,44,45,46,47,48,49,50,51, 53,54,55,56].

Table 2 General data on studies

Table 3 Classification of type of surgery at patient inclusion

Variables used in included studies

Logistic regression was used as a variable selection method in a total of 20 studies [18, 19, 21, 23,24,25,26,27, 29, 31, 33,34,35, 39,40,41, 43,44,45,46,47,48, 51, 52, 54,55,56] and was not well described in the other studies [17, 20, 22, 28, 30, 32, 36,37,38, 42, 49, 50, 53] (Supplementary Table 3).

The most commonly studied biological variables were haemoglobin, platelet count, INR, aPTT, fibrinogen and, creatinine, white blood cells (WBC) and albumin. Haemoglobin was the only biological parameter used in a total of 10 trials [20, 21, 27, 35, 41,42,43, 47, 52, 56]. Only 4 studies included TEG or ROTEM as monitored parameters [22, 23, 29, 46] (Table 4, Supplementary Table 2).

Table 4 Variables studied, ranked according to the number of studies in which the variable was studied

Regarding demographic data, the most commonly analysed variables were age, sex, weight, body mass index (BMI) and body surface area (BSA), medical history (American Society of Anesthesiology (ASA) physical status, presence of diabetes mellitus or chronic obstructive pulmonary disease). Surgical characteristics were often analysed (duration, type of surgery, blood loss, intraoperative blood pressure and identification of the surgeon). Eleven studies didn’t report patient sex and age [23, 27, 29, 30, 32, 37, 45,46,47,48, 56] (Table 3, Supplementary Tables 1 & 2).

Software used and model created

The software used for ML (Table 1) was not described in 6 studies [17, 18, 24, 30, 37, 53]. Eight studies used R language [21, 23, 25, 27, 32, 42, 44, 54] or SPSS software [21, 23, 26, 27, 39,40,41, 45], seven studies used SAS software [19, 33, 34, 43, 47, 50, 54] or Python language [20, 36, 38, 39, 49, 52, 55], (Supplementary Table 3).

Thirty-one studies didn’t publish the code of their work [21,22,23,24,25, 28, 30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45, 47,48,49,50,51,52,53,54], 8 published it [17, 18, 20, 26, 27, 29, 55, 56] and 1 described a pseudo code [46] (Table 1). The publication of the exact and precise code allows an identical reproduction of the protocol and thus a possible additional external verification by the reader. This is an add value for the manuscript.

More than half (23) of the studies produced a model based on logistic regression [17,18,19, 21, 31,32,33,34,35, 40,41,42,43,44,45, 47, 48, 50,51,52,53, 55, 56], 6 produced a neural network (4,8,14,16,20,21), 5 studies used a nomogram [23, 25,26,27, 44]. Ten studies defined multiples methods for the model used [17, 20, 24, 26, 30, 32, 36, 37, 49, 54] (Fig. 2, Supplementary Table 3).

Fig. 2

Model described and constructed in the different studies. Model type on the x-axis in descending order of the number of studies with this inclusion criterion. Each model is descripted in Table 1

Model validation was mostly internal with data splitting in 16 studies [17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32, 38, 39, 42, 43, 46, 47, 49, 50, 52, 54, 56] and reality checking in 19 others (Supplementary Table 3). There was no validation process in 4 studies [34, 48, 51, 53] and only one study used external validation [55].

Management of missing data

Eleven studies used a data imputation procedure [17, 22, 24, 28, 32, 37, 38, 42, 44, 54] while 8 others excluded patients with missing data [17, 23,24,25, 29, 31, 34, 36, 41, 43, 46, 49, 52, 54, 56] (Supplementary Table 2). The remaining studies simply did not describe how they handled missing data [18,19,20,21, 26, 27, 30, 33, 35, 40, 45, 47, 48, 50, 51, 53, 55].

Discussion

In this scoping review, we analysed the existing literature on modelling the indication for transfusion, highlighting the diversity of methods and tools that have been developed. To the best of our knowledge, this is the first review on this topic that looks at the details of the ML models that have been developed to predict the need for transfusion.

Based on the reviewed articles, the most commonly used biological variables are : haemoglobin, platelets, haematocrit, creatinine, INR, aPTT, albumin, WBC, PT, TEG and fibrinogen. The most commonly used clinical variables are : age, sex, surgery type, blood pressure, weight, duration of surgery, ASA physical status, blood loss, BMI and diabetes.

Most of the trials analysed used the same biological or demographic variables. For example, haemoglobin is an important variable in the assessment of blood product transfusion. It should be remembered that the transfusion of red blood cells, which mainly modifies this parameter, is both an agent (correction of haemostasis disorders) and a measure of haemorrhagic shock (measurement of the amount of red blood cells transfused). The lack of studies analysing TEG or ROTEM point-of-care devices is surprising given their increasing use in clinical contexts of massive haemorrhage. The frequent analysis of white blood cells and albumin is noteworthy. The evolution of white blood cells can be rapid, but this evolution can also be modified by several other causes. As for albumin, its evolution is very slow compared to the changes in patient’s clinical condition. Its inclusion in several studies suggests its potential value in predicting transfusion needs during massive transfusion. The demographic parameters used were also mostly classical (age, sex, weight, height, etc.). Furthermore, this scoping does not answer the question of the influence of the experience of the treating physician on patient morbidity and mortality, which has been reported in several articles [57, 58].

Regarding the software used, the distribution observed in this review reflects current practices in ML, with a clear dominance of the four main tools (Python, R, SAS, and SPSS). Similarly, the models developed are in line with current advances in the field, with logistic regression modelling and the emergence of more innovative techniques (e.g. random forest, neural network, decision tree)being prominent. Logistic regression is easier to implement and understand, but the binary results limit detailed analysis. So far, no machine learning model has shown significant superiority over the others [59]. Similarly, the use of simple logistic or linear regression appears to provide as much information and relevance as much more complex models [60]. The relevance of using complex models should therefore be discussed. The principle of Occam’s razor can be applied to this issue: “Entities are not to be multiplied without necessity.”

If we were to propose a list of factors that would predict the need for massive transfusion, we would make the following list. For clinical variables: Age, sex, type of surgery, weight, duration of surgery, ASA status, blood loss. For biological variables: Haemoglobin, platelets, creatinine, INR, APTT. The model to be used would be logistic regression.

The studies analysed have several limitations. The first limitation is the non-publication of the model which severely limits the external validity of the publications. Publication of the model would increase the transparency and thus the quality of the study by avoiding possible bias and allowing the reader to reproduce the study. Secondly, only five studies clearly described their primary objectives, and most lacked clarity, making it difficult to fully accept the results. In addition, it was often difficult to understand the methodology used to select data for analysis, as data collection, cleaning and missing data management were rarely described. Finally, very few studies published their a priori research protocol in a registry database and none published the exact research methodology, raising concerns about potential data trawling (i.e. including as much data as possible with as many models as possible and then seeing which set is most relevant).

This literature review is innovative in that it examines not only the variables collected but also the ML methodology used to predict transfusion in massive transfusion scenarios. It is important to understand the machine learning processes involved in model design and variable selection methodology. This allows the clinician to critique the results reported by the authors, just as the analysis of a correlation coefficient can be used to moderate the results of a prediction study.

In summary, numerous models are described, some of which apply to the same populations with the same analysed values. The question arises as to the clinical relevance of these models, as most of the articles do not suggest any change in practice that should (or could) be made locally. Only two manuscripts lead to the production of easily accessible online resources.

Further work in the field of haemorrhagic risk prediction in surgery would be to describe the implementation of these models prospectively as an aid to prescribing and then to look at the effects on mortality, morbidity and the amount of transfusion given. A proposed checklist of the various items of information that should be included in the publication of a manuscript describing the design of a machine learning model is given in Table 5. It lists the various items that were evaluated during the course of this review.

Table 5 Proposition of check-list for prediction model publication

Conclusion

This scoping review provided a descriptive overview of ML modelling of transfusion risk conception during surgery. We found that most studies investigated similar biological and clinical variables and used comparable methodologies. Unfortunately, the majority of articles inadequately described their methodology making reproducibility difficult. Future publications should include an appendix detailing the various methodological aspects of the ML methods used, thus promoting transparency and facilitating replication.