Skip to main content

Prediction of adverse pregnancy outcomes using machine learning techniques: evidence from analysis of electronic medical records data in Rwanda

Abstract

Background

Despite substantial progress in maternal and neonatal health, Rwanda’s mortality rates remain high, necessitating innovative approaches to meet health related Sustainable Development Goals (SDGs). By leveraging data collected from Electronic Medical Records, this study explores the application of machine learning models to predict adverse pregnancy outcomes, thereby improving risk assessment and enhancing care delivery.

Methods

This study utilized retrospective cohort data from the electronic medical record (EMR) system of 25 hospitals in Rwanda from 2020 to 2023. The independent variables included socioeconomic status, health status, reproductive health, and pregnancy-related factors. The outcome variable was a binary composite feature that combined adverse pregnancy outcomes in both the mother and the newborn. Extensive data cleaning was performed, with missing values addressed through various strategies, including the exclusion of variables and instances, imputation techniques using K-Nearest Neighbors and Multiple Imputation by Chained Equations. Data imbalance was managed using a synthetic minority oversampling technique. Six machine learning models—Logistic Regression, Decision Trees, Support Vector Machine, Gradient Boosting, Random Forest, and Multilayer Perceptron—were trained using 10-fold cross-validation and evaluated on an unseen dataset with–70 − 30 training and evaluation splits.

Results

Data from 117,069 women across 25 hospitals in Rwanda were analyzed, leading to a final dataset of 32,783 women after removing entries with significant missing values. Among these women, 5,424 (16.5%) experienced adverse pregnancy outcomes. Random Forest and Gradient Boosting Classifiers demonstrated high accuracy and precision. After hyperparameter tuning, the Random Forest model achieved an accuracy of 90.6% and an ROC-AUC score of 0.85, underscoring its effectiveness in predicting adverse outcomes. However, a recall rate of 46.5% suggests challenges in detecting all the adverse cases. Key predictors of adverse outcomes identified in this study included gestational age, number of pregnancies, antenatal care visits, maternal age, vital signs, and delivery methods.

Conclusions

This study recommends enhancing EMR data quality, integrating machine learning into routine practice, and conducting further research to refine predictive models and address evolving pregnancy outcomes. In addition, this study recommends the design of AI-based interventions for high-risk pregnancies.

Clinical trial number

Not applicable.

Peer Review reports

Background

Over the past two decades, Rwanda has achieved substantial progress in reducing maternal and neonatal mortality [1]. Between 2000 and 2020, the maternal mortality ratio declined by over 70%, dropping from 1,071 to 203 deaths per 100,000 live births, while the neonatal mortality rate decreased from 44 to 19 deaths per 1,000 live births [2, 3]. Despite these significant improvements, maternal and neonatal mortality rates in Rwanda remain unacceptably high when compared to developed countries, where the maternal mortality ratio is 13 deaths per 100,000 live births and the neonatal mortality rate is 2.9 deaths per 1,000 live births [4, 5]. Furthermore, Rwanda’s current maternal and neonatal mortality levels fall short of achieving the Sustainable Development Goals (SDGs) as well as national targets outlined in Vision 2050 and the National Health Sector Strategic Plan [6, 7].

These challenges in maternal and neonatal mortality persist despite significant progress in healthcare service coverage and utilization—including 95% of births occurring in health facilities attended by skilled healthcare providers, 98% of women receiving at least one antenatal visit, and widespread health insurance coverage [3, 8]. However, addressing these challenges requires shifting focus beyond service coverage to improving the quality of care, which has been identified as critical to accelerating reductions in maternal and neonatal mortality [9, 10]. Recent analyses of the health sector and multiple studies have identified gaps in the quality of care, revealing that a significant proportion of maternal deaths occur in health facilities, with 72% considered preventable [11, 12]. Improving and strengthening quality of maternal and neonatal health services requires enhancing evidence-based decision-making, developing tailored health interventions, and leveraging advanced technologies—such as machine learning—for the early identification of high-risk pregnancies [13,14,15].

Rwanda has made significant strides in digitalizing its health system, with the introduction of Electronic Medical Records (EMRs) across all public hospitals and health centers [16, 17]. This has enabled the collection of extensive datasets, offering substantial potential to enhance health outcomes through improved data-driven decision-making and the design of tailored interventions. Most importantly, this has enabled the utilization of machine learning, a branch of artificial intelligence, which offers a promising approach to leverage the collected data in developing predictive tools that assist healthcare providers in identifying and managing high-risk pregnancies [18,19,20,21]. By analyzing large datasets containing various factors such as maternal age, medical history, lifestyle, and biomarkers, machine learning has shown great capacity to identify patterns and predict the likelihood of pregnancy complications [22]. Several studies have demonstrated the effectiveness of machine learning models in predicting pregnancy complications such as preeclampsia [23,24,25], gestational diabetes [26], preterm birth [27], labor dystocia [28] and postpartum depression [29].

In light of these challenges and advancements, this study seeks to explore innovative solutions to further reduce maternal and neonatal mortality in Rwanda. In recent years, various technology-driven solutions have been introduced to improve maternal and newborn health services, with a focus on increasing service coverage for pregnant women and developing decision-making algorithms to assist healthcare providers in identifying at-risk women [13, 15]. One study that applied machine learning to predict adverse pregnancy outcomes primarily used non-clinical data obtained from demographic health surveys [30]. This study, in contrast, aims to utilize the power of machine learning in combination with EMR data to develop predictive tools that will enhance the early identification and management of high-risk pregnancies, ultimately improving the quality of care. This approach not only aligns with Rwanda’s ongoing digital health transformation but also provides a data-driven framework to address critical gaps in healthcare quality.

Methods

Data source and study design

This study utilized retrospective data extracted from the electronic medical record system of 25 public district hospitals in Rwanda. Hospitals were selected based on their use of “OpenMRS,” an open-source EMR system customized by the Ministry of Health [31]. The time frame chosen spans from 2020, when the system was first integrated into clinical operations, up to 2023, the year preceding data extraction. This study included women who delivered public district hospitals, which represent a secondary level of care in Rwanda’s healthcare system. The Rwandan health service tier is organized into three levels: primary healthcare (health posts and health centers), secondary healthcare (district hospitals), and tertiary healthcare (referral and teaching hospitals [7]. District hospitals, which handle a significant portion of deliveries, particularly those transferred from health centers, account for 35% of births in Rwanda, with the majority occurring at health centers (58%) and the remaining at tertiary facilities [32].

Variable description

The independent variables encompass a wide spectrum of socioeconomic, health status, reproductive health, and pregnancy-related factors. The socioeconomic variables included geographical location, age, marital status, occupation, and access to health insurance. Health status factors included Body Mass Index (BMI), history of chronic diseases, surgical history, and infectious disease status, including Hepatitis B, Hepatitis C, and HIV. Reproductive health variables included the number of pregnancies, live births, abortions, preterm deliveries, and gestational age, which provided critical information on the participants’ reproductive history and current pregnancy status. Variables related to current pregnancy and delivery were also included. These included vital signs and laboratory results at admission, fetal heart rate at admission, triage classification, presence of danger signs, and method of delivery.

The outcome variable in this study was defined as a binary indicator of adverse pregnancy outcomes, denoted by 1 for cases with adverse outcomes and 0 for those without adverse outcomes. This composite variable integrated both neonatal and maternal health outcomes. Neonatal adverse outcomes were classified based on any of the following criteria: delivery of a newborn with an APGAR score below 7 at 5 min, delivery of a stillbirth, or delivery of a neonate with a birth weight less than 1.8 kg or greater than 4.5 kg. Meeting any of these criteria resulted in the neonatal outcome being classified as adverse. Similarly, adverse maternal outcomes are determined by several indicators such as interventions performed after delivery (such as laparotomy or re-look surgeries), postpartum observations for events such as postpartum hemorrhage (PPH), the requirement for blood transfusions, and critical maternal outcomes, including maternal mortality or transfer to a higher-level facility for advanced care.

Data cleaning and pre-processing

Data cleaning was conducted using Python Version 3 [33]. Initial data cleaning consisted of restructuring data from a wide to a long format and the creation of the patient’s unique identifiers. Outlier detection and handling were systematically performed for all numerical variables to ensure data accuracy and integrity. Duplicate entries were identified using patient identifiers, and any duplicates found were addressed by removing one of the entries. Variables exhibiting inconsistent reporting across and/or within hospitals were also excluded.

Missing data was addressed using a systematic approach involving both imputation and exclusion techniques. Variables with a high proportion of missing values were closely examined to determine whether the missingness was random or systematic. Independent variables with more than 20% missing data were excluded from the analysis, as were women with missing values in the outcome variable. Key variables for predicting adverse pregnancy outcomes were selected based on McCarthy and Maine’s theoretical framework, which categorizes causes into distant determinants (such as age, marital status, and occupation), intermediate determinants (such as health status, access to healthcare, and healthcare-seeking behavior), and pregnancy-related factors (such as mode of delivery, gestational age, vital signs and laboratories) [34]. A conceptual framework based on this model was developed, taking into consideration the available data in the EMR dataset. For the remaining missing data, we applied Multiple Imputation by Chained Equations (MICE) for numerical variables and K-Nearest Neighbors (KNN) for categorical variables, based on their effectiveness in handling missing data in healthcare datasets [29, 35, 36]. Additional processing included reducing categories for nominal categorical variables, such as classifying blood groups into Rhesus negative and positive, and calculating known medical parameters, such as Body Mass Index (BMI), which combines weight and height.

Data transformation for machine learning

Categorical variables, which contained non-numeric data, were converted into a numerical format through feature encoding. The numerical variables were standardized using min–max scaling to bring all features into a uniform range, which is crucial for algorithms sensitive to the scale of the input data.

Addressing class imbalance

To tackle the issue of imbalance in the dataset, where adverse pregnancy outcomes were significantly underrepresented, we applied the synthetic minority oversampling technique (SMOTE), which was selected because it was performed in similar previous studies [36]. SMOTE generates synthetic samples for the minority class by interpolating between existing minority instances and their nearest neighbors, effectively balancing the dataset and improving the model’s ability to learn from both classes. However, SMOTE can lead to overfitting, introduce noise, and generate synthetic samples that may not accurately represent the underlying data distribution, potentially affecting model generalization [37]. This oversampling was performed before the model training phase, ensuring that the machine learning algorithms could generalize better and make accurate predictions for adverse outcomes.

Machine learning

Model training

The data analysis process commenced by splitting the processed dataset into training and testing sets at a 70 − 30 ratio. Based on the existing literature and their proven efficacy in healthcare data applications, six machine-learning algorithms were selected for initial training. These algorithms include logistic regression, decision trees, random forests, gradient boosting machines (GBM), support vector machines (SVM), and neural networks. Each model underwent training using 10-fold cross-validation. The top three algorithms, based on the evaluation metrics from k-fold cross-validation, were selected for hyperparameter tuning. A grid search was employed to optimize the hyperparameters and systematically explore predefined values to determine the best-performing combination.

Model evaluation

The trained models were evaluated using the testing dataset by employing a suite of key performance metrics to assess their predictive capabilities comprehensively. The primary metrics utilized were accuracy, precision, recall, F1-score, and area under the receiver operating characteristic curve (AUC-ROC).

Ethics and privacy consideration

Prior to conducting this research, ethics clearance was granted by the Institutional Review Board of the University of Rwanda with reference number CMHS/IRB/338/2024. Prior to extracting data from the database for storage and analysis, rigorous data anonymization measures were implemented. All personal identifiable information (PII) such as patient names, national identification numbers, and addresses were removed.

Results

Descriptive statistics

This study used data from 117,069 women who delivered and whose records were captured in the database of 25 hospitals across Rwanda. After excluding entries with a substantial number of missing values, the final dataset comprised of 32,783 women which constitute 28% of all women in the database. Of these, 16.5% (5,424 out of 32,783) experienced adverse pregnancy outcomes. Table 1 provides an overview of the descriptive statistics of 32,783 women included in this study.

Social and demographic characteristics

Participants were distributed across Rwanda’s five provinces, with the majority from the Western Province (34%), followed by the Eastern Province (28%), Northern Province (20%), and Southern Province (18%). A small proportion (0.2%) resided in Kigali City. Adverse pregnancy outcomes were more prevalent in the Southern Province (30%) and Eastern Province (30%) compared to other provinces. Only 7.4% of participants resided in urban districts, with a slightly lower proportion of adverse outcomes (4.9%) among urban residents. The median age was 29 years (interquartile range: 24–34 years) across both groups.

Regarding marital status, most participants (90%) were married, while 8.4% were separated, divorced, or widowed, and 1.2% were single. These proportions were similar across groups with and without adverse outcomes. In terms of occupation, the majority (84%) were farmers, followed by employed professionals (7.3%) and those not working (6.9%). Occupational patterns were consistent across groups. Nearly all participants (97%) reported having health insurance coverage, with a slightly higher proportion (98%) among those who experienced adverse pregnancy outcomes compared to those without adverse outcomes (97%).

Health status and medical history

The median body mass index (BMI) was 24.4 kg/m² (IQR: 22.9–26.6) and was comparable between groups with and without adverse pregnancy outcomes. A small proportion (3.9%) of participants were Rhesus negative, with a slightly higher percentage (4.1%) observed among those with adverse outcomes. Prevalence of infections such as hepatitis B (0.7%) and hepatitis C (0.7%) was low, showing minimal differences between groups. HIV prevalence was 1.8%, but it was notably higher (2.6%) among participants with adverse outcomes compared to those without (1.6%).

Regarding obstetric and surgical history, 26% of participants had a previous uterine scar, with a higher proportion (29%) among those with adverse outcomes. Similarly, a history of previous surgeries was reported by 6.4% of participants, rising to 11% among those with adverse outcomes. Chronic diseases were reported by 2.8% of participants, with a slightly higher prevalence (3.3%) among those with adverse pregnancy outcomes.

The median number of tetanus vaccine doses received was 3 (IQR: 2–3) across both groups. The median number of antenatal care (ANC) visits was 3 (IQR: 3–4), though participants without adverse outcomes had a slightly higher median of 4 visits compared to 3 visits among those with adverse outcomes. Regarding obstetric history, the median number of pregnancies was 2 (IQR: 1–4) and the median number of live births was 1 (IQR: 0–2) in both groups. Abortion history was rare, with a median of 0 (IQR: 0–0) across all participants. Similarly, premature deliveries and stillbirths were uncommon, each having a median of 0 (IQR: 0–0) in both groups.

Status at admission and delivery

The median gestational age at admission was 39 weeks (IQR: 38–40) across all participants. Those with adverse pregnancy outcomes had a slightly wider range (37–40 weeks) compared to those without (38–40 weeks). Most participants were carrying singleton pregnancies (97%), with a slightly higher proportion of twin pregnancies (3.0%) among those with adverse outcomes compared to 2.2% in the non-adverse group.

Among participants, 13% were admitted in labor, with a significantly higher proportion in the adverse outcomes group (18%) compared to the non-adverse group (12%). The median fetal heart rate (FHR) at admission was consistent across groups, at 140 beats per minute (IQR: 134–146). Triage classification revealed that 31% of participants were categorized as red, indicating severe cases requiring immediate attention, and this proportion was consistent across groups. Additionally, 29% were classified as orange (moderate risk), 17% as yellow (low risk), and 23% as green (stable condition). Participants with adverse outcomes had a higher proportion classified as orange (32%) compared to those without adverse outcomes (29%). Furthermore, 18% of participants with adverse outcomes were identified as emergencies at admission, compared to 7.8% in the non-adverse group. Danger signs were observed in 12% of participants overall, with a slightly higher prevalence in the adverse outcomes group (13%) compared to 12% in the non-adverse group.

Laboratory and vital signs at admission

Participants exhibited generally stable vital signs upon admission. The median heart rate was 84 beats per minute (IQR: 77–92), with slightly higher values observed among those with adverse outcomes (85 bpm, IQR: 78–94) compared to those without adverse outcomes (83 bpm, IQR: 77–91). Median systolic and diastolic blood pressures were 118 mmHg (IQR: 111–125) and 72 mmHg (IQR: 66–78), respectively, with marginally higher values in the adverse outcomes group (119/73 mmHg) compared to the non-adverse group (118/72 mmHg). Body temperature was consistent across both groups, with a median of 36.5 °C (IQR: 36.4–36.7). Respiratory rates were slightly lower in the adverse outcomes group (18 breaths per minute, IQR: 18–20) compared to the non-adverse group (19 breaths per minute, IQR: 18–20). Blood oxygen saturation levels remained stable across groups, with a median of 98% (IQR: 98–99).

Laboratory findings showed a median hemoglobin level of 13.0 g/dL (IQR: 12.1–13.9) overall, with slightly lower values in the adverse outcomes group (12.9 g/dL, IQR: 11.9–13.9) compared to the non-adverse group (13.0 g/dL, IQR: 12.1–13.9). Blood platelet counts had a median of 208 × 10³/µL (IQR: 169–250), with no substantial differences between groups. The median white blood cell count was 8.5 × 10³/µL (IQR: 6.8–10.8), slightly higher in participants with adverse outcomes (8.7, IQR: 6.9–11.1) than those without (8.5, IQR: 6.8–10.8).

Mode of delivery and healthcare provider

The majority of deliveries (68%) were attended by medical doctors, with a slightly lower proportion (67%) among participants with adverse pregnancy outcomes compared to those without (69%). Midwives assisted in 30% of deliveries overall, with similar proportions in both groups (31% in the adverse outcomes group and 30% in the non-adverse group). A small proportion (1.3%) of deliveries were attended by other healthcare providers, which was slightly higher (1.9%) in the adverse outcomes group compared to 1.2% in the non-adverse group. Spontaneous vaginal delivery was the most common method, accounting for 54% of all births. However, participants with adverse pregnancy outcomes were more likely to undergo cesarean Sect. (59%) compared to 43% in the non-adverse group. Assisted vaginal deliveries were uncommon, representing less than 1% of all births, but were slightly higher in the adverse outcomes group (1.3%) than in the non-adverse group (0.7%).

Table 1 Descriptive statistics

Initial machine learning training

The results of the machine learning training revealed varying performances across different models, as illustrated in Table 2. The Random Forest model achieved the highest accuracy of 90.38% and highest AUC-ROC score (0.8384). The precision of this model was notably high (86.79%), although its recall was relatively low (47.74%). In comparison, the Gradient Boosting model also performed well, with an accuracy of 88.49% and AUC-ROC score of 0.8221. This model exhibited a balance between precision and recall, with a precision of 72.12% and a recall of 46.89%.

Table 2 Performance of machine learning models

Other models, such as logistic regression and support vector machines (SVM), displayed moderate performance. Logistic Regression had an accuracy of 74.18%, with an AUC-ROC score of 0.7755. It had a relatively lower precision (34.61%) and recall (67.17%). Similarly, the SVM achieved an accuracy of 81.55% and an AUC-ROC of 0.7999, with a precision of 44.69% and recall of 59.53%. The Decision Tree model also demonstrated a strong performance, with an accuracy of 82.9% and an AUC-ROC score of 0.7202. Its precision was 47.55% and recall was 55.94%, indicating a reasonable balance, but still lower than that of the Random Forest and Gradient Boosting models. The multilayer perceptron (MLP) classifier had an accuracy of 81.27% and an AUC-ROC score of 0.7832, with a precision of 43.94% and recall of 57.45%. Although it offered a good balance, it did not surpass the Random Forest and Gradient Boosting models in terms of overall performance.

Hyperparameter tuning

Three models; Random Forest, Gradient Boosting Classifier, and Multilayer Perceptron—were selected for hyperparameter tuning. Based on their overall performance across key evaluation metrics, particularly AUC-ROC and accuracy, as shown in Table 2, Random Forest and Gradient Boosting Classifiers were the top two algorithms, followed by SVM and MLP. Although SVM showed reasonable performance, recent literature has demonstrated the high performance of neural network-based models, which is why the MLP classifier was prioritized for hyperparameter tuning over SVM. This decision aimed to leverage the strengths of neural networks for predictive accuracy in healthcare applications. Figure 1 presents the Receiver Operating Characteristic (ROC) curves and the area under the curve (AUC) for the three best-performing models after hyperparameter tuning.

Following hyperparameter tuning, the Random Forest model was refined with the following optimal parameters: bootstrap = False, max_depth = None, min_samples_leaf = 1, min_samples_split = 2, and n_estimators = 300. This final tuned model achieved a test accuracy of 90.6% and an ROC-AUC score of 0.85. The precision and recall were 90.8% and 46.5%, respectively. Figure 2 shows the confusion matrix of Random Forest model after hyperparameter tuning.

Fig. 1
figure 1

Receiver operating characteristic - area under the Curve (ROC-AUC) after hyperparameter tuning

Fig. 2
figure 2

Confusion matrix of random forest model

Feature importance

Using the optimized random forest model, features were ranked according to their importance in predicting adverse pregnancy outcomes. Figure 3 presents the ranking of the various variables based on their importance in predicting adverse pregnancy outcomes. The most influential feature was gestational age, with an importance score of 0.107119. The number of pregnancies followed with an importance score of 0.084591, and the number of antenatal visits also showed a high importance score of 0.069602.

Vital sign parameters, especially respiratory rate and oxygen saturation at admission, had importance scores of 0.064599 and 0.045226, respectively, followed by tetanus vaccination, with an importance score of 0.063007. The number of children born alive had an important score of 0.040902, while the geographical location (Southern Province) was also a notable factor with a score of 0.039592. Fetal heart rate at admission had an importance score of 0.037457, and other vital signs parameters: temperature and heart rate at admission, followed by importance scores of 0.037077 and 0.029227, respectively. This was followed by laboratory parameters such as white blood cell and platelet counts at admission, with feature importances of 0.026270 and 0.024837, respectively. The mother’s age and body mass index (BMI) were also relevant, with scores of 0.025958 and 0.025903, respectively. The diastolic blood pressure at admission had an importance score of 0.024744, and the hemoglobin level at admission was 0.024381. Systolic blood pressure at admission closely followed the importance score of 0.024191. The delivery methods, specifically spontaneous vaginal delivery and cesarean section had scores of 0.021371 and 0.019242, respectively. Northern Province had an important score of 0.015233, and the number of abortions was 0.015165. Healthcare providers at birth (midwives) and emergency cases had importance scores of 0.013549 and 0.012501, respectively.

Other features such as the Western Province (0.011105), number of stillbirths (0.008580), previous uterine scar (0.007114), and previous surgical history (0.006580) were also noted. Triage classification (orange, red, yellow), presence of danger signs, and occupation (farmer) were included with varying importance scores, along with marital status, and rhesus negative status. Features such as the number of fetuses, preterm deliveries, history of chronic diseases, access to insurance, HIV status, healthcare provider at birth (others), and various provinces and religious affiliations had lower importance scores but were still considered in the model.

Fig. 3
figure 3

Ranking of importance of features in prediction of adverse outcome

Discussion

The primary objective of this study was to leverage machine learning techniques to predict adverse pregnancy outcomes using an extensive dataset derived from the electronic medical record (EMR) system in Rwanda. Despite the implementation of electronic medical record (EMR) systems in Rwanda since 2006 [17], research utilizing these data has been limited due to their complex structure, restricted access, and quality challenges [38,39,40]. This study is notable for being one of the first in Rwanda to utilize nationwide EMR data from 25 hospitals and the first to apply machine-learning techniques to these datasets. This represents a significant advancement in leveraging EMR data for predictive modeling in healthcare.

Data quality, particularly completeness, was a significant consideration in this research, aligning with previous findings that identified data quality as a major obstacle in utilizing EMR data for research purposes [38]. Of the 117,069 women who delivered, only 32,783 were included in the analysis because of extensive missing data. This represents just 28% completeness, a notably lower figure than the 85% completeness observed in a prior study in Rwanda that assessed EMR data for HIV-related care [41]. This discrepancy may be attributed to the fact that EMR systems were initially implemented for HIV care, benefiting from nearly two decades of development and refinement, whereas the broader application of EMR data in other areas, such as hospitalization and maternity, is still in its early stages [42, 43].

The results revealed that the optimized Random Forest model achieved the highest performance with an accuracy of 90.6% and an ROC-AUC score of 0.85. A model precision of 90.8% underscores its effectiveness in identifying adverse outcomes, although its recall of 46.5% highlights the challenge of detecting all adverse cases. In addition to random Forest, Gradient Boosting Classifier also achieved a higher accuracy and ROC-AUC of 88.49% and 0.822, respectively. These results are consistent with previous studies that consistently found Random Forest and Gradient Boosting Models to be the best performing algorithms in the prediction of adverse pregnancy outcomes [36, 44, 45].

Although our machine learning model achieved a high accuracy of 90.6%, AUC of 0.85, and precision of 90.8%, it performed poorly on the recall metric, yielding a result of 46.5%. This highlights the challenges in handling such an unbalanced dataset, where only 16.5% of the women experienced adverse pregnancy outcomes. Although we employed the synthetic minority oversampling technique (SMOTE) to mitigate this imbalance, the distinct characteristics of the minority abnormal samples remain difficult to capture. As a result, many abnormal samples were misclassified as normal, leading to a low recall. This observation is also consistent with previous literature, especially a study by Yuwei Hang, who used EMR data to predict adverse pregnancy outcome [36].

In addition, the high performance of the random forest and gradient boosting model is also evident in the prediction of individual maternal and neonatal conditions, such as pre-eclampsia [46, 47], method of delivery [48, 49], low birth weight, prematurity [50], and other conditions.

This study highlights the importance of gestational age in predicting adverse pregnancy outcomes. This is self-explanatory, as delivery beyond term gestational age, such as prematurity, remains among the major causes of neonatal morbidity and mortality in Rwanda [51]. Moreover, post-term delivery is associated with adverse neonatal outcomes. Additionally, the number of pregnancies has emerged as an important factor for predicting pregnancy outcomes. This is consistent with previous ML studies that identified the number of pregnancies and parity as important predictors of pregnancy outcomes [36, 44]. This is also consistent with previous epidemiological studies that have established a relationship between the number of pregnancies, deliveries, and pregnancy outcomes [52]. The number of antenatal visits and services provided during ANC such as Tetanus Vaccination, has also emerged as an important proxy indicator of health service utilization. This is a well-established relationship that links antenatal care underutilization with the development of adverse pregnancy outcomes [53]. Maternal age was also identified as a contributing factor to the maternal and neonatal outcomes. This finding is in line with previous studies that have established a strong relationship between maternal age and outcomes.

In addition, this study highlights the importance of vital sign parameters at admission in predicting maternal and neonatal pregnancy outcomes. In this study, vital sign parameters such as heart rate, oxygen saturation, temperature, blood pressure, and respiratory rate were important for predicting pregnancy outcomes. Similarly, laboratory results, such as hemoglobin, platelet counts, and white blood cell counts at admission, were also important in the prediction. This is a crucial impact of vital signs, and the laboratory results are consistent and have been documented in several studies [54, 55]. Not surprisingly, the status of the fetus, which was approximately measured using the fetal heart rate, was of great importance in predicting pregnancy outcomes, especially adverse neonatal outcomes. The predictive value of geographical location, specifically the province, may be indicative of underlying socioeconomic factors, including disparities in literacy levels, wealth, access to healthcare, and care-seeking behaviors, all of which are frequently associated with regional disparities in health outcomes. This observation is similar to those observed in previous study that analyzed in Spatio-temporal disparities in maternal health service utilization in Rwanda [56].

The method of delivery, especially cesarean section, is highly important in the prediction of pregnancy outcomes. This observation is well-documented in previous studies that showed an increased risk of maternal and neonatal mortality and morbidity following cesarean delivery [57]. Adverse outcomes among women who delivered via cesarean section were also observed in Rwanda.

Limitation of the study

This study leverages a significant strength in utilizing a large dataset derived from EMRs across 25 public district hospitals in Rwanda. This comprehensive dataset provides a valuable resource for investigating factors associated with adverse pregnancy outcomes. However, limitations and threats to external validity of this study should be acknowledged.

Firstly, the study population may not be fully representative of all pregnant women in Rwanda. This study included women who delivered at public district hospitals, which represent approximately 35% of all births in Rwanda [32]. Women who delivered at public health centers (primary level of care) and those who delivered at tertiary public facilities or private facilities were excluded due to the use of different EMR systems. This exclusion of a substantial portion of the population may introduce selection bias and limit the generalizability of the findings to all pregnant women in Rwanda. Secondly, data quality issues, such as missing data and potential inconsistencies within and between hospitals, could have influenced the model’s performance and limited its accuracy. Thirdly, the external validity of the model may be limited. The model’s performance may not be generalizable to other healthcare settings with different patient populations, resources, and care practices. Furthermore, the temporal validity of the model may be limited as healthcare practices, the prevalence of certain risk factors, and the characteristics of the population may evolve over time.

Conclusion

This study represents a pioneering effort to leverage machine learning techniques to predict adverse pregnancy outcomes using nationwide electronic medical record (EMR) data from Rwanda. Despite the challenges associated with data quality and completeness, the study demonstrated the potential of machine learning models, particularly Random Forest and Gradient Boosting Classifiers, to achieve high accuracy and precision in predictive modeling. The optimized Random Forest model achieved an accuracy of 90.6%, a ROC-AUC score of 0.85, and a precision of 90.8%, demonstrating its effectiveness in identifying adverse outcomes. However, its recall of 46.5% highlights a notable limitation in detecting all adverse cases, particularly within a highly imbalanced dataset. While this may reduce its practicality in scenarios where high sensitivity is crucial—such as healthcare settings that require early detection and intervention—the model’s high precision makes it more suitable for applications where minimizing false positives and avoiding unnecessary interventions are prioritized. Nonetheless, the low recall underscores the need for further model refinement or complementary approaches to improve sensitivity and ensure broader applicability in clinical practice.

Key factors, such as gestational age, number of pregnancies, antenatal visits, maternal age, vital sign parameters, and method of delivery, were identified as significant predictors of adverse pregnancy outcomes. These findings align with existing literature and underscore the importance of comprehensive data collection and monitoring during pregnancy and delivery.

Implications for practice and policy recommendations

The findings of this study demonstrate the potential for integrating machine learning models into Rwanda’s maternal and neonatal health monitoring systems to enhance early risk identification and intervention planning. Predictive tools, such as those developed in this study, could assist healthcare providers in triaging high-risk pregnancies, enabling timely interventions and more efficient resource allocation. However, the observed limitations in recall emphasize the need to complement high-specificity models with approaches that improve sensitivity, ensuring that critical cases are not overlooked in clinical applications where early detection is vital.

Policymakers should prioritize investments in strengthening EMR systems by improving data quality, completeness, and interoperability to ensure reliable inputs for predictive modeling. Efforts should also focus on building healthcare provider capacity to interpret and apply model predictions effectively. Expanding EMR systems to capture more granular data, particularly for rare but severe outcomes, will further enhance model performance and scalability. Promoting the adoption of machine learning-based decision-support tools in maternal and child health programs can help reduce adverse outcomes and advance data-driven healthcare practices across Rwanda.

Data availability

The datasets generated and/or analyzed during the current study are not publicly available due to privacy law and policies posing restrictions to data related to health records. However, data can be available from the corresponding author on reasonable request and with approval from Rwanda National Ethics Committee.

References

  1. Abbott P, Sapsford R, Binagwaho A. Learning from Success: how Rwanda achieved the Millennium Development Goals for Health. World Dev. 2017;92:103–16.

    Article  Google Scholar 

  2. National Institute of Statistics of Rwanda II. Rwanda Demographic and Health Survey 2020. 2020.

  3. National Institute of Statistics of Rwanda (NISR). [Rwanda], Ministry of Health (MOH) [Rwanda], ICF. Rwanda Demographic and Health Survey 2019-20 final report. Kigali, Rwanda, and Rockville. Maryland, USA: NISR and ICF; 2021.

    Google Scholar 

  4. Sharrow D, Hug L, You D, Alkema L, Black R, Cousens S, et al. Global, regional, and national trends in under-5 mortality between 1990 and 2019 with scenario-based projections until 2030: a systematic analysis by the UN Inter-agency Group for Child Mortality Estimation. Lancet Glob Health. 2022;10:e195–206.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  5. Cresswell J. Trends in maternal mortality 2000 to 2020: estimates by WHO, UNICEF, UNFPA, World Bank Group and UNDESA/Population Division. 1st ed. Geneva: World Health Organization; 2023.

    Google Scholar 

  6. Government of Rwanda. Vision 2050. 2015.

  7. Ministry of Health. Rwanda. Fourth Health Sector Strategic Plan. 2018.

  8. Saksena P, Antunes AF, Xu K, Musango L, Carrin G. Mutual health insurance in Rwanda: evidence on access to care and financial risk protection. Health Policy. 2011;99:203–9.

    Article  PubMed  Google Scholar 

  9. Sayinzoga F, Bijlmakers L, Van Der Velden K, Van Dillen J. Severe maternal outcomes and quality of care at district hospitals in Rwanda– a multicentre prospective case-control study. BMC Pregnancy Childbirth. 2017;17:394.

    Article  PubMed  PubMed Central  Google Scholar 

  10. Manzi A, Nyirazinyoye L, Ntaganira J, Magge H, Bigirimana E, Mukanzabikeshimana L, et al. Beyond coverage: improving the quality of antenatal care delivery through integrated mentorship and quality improvement at health centers in rural Rwanda. BMC Health Serv Res. 2018;18:136.

    Article  PubMed  PubMed Central  Google Scholar 

  11. Sayinzoga F, Bijlmakers L, Van Dillen J, Mivumbi V, Ngabo F, Van Der Velden K. Maternal death audit in Rwanda 2009–2013: a nationwide facility-based retrospective cohort study. BMJ Open. 2016;6:e009734.

    Article  PubMed  PubMed Central  Google Scholar 

  12. Ministry of Health, Rwanda. Health Sector Situation Analysis, 2019–24. 2024.

  13. Ngabo F, Nguimfack J, Nwaigwe F, Mugeni C, Muhoza D, Wilson DR et al. Designing and implementing an innovative SMS-based alert system (RapidSMS-MCH) to monitor pregnancy and reduce maternal and child deaths in Rwanda. Pan Afr Med J. 2012;13.

  14. Ngizimana B, Nizeyimana P. Clustering and Analysis of Maternal Mortality Causes in Rwanda: implications for targeted Health interventions. In: Proceedings of the 2024 IEEE SmartBlock4Africa. Accra, Ghana: IEEE; 2024. p. 1–10.

    Google Scholar 

  15. Nishimwe A, Ibisomi L, Nyssen M, Conco DN. The effect of a decision-support mHealth application on maternal and neonatal outcomes in two district hospitals in Rwanda: pre– post intervention study. BMC Pregnancy Childbirth. 2022;22:52.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Ministry of Health, Rwanda. The National Digital Health Strategic Plan 2018–2023. 2018.

  17. Seymour RP, Tang A, DeRiggi J, Munyaburanga C, Cuckovitch R, Nyirishema P, et al. Training software developers for electronic medical records in Rwanda. In: Safran C, Reti S, Marin HF, editors. Proceedings of the 13th World Congress on Medical Informatics (MEDINFO 2010). Studies in Health Technology and Informatics, Vol. 160. Amsterdam: IOS Press; 2010. p. 585–9. https://doiorg.publicaciones.saludcastillayleon.es/10.3233/978-1-60750-588-4-585.

  18. Khadidos AO, Saleem F, Selvarajan S, Ullah Z, Khadidos AO. Author correction: Ensemble machine learning framework for predicting maternal health risk during pregnancy. Sci Rep. 2024;14:22679.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  19. Bertini A, Salas R, Chabert S, Sobrevia L, Pardo F. Using machine learning to predict complications in pregnancy: a systematic review. Front Bioeng Biotechnol. 2022;9:780389.

    Article  PubMed  PubMed Central  Google Scholar 

  20. Kopanitsa G, Metsker O, Kovalchuk S. Machine learning methods for pregnancy and childbirth risk management. J Pers Med. 2023;13:975.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Ravikumar S, Kannan E. Machine learning techniques for identifying fetal risk during pregnancy. Int J Image Graph. 2022;22:2250045.

    Article  Google Scholar 

  22. Islam MN, Mustafina SN, Mahmud T, Khan NI. Machine learning to predict pregnancy outcomes: a systematic review, synthesizing framework and future research agenda. BMC Pregnancy Childbirth. 2022;22:348.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Aljameel SS, Alzahrani M, Almusharraf R, Altukhais M, Alshaia S, Sahlouli H, et al. Prediction of Preeclampsia using Machine Learning and Deep Learning models: a review. Big Data Cogn Comput. 2023;7:32.

    Article  Google Scholar 

  24. Sufriyana H, Wu Y-W, Su EC-Y. Artificial intelligence-assisted prediction of preeclampsia: development and external validation of a nationwide health insurance dataset of the BPJS Kesehatan in Indonesia. EBioMedicine. 2020;54:102710.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Lee SM, Nam Y, Choi ES, Jung YM, Sriram V, Leiby JS, et al. Development of early prediction model for pregnancy-associated hypertension with graph-based semi-supervised learning. Sci Rep. 2022;12:15793.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  26. Kurt B, Gürlek B, Keskin S, Özdemir S, Karadeniz Ö, Kırkbir İB, et al. Prediction of gestational diabetes using deep learning and bayesian optimization and traditional machine learning techniques. Med Biol Eng Comput. 2023;61:1649–60.

    Article  PubMed  Google Scholar 

  27. Raja R, Mukherjee I, Sarkar BK. A machine learning-based prediction model for Preterm Birth in Rural India. J Healthc Eng. 2021;2021:1–11.

    Article  Google Scholar 

  28. Shazly SA, Borah BJ, Ngufor CG, Torbenson VE, Theiler RN, Famuyide AO. Impact of labor characteristics on maternal and neonatal outcomes of labor: a machine-learning model. PLoS ONE. 2022;17:e0273178.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  29. Amit G, Girshovitz I, Marcus K, Zhang Y, Pathak J, Bar V, et al. Estimation of postpartum depression risk from electronic health records using machine learning. BMC Pregnancy Childbirth. 2021;21:630.

    Article  PubMed  PubMed Central  Google Scholar 

  30. Kubahoniyesu T, Kabano IH. Predicting adverse pregnancy outcome in Rwanda using machine learning techniques. PLoS ONE. 2024;19:e0312447.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  31. Uwambajimana E, Rugirangoga P, Musabyimana E, Ingabire N, Ikibasumba J, Turikumwenimana R et al. Assessment of the use of electronic medical records system and barriers in Rwanda. 2024.

  32. National Institute of Statistics of Rwanda. Rwanda Statistical Yearbook 2023. National Institute of Statistics of Rwanda; 2023.

  33. Van Rossum G. Python programming language. In: USENIX annual technical conference. Santa Clara, CA; 2007. pp. 1–36.

  34. McCarthy J, Maine D. A Framework for analyzing the determinants of maternal mortality. Stud Fam Plann. 1992;23:23.

    Article  CAS  PubMed  Google Scholar 

  35. Joel LO, Doorsamy W, Paul BS. On the Performance of Imputation Techniques for Missing Values on Healthcare Datasets. 2024.

  36. Hang Y, Zhang Y, Lv Y, Yu W, Lin Y. Electronic medical record based machine learning methods for adverse pregnancy outcome prediction. In: Tian J, Mao K, Qian D, Xie Y, Lyu Y, editors. Twelfth International Conference on Signal Processing Systems. Shanghai, China: SPIE; 2021. p. 17.

  37. Elreedy D, Atiya AF. A comprehensive analysis of synthetic minority oversampling technique (SMOTE) for handling class imbalance. Inf Sci. 2019;505:32–64.

    Article  Google Scholar 

  38. Edmondson ME, Reimer AP. Challenges frequently encountered in the secondary use of Electronic Medical Record Data for Research. CIN Comput Inf Nurs. 2020;38:338–48.

    Google Scholar 

  39. Yim W-W, Wheeler AJ, Curtin C, Wagner TH, Hernandez-Boussard T. Secondary use of electronic medical records for clinical research: challenges and opportunities. Converg Sci Phys Oncol. 2018;4:014001.

    Article  PubMed  PubMed Central  Google Scholar 

  40. Goldstein BA, Navar AM, Pencina MJ, Ioannidis JPA. Opportunities and challenges in developing risk prediction models with electronic health records data: a systematic review. J Am Med Inf Assoc. 2017;24:198–208.

    Article  Google Scholar 

  41. Fraser HSF, Mugisha M, Bacher I, Ngenzi JL, Seebregts C, Umubyeyi A, et al. Factors Influencing Data Quality in Electronic Health Record Systems in 50 Health Facilities in Rwanda and the role of clinical Alerts: cross-sectional observational study. JMIR Public Health Surveill. 2024;10:e49127.

    Article  PubMed  PubMed Central  Google Scholar 

  42. Amoroso CL, Akimana B, Wise B, Fraser HS. Using electronic medical records for HIV care in rural Rwanda. In: Safran C, Reti S, Marin HF, editors. Proceedings of the 13th World Congress on Medical Informatics (MEDINFO 2010). Studies in Health Technology and Informatics, Vol. 160. Amsterdam: IOS Press; 2010. p. 337–41.

  43. Anokwa Y, Allen C, Parikh T. Deploying a Medical Record System in Rural Rwanda. HCI Community Int Dev CHI. 2008;2008.

  44. Escobar GJ, Soltesz L, Schuler A, Niki H, Malenica I, Lee C. Prediction of obstetrical and fetal complications using automated electronic health record data. Am J Obstet Gynecol. 2021;224:137–e1477.

    Article  PubMed  Google Scholar 

  45. Sufriyana H, Husnayain A, Chen Y-L, Kuo C-Y, Singh O, Yeh T-Y, et al. Comparison of Multivariable Logistic Regression and other machine learning algorithms for prognostic prediction studies in pregnancy care: systematic review and Meta-analysis. JMIR Med Inf. 2020;8:e16503.

    Article  Google Scholar 

  46. Moreira MWL, Rodrigues JJPC, Oliveira AMB, Saleem K, Venancio Neto AJ. Predicting hypertensive disorders in high-risk pregnancy using the random forest approach. In: 2017 IEEE International Conference on Communications (ICC). Paris: IEEE; 2017. pp. 1–5.

  47. Melinte-Popescu A-S, Vasilache I-A, Socolov D, Melinte-Popescu M. Predictive performance of machine learning-based methods for the prediction of Preeclampsia—A prospective study. J Clin Med. 2023;12:418.

    Article  PubMed  PubMed Central  Google Scholar 

  48. De Ramón Fernández A, Ruiz Fernández D, Prieto Sánchez MT. Prediction of the mode of delivery using artificial intelligence algorithms. Comput Methods Programs Biomed. 2022;219:106740.

    Article  PubMed  Google Scholar 

  49. Meyer R, Hendin N, Zamir M, Mor N, Levin G, Sivan E, et al. Implementation of machine learning models for the prediction of vaginal birth after cesarean delivery. J Matern Fetal Neonatal Med. 2022;35:3677–83.

    Article  PubMed  Google Scholar 

  50. Despotovic D, Zec A, Mladenovic K, Radin N, Turukalo TL. A Machine Learning Approach for an Early Prediction of Preterm Delivery. In: 2018 IEEE 16th International Symposium on Intelligent Systems and Informatics (SISY). Subotica: IEEE; 2018. pp. 000265–70.

  51. National Institute of Statistics of Rwanda (NISR). Fifth Rwanda Population and Housing Census, Thematic Report: Mortality. National Institute of Statistics of Rwanda; 2023.

  52. Bai J, Wong FWS, Bauman A, Mohsin M. Parity and pregnancy outcomes. Am J Obstet Gynecol. 2002;186:274–8.

    Article  PubMed  Google Scholar 

  53. Raatikainen K, Heiskanen N, Heinonen S. Under-attending free antenatal care is associated with adverse pregnancy outcomes. BMC Public Health. 2007;7:268.

    Article  PubMed  PubMed Central  Google Scholar 

  54. Kern-Goldberger AR, Ewing J, Polin M, D’Alton M, Friedman AM, Goffman D. The predictive value of vital signs for morbidity in pregnancy: evaluating and optimizing maternal early warning systems. Am J Perinatol. 2023;40:1590–601.

    Article  PubMed  Google Scholar 

  55. Ueno R, Xu L, Uegami W, Matsui H, Okui J, Hayashi H, et al. Value of laboratory results in addition to vital signs in a machine learning algorithm to predict in-hospital cardiac arrest: a single-center retrospective cohort study. PLoS ONE. 2020;15:e0235835.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  56. Kpienbaareh D, Atuoye KN, Ngabonzima A, Bagambe PG, Rulisa S, Luginaah I, et al. Spatio-temporal disparities in maternal health service utilization in Rwanda: what next for SDGs? Soc Sci Med. 2019;226:164–75.

    Article  PubMed  Google Scholar 

  57. Shamsa A, Bai J, Raviraj P, Gyaneshwar R. Mode of delivery and its associated maternal and neonatal outcomes. Open J Obstet Gynecol. 2013;03:307–12.

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to express their gratitude to the Rwanda Biomedical Center, Division of Research, Data Science, and Innovation, for their immense support in providing the necessary data for this study. We also extend our sincere appreciation to the University of Rwanda’s African Centre of Excellence in Data Science for providing the required resources to execute this project. Additional thanks go to the Data Science Training for Health program for offering invaluable technical support throughout the course of this research.

Funding

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

Author information

Authors and Affiliations

Authors

Contributions

SMH conceptualized the study, designed the research methodology, and conducted the data analysis, with this research forming a part of his thesis work. He also drafted the initial manuscript. ECN contributed to data retrieval from electronic medical records systems and provided critical revisions to the manuscript for intellectual content. MU assisted in the development of machine learning models and contributed to the interpretation of results, actively participating in manuscript revisions. FN supported the statistical analysis and assisted in data visualization, significantly enhancing the clarity of the presented findings. IK contributed to the literature review and assisted in framing the research questions, offering valuable feedback on the manuscript drafts. IN supervised the overall research project, providing guidance throughout the study and reviewing the final manuscript for submission.

Corresponding author

Correspondence to Muzungu Hirwa Sylvain.

Ethics declarations

Ethics approval and consent to participate

Ethical approval for this study was obtained from the Institutional Review Board (IRB) of the University of Rwanda with reference number CMHS/IRB/338/2024. All data used in this research were anonymized before analysis to ensure the privacy and confidentiality of participants. Identifiable information such as patient names, national identification numbers, and addresses were removed in compliance with ethical standards. As this study involved secondary data analysis of retrospective electronic medical records, formal consent from participants was waived by the IRB, given that no direct interaction with patients occurred. This study was conducted in accordance with the ethical principles outlined in the Declaration of Helsinki and relevant national guidelines.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sylvain, M.H., Nyabyenda, E.C., Uwase, M. et al. Prediction of adverse pregnancy outcomes using machine learning techniques: evidence from analysis of electronic medical records data in Rwanda. BMC Med Inform Decis Mak 25, 76 (2025). https://doiorg.publicaciones.saludcastillayleon.es/10.1186/s12911-025-02921-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doiorg.publicaciones.saludcastillayleon.es/10.1186/s12911-025-02921-z

Keywords