Design and evaluation of an electronic follow-up questionnaire for patients after percutaneous coronary intervention

Abstract

Background

Patient-centered, measurable, and transparent care is essential for improving healthcare outcomes, particularly for patients undergoing percutaneous coronary intervention (PCI) procedures. Electronic follow-up questionnaires offer the potential for efficient and accurate data collection, enhancing the monitoring of patient experiences and outcomes. This study aimed to design and evaluate an electronic follow-up questionnaire tailored for post-PCI patients, focusing on real-time symptom monitoring and data collection.

Methods

This developmental study was conducted in 2020 in three phases. In the first phase, a follow-up questionnaire was developed through a needs assessment and expert consultations. Each item’s content validity ratio (CVR) and content validity index (CVI) were evaluated to ensure content validity. The finalized questionnaire elements were then reviewed and refined by a panel of ten cardiologists using the Delphi technique. In the second phase, an electronic platform was designed to host the follow-up questionnaire. The tool’s effectiveness for post-PCI follow-up was evaluated in the third phase.

Results

Cardiologists confirmed all items in the Delphi technique’s first round, validating the follow-up questionnaire’s content. A total of 41 patients undergoing PCI were enrolled in the study. The most frequently reported symptoms included issues at the catheter insertion site, chest discomfort, digestive complications, and shortness of breath. Of these patients, 21 (51.2%) utilized the electronic follow-up tool. The primary reasons for non-participation were busy schedules, forgetfulness, and perceived recovery. Among the participants, 16 (76.2%) expressed high or very high satisfaction with the tool.

Conclusion

The findings suggest that this electronic follow-up questionnaire has the potential to effectively collect clinical data, support academic research, and improve the quality of post-PCI care. However, addressing barriers to patient participation and involving patients in the tool’s iterative development will be critical for enhancing its adoption and impact.

Introduction

Coronary artery disease (CAD) is a major health concern caused by the formation of atherosclerotic plaques in coronary arteries. It is associated with significant morbidity, mortality, and economic burden [1, 2]. There are various diagnostic methods for coronary artery diseases, including angiography, exercise tests, echocardiography, and heart scans. Definitive diagnosis of cardiovascular diseases is made by angiography. Based on the anatomical conditions of the coronary arteries, drug treatment, coronary artery bypass graft (CABG), or percutaneous coronary intervention (PCI) are selected for treatment [1]. PCI is widely recognized as an effective treatment for obstructive coronary artery disease, offering fewer side effects compared to CABG [3]. In the United States (US), it is estimated that more than 600,000 PCIs are performed each year [4]. Despite its benefits, PCI presents challenges in managing both short- and long-term side effects. Key complications include contrast-induced nephropathy (CIN), myocardial infarction, emergency revascularization, and bleeding [5, 6]. Effective post-PCI care is critical to mitigate these risks and improve patient outcomes. In recent years, electronic follow-up questionnaires have emerged as valuable tools to enhance patient participation in disease management processes. These tools enable patients to record and report symptoms, facilitating better communication between patients and healthcare providers [7, 8]. Unlike traditional paper-based methods, electronic questionnaires allow for real-time data collection and processing, reducing the likelihood of ambiguous responses and ensuring data accuracy [9,10,11,12,13]. The widespread adoption of smartphones and tablets has further simplified the use of these tools, making them accessible across diverse populations [10]. Research highlights the preference of healthcare teams for electronic tools over paper questionnaires due to their efficiency and improved data quality [14]. However, there remains a gap in the availability of suitable electronic tools specifically designed for post-angioplasty follow-up. Addressing this need, the present study aimed to develop and evaluate an electronic follow-up questionnaire tailored for patients undergoing PCI. The objectives included assessing the validity and reliability of the tool, its usability, and its impact on patient engagement and satisfaction during post-PCI care.

Methods

The present study was a developmental, single-center, prospective study conducted in a university hospital in 2020. The Ethics Committee of Kashan University of Medical Sciences approved the study (IR.KAUMS.REC.1398.051). The study consisted of three distinct phases:

• 1. Development of follow-up questionnaires.

• 2. Design of an electronic platform for reporting outcomes.

• 3. Evaluation of the tool’s effectiveness for patient follow-up.

Phase I: Development of the follow-up questionnaires

This phase was carried out in two stages: (1) designing questionnaires and (2) determining the items of the questionnaires by surveying experts (Fig. 1).

Fig. 1

Phase I of research to identification of data elements and development of standardized questionnaires to gather PRO

First stage - Designing questionnaires

In the first stage, a comprehensive needs assessment was conducted to identify the necessary items for effective follow-up. The needs assessment process included an extensive review of existing post-discharge questionnaires and consultations with a multidisciplinary panel of experts, including cardiologists, epidemiologists, and medical informatics specialists. These questionnaires followed the guidelines stipulated by the Ministry of Health of Iran. This thorough approach ensured that all relevant information elements required for patient self-care and monitoring were captured in the designed questionnaires. The initial draft of the questionnaire included 35 questions: 10 related to follow-up in the first two weeks after angioplasty and 25 questions between 2 weeks to 3 months after the angioplasty procedures.

Additionally, an open-ended question was included for specialists’ comments. The questionnaire was given to three cardiologists and a Cardiac intervention sub-specialist with at least three years of work experience to assess face validity. The participants were asked to comment on the questions’ difficulty level, relevancy, and ambiguity. The participants’ opinions about the items were carefully recorded, and they were asked to explain more where necessary. Finally, with the advice of the experts, changes were applied to the questionnaire, and face validity was approved. The content validity of the questionnaires was assessed by a 10-member panel consisting of faculty members of Kashan University of Medical Sciences (cardiologists = 6, epidemiologist = 2, and health information management = 2). The validity of each item (i.e., relevance, clarity, and simplicity) plus the validity index of the entire tool was determined based on a four-point Likert scale from “unfavorable” (score of 1) to “totally favorable” (score of 4). The content validity of each item was assessed based on the content validity index (CVI), and if less than 0.75, the item was eliminated. The content validity ratio (CVR) was also determined based on Lawshe’s Table [15], and the items with values less than 0.62 were eliminated. The Content Validity Ratio (CVR) for all items in the questionnaire exceeded 0.7, indicating that no items were eliminated based on this metric. Additionally, the Content Validity Index (CVI) for relevance, clarity, and simplicity of all questionnaire items was above 0.8, ensuring that no items were removed based on this criterion. After summarizing the opinions, the final questionnaires included demographic and clinical information, follow-up-related questions, and an open-ended question for specialists to deliver their commentaries (Additional file 1, 2). To ensure the reliability of the questionnaire, expert validation and the Delphi technique were employed, focusing on content consistency and clinical relevance.

Second stage - Determining the items of the questionnaires using the Delphi technique

After determining the validity and reliability, the final questionnaires were provided to 10 cardiologists by email. The Delphi technique was employed to reach a consensus on the necessity of each item. After providing a complete explanation of the study objectives, they were asked to comment on the necessity of each item identified in the questionnaire using a 5-point Likert scale from ‘strongly agree’ to ‘strongly disagree’. A score of 1 meant ‘strongly disagree’, 2 meant ‘disagree’, 3 meant ‘neutral’, 4 meant ‘agree’, and 5 meant ‘strongly agree’. Then, the average final score for each item was calculated. Items with an average score of 3.75 and above were finally confirmed, and items with an average final score of less than 2.5 were discarded. Items scoring between 2.5 and 3.75 underwent an additional round of review to reach an agreement on their confirmation or exclusion.

Phase II: Designing the electronic follow-up tool

Before coronary angioplasty, the cardiology department created comprehensive demographic profiles for the 21 patients scheduled for the procedure. After explaining the project and introducing the electronic follow-up system to the patients, demographic information, phone number, address, caregiver’s details, internet access status, history of underlying diseases, family medical history, and patient habits were collected. Detailed verbal instructions on using the system and the dates when the questionnaires would be available were provided. A user account was created for the patient, and the username and password were given to the patient or their companion. A detailed instruction sheet on how to log in to the website, use it, access the questionnaires, and contact the research team if needed was provided to the patient. This information was also sent via a website link in a text message to the patients or their caregivers. Subsequently, the patient’s information was fully registered in the system, and the availability dates of the questionnaires were set. After determining items and preparing follow-up questionnaires, the electronic follow-up tool was designed as a website. The website was implemented and provided by the Sarafraz Rayan Ghasedak Company. This website has two access levels for doctors and patients. By using this website, doctors could create user accounts, record patients’ basic information (demographic characteristics and clinical history), design electronic questionnaires, and schedule the completion of each questionnaire based on the time of the patient’s angioplasty. The electronic follow-up tool was designed to be accessed on various devices, including computers, tablets, and smartphones. This multi-device compatibility ensures that patients can report their outcomes and use the most convenient device to engage with the follow-up process.

Phase III: Determining the effectiveness of the electronic follow-up tool

During patient recruitment for phase III, subjects scheduled for coronary angioplasty were screened for eligibility. Inclusion criteria included internet access and the ability to use it. After explaining the study’s objectives and obtaining informed consent, demographic and clinical characteristics were collected through interviews and the hospital information system (HIS). After explaining the study’s objectives and obtaining informed consent, demographic and clinical characteristics were collected through interviews and the hospital information system (HIS). Participants were informed that they could withdraw from the research and that their answers would remain private. Detailed explanations about using the system and completing the questionnaires were given orally to the patients. Following these explanations, a structured follow-up process was implemented to ensure adherence to the advice and monitor patient outcomes. This involved daily website monitoring for patient responses. In cases where complications arose, our team took necessary measures based on the severity of the complications. For mild complications, we promptly provided recommendations to the patient via SMS. These recommendations were derived from established clinical protocols and guidelines for post-percutaneous coronary intervention (PCI) care, ensuring accuracy and appropriateness. Furthermore, in cases where danger signs were identified, the patients and their caregivers were promptly advised to seek immediate medical attention at the hospital’s clinic or emergency department. Patients were required to complete the follow-up questionnaires twice according to the plan. The questionnaires were scheduled to be completed in the first two weeks and between two weeks to three months after the percutaneous coronary intervention (PCI). This schedule ensured continuous monitoring and timely intervention if needed.

Statistical analysis

We report frequencies (n) and percentages (%) for categorical data and means (M) and standard deviations (SD) for continuous data.

Results

Identification of items and development of questionnaires

During the Delphi technique stage in the first phase of this study, seven experts participated, all of whom were cardiologists. The average age of the participants was 45 years, ranging from 37 to 60 years old. The cardiologists confirmed all data elements in the first round of the Delphi technique.

Patient characteristics

A total of 49 patients undergoing PCI were identified. Eight (16.32%) were excluded due to lack of internet access. Among the remaining 41 patients, 21 (51.2%) were women and 20 (48.8%) were men. The average age of the patients was 61.8 years. Only one patient (2.4%) was single. Ten patients (24.4%) were illiterate, while seven (17.1%) held at least a diploma. In contrast, none of the patients’ caregivers were illiterate, with 29 caregivers (70.7%) having higher education. Among the 41 included patients, 21 (51.2%) were admitted as emergencies, and 20 (48.8%) were admitted electively. The most common diagnoses requiring PCI were two-vessel disease (2VD) (46.3%) and three-vessel disease (3VD) (31.7%) (see Table 1).

Table 1 Baseline characteristics of patients (n = 41)

Only 20 patients (48.8%) reported common symptoms during the first two weeks after PCI. The most frequently recorded complaints were related to the catheter insertion site (pain, swelling, and bruising) (61.0%), chest discomfort (24.4%), and digestive complications (22.0%). After informing patients about their conditions, necessary instructions were provided. During outpatient visits, none of the patient’s symptoms were identified as severe problems, and none required repeat PCI. From 2 weeks to 3 months after PCI, only 11 patients reported common symptoms. The most frequent complaint was chest discomfort, reported by nine patients (22.0%). The second most common complaint was shortness of breath, affecting six patients (14.6%) during this period (see Table 2).

Table 2 Findings related to the symptoms reported by patients after PCI

During the first two weeks after PCI, 36 patients (87.8%) returned to their usual status. Four patients could only care for themselves, and one patient was inactive. Thirty-four patients left home during this period, with 20 of them resuming work activities. Within three months after PCI, 38 patients (92.7%) returned to their usual activities at home, and 29 (70.7%) engaged in heavy activities outside the home, including work. One patient (2.4%) participated in vigorous sports activities, while 17 patients (41.5%) engaged in light sports activities such as walking (see Table 3).

Table 3 Findings related to the activity status of patients after PCI

Evaluation of effectiveness

In total, 21 patients (51.2%) completed at least one of their electronic follow-up questionnaires, while follow-up for other patients was conducted via telephone. The highest participation rate was observed in the first week’s questionnaire, with 20 patients (48.8%), while the lowest was three months after PCI, with 9.8%. We compiled the most common patient queries throughout the follow-up period, totaling 16 questions. Fourteen questions were addressed through the electronic system, and two were handled via phone calls or text messages. The most frequently asked question pertained to medication usage. Of those with inquiries, six individuals (37.5%) were advised to consult a doctor promptly, with three (18.8%) being urged to visit the hospital immediately (see Table 4). Patient satisfaction with the electronic follow-up tool was notably high, with 76.2% of participants reporting high satisfaction levels (see Table 5). The primary reasons for non-participation were busy work schedules and forgetfulness (see Table 6).

Table 4 Findings related to patient questions after PCI

Table 5 Findings related to patients’ satisfaction with the electronic patient-reported outcomes tool

Table 6 Findings related to the reason for not participating in electronic patient-reported outcomes tool

Discussion

In this study, we developed an electronic follow-up tool based on expert opinions to report outcomes. Previous research has demonstrated that compared to routine surveillance, electronic follow-up tools can improve survival and quality of life in various patient populations [9, 15]. To our knowledge, this study is the first prospective investigation of electronic follow-up tools for the follow-up of patients who have undergone PCI. However, more research is needed to fully understand the impact of these tools on patients with cardiovascular problems. We used two questionnaires in this study to monitor the long-term symptoms of patients who underwent PCI. Cardiovascular specialists identified the data elements for these questionnaires, whereas, in some studies, the needs assessment was conducted solely by patients. Recent studies have highlighted the lack of specialist involvement in designing and creating such tools, raising concerns about the medical content’s uncertainty and endangering patient health and safety. Therefore, medical applications for important decisions must be developed with input from clinical experts [16]. Analysis of the needs assessment data revealed that all information elements were necessary. The most important parameters for the self-care of patients were monitoring complications, checking the patient’s activity status, and following the treatment regimen. In the study by Barker et al. (2021), the most important symptoms identified in patients after PCI were chest discomfort, shortness of breath during various activities, lack of confidence in performing usual activities, sleep disturbances, feelings of unhappiness, and problems related to sleep [17]. The differences between the information requirements determined in this study and other studies can be attributed to our focus on patients who have undergone PCI. Previous studies investigating electronic follow-up tools in cardiovascular patients likely involved individuals with a broader range of cardiovascular conditions. Alternatively, they may have focused on general disease management. However, our study aimed to identify post-procedural complications as quickly as possible. This focus on early detection of issues might have led to a different set of information needs compared to studies primarily concerned with long-term cardiovascular health management. In most of the reviewed studies related to the follow-up of cardiovascular patients, various tools such as email and telephone methods have been used for patient follow-up. A systematic review showed the positive effect of telephone follow-up on the quality of life, pain, physical activity, mental state, patient information, medication compliance, and even the lipid profile of patients [18]. Another review examining email as a communication method between patients/caregivers and healthcare professionals found that email was not significantly superior to traditional follow-up methods. While telephone follow-up promotes more patient lifestyle changes, email does not exhibit this effect [19]. Additionally, a recent meta-analysis [20] regarding the use of mobile phone technology in managing ischemic heart disease, heart failure, and blood pressure showed that mobile phone technology significantly reduces the re-hospitalization rate of patients. Moreover, a recent meta-analysis on the use of mobile phone technology in managing ischemic heart disease, heart failure, and blood pressure highlighted that such technology significantly reduces the re-hospitalization rate of patients [21].

Despite the promising results, the present study faced a significant limitation concerning the participation rate. The participation rate of patients was 51.2%. Only 20 of the 41 participants completed the first week’s questionnaire. This rate gradually decreased in subsequent questionnaires. Our findings indicated that demographic and clinical characteristics did not significantly influence participation. The most common reasons for non-participation included busy work schedules, forgetfulness, and a sense of recovery. Additionally, the usability and layout of the website may have hindered patient engagement. Also, patients may have difficulty navigating or interacting with the website, leading to reduced participation rates. Future studies should consider conducting usability testing to identify and address potential barriers to patient engagement with electronic follow-up tools.

Moreover, the patient and their caregivers cited other reasons for not participating in the plan, such as not paying attention to the participation process and lacking awareness of the importance of managing the disease condition. In 2019, Zhu et al. developed internet software aimed at monitoring patients post-heart surgery [22]. The participation rate in their study was notably higher at 90.2% compared to our current. Also, the telephone interview results of the 123 patients who did not participate in the follow-up assessment were as follows: 34 patients did not pay attention to postoperative follow-up; 18 patients had mobile phone problems (the primary contact number was a relative’s phone number, not in service, no longer listed, or incorrect) and did not receive reminder text messages; 32 elderly patients did not know how to use the Internet, the app, or text messaging; 21 patients directly contacted the doctor in charge, and 15 patients were hospitalized at a local hospital or during the follow-up of the local hospital. In addition, three patients had not been examined for INR after discharge [22]. Several strategies can enhance engagement to address the issue of missing patients. Educating patients about the importance of follow-up and the benefits of electronic follow-up tools, sending regular reminders via SMS, email, or calls, and offering incentives for completing follow-up questionnaires can boost participation. Also, simplifying access to these tools with a mobile app version and user-friendly interface, establishing a support team for technical issues, personalizing follow-up schedules, and ensuring robust data security with clear privacy policies are essential. Collaborating with healthcare providers to integrate these tools into routine care and implementing a feedback system to include patient feedback are crucial steps in fostering a sense of involvement among patients in the improvement process. The level of patient satisfaction reported in the study was notably high, which reflects positively on the acceptability and effectiveness of the electronic follow-up tool. In a study conducted at a heart failure clinic on the implementation of real-time assessment of patient-reported outcomes, the questionnaire completion rate was reported as 58% [23]. Similarly, a mobile application study for heart surgery patients reported high user satisfaction (94% recommended the program, 98% found it useful) and highlighted the potential for reduced healthcare visits [24].

Limitation

This study has several limitations. First, the questionnaire was developed primarily based on expert opinions, without direct involvement of patients in the item selection and validation process. Lack of patient participation in the initial phases of questionnaire development may have limited the tool’s ability to fully capture patient priorities and concerns. Future studies should incorporate direct patient feedback in content validation to enhance the questionnaire’s relevance and usability. Second, the study was conducted in a single-center setting, which may limit the generalizability of the findings. Differences in patient demographics, healthcare infrastructure, and follow-up practices across different institutions may influence the effectiveness of the electronic follow-up questionnaire. Multi-center studies are needed to validate and refine the tool in diverse clinical settings. Third, the participation rate in the electronic follow-up was moderate, with only 51.2% of patients completing at least one questionnaire. The most common reasons for non-participation included busy schedules, forgetfulness, and a perceived sense of recovery. Additionally, usability barriers such as difficulty navigating the platform may have contributed to lower engagement. Future studies should explore strategies to improve patient adherence, such as user-friendly mobile applications, reminder systems, and enhanced patient education on the benefits of electronic follow-up. Lastly, the study did not assess psychometric properties such as internal consistency and construct validity, as the tool was designed for structured symptom reporting rather than for measuring a latent construct. Future research should consider alternative validation methods, such as test-retest reliability or usability testing, to further evaluate the tool’s robustness and effectiveness.

Conclusion

This new follow-up platform demonstrates potential in effectively collecting clinical data and providing technical support for academic research. However, patient adherence was moderate, with only 51.2% of patients completing the electronic follow-up tool. The primary reasons for non-participation were busy work schedules, forgetfulness, and a sense of recovery. Future efforts should address these barriers to improve patient engagement and adherence.