Development of minimum data set and electronic registry for hemodialysis patients management

Abstract

Background

This study aimed to develop a minimum dataset and an electronic registry system for hemodialysis patients to evaluate hemodialysis patients’ treatment procedures and outcomes, conduct related research, and design therapeutic interventions.

Methods

This developmental research was performed in multiple phases, including content determination using the Delphi technique; database designing using MySQL; building a user interface using PHP; usability evaluation using the think-aloud method by 10 evaluators through a scenario consisting of 7 tasks; and finally, the system was piloted by entering the 160 patients’ paper records into the system.

Results

Following the CVR and CVI content validity assessment, 108 of the 118 extracted data elements (DEs) were validated. Then, using the Delphi technique, nephrologists chose 57 DEs and divided them into 4 information categories, including the patient’s clinical history, hemodialysis episodes, laboratory findings, and the outcomes of hemodialysis. The three tabs that made up the user interface were the homepage, information recording, reports, and definitions. The problems with appearance and performance were discovered using the think-aloud method, and they were then resolved. Finally, users had the opportunity to identify issues, improve the system’s capabilities, and express their satisfaction throughout the system’s three-month test period.

Conclusions

The E-hemodialysis registry was created based on knowledge gained from industrialized nations, opinions and suggestions from medical specialists, and the facilities that were accessible. Information from this system can be utilized as a starting point for evaluating the hemodialysis patients’ status, identifying problems, and making sensible decisions for the best possible planning and management of end-stage renal disease.

Background

The National Committee on Vital and Health Statistics (NCVHS) defines health data registries as “organized systems for the collection, storage, retrieval, analysis, and dissemination of information about patients who are either prone to a health-related event or have experienced it before [1].”

Record, store, process, and follow by instant access to clinical data allow clinicians to assess a patient’s clinical needs and understand how patients’ responses to interventions improve, whether they are in remission or relapse. This data is also used to evaluate the performance of healthcare organizations, compare them with each other, and determine the utilization of resources and treatment outcomes [2, 3].

End-stage renal disease (ESRD) is a critical health issue worldwide [4], and its treatments, including dialysis and transplantation, are costly both socially and economically [5]. Given the annual growth of 5–6% of ESRD patients compared to the world population growth (1.1%), control of this disease is one of the major healthcare concerns in all countries and deserves special attention from healthcare policy-makers. However, for any decision-making and planning, it is essential to have access to accurate, comprehensive, and up-to-date statistical data and information sources [6, 7]. A system with longitudinal data storage can provide an accurate image of renal care, disease outcomes, and the effectiveness of treatments over time.

Multiple studies have evaluated the effect of renal registries on the care of patients with ESRD. For example, Valent et al. showed in a study that the renal registry has been effective in providing incidence and prevalence rates of ESRD, mortality, and comorbidities and in conducting studies related to ESRD patients in northeast Italy [8].

Couchoud et al. reported that the new ESRD registry in France (REIN) can provide the right data on monitoring ESRD patients’ improvement and enhance the decision-making processes [9].

Liu et al. also indicated that determining treatment cost data in the United States by the US Renal Data System (USRDS) has changed healthcare policies through the ESRD prospective payment system, which is designed to better manage the treatment costs for ESRD patients [2].

Therefore, it was decided to create such a register for hemodialysis patients at the dialysis center of the study site, where dialysis patients’ information has often been in the paper-pencil records. In this study, we illustrate the developmental steps of the Iranian electronic hemodialysis registry (IEHR) in detail.

Methods

Study context

This study was performed in the dialysis center of Kashan University of Medical Sciences. At this center, dialysis services are provided to an average of 300 patients per month with at least two nephrologists and 10 nurses.

Analysis of the status quo

In the feasibility study period for starting this registry, the clinical data of dialysis patients was recorded by paper-pencil forms. We observed that a large amount of data was dispersed in the paper-pencil forms and hospital information system (HIS) without any coherence between them for aggregation and analysis.

Also, the medical professionals’ primary concern was that this type of registry has no efficiency in recording, accessing, and reporting data for these patients, given that access to prior clinical data is critical for their future referrals. Therefore, an electronic recording of clinical data was proposed.

Subsequently, a meeting was convened with the clinical director and staff to ascertain the information needs and priorities. The current state of patient clinical information registration was reviewed, and based on feedback, recommendations, and information requirements, a decision was made to create an electronic registry system that would reliably collect, store, and disseminate clinical data from hemodialysis patients while also tracking their health status over time.

Study design

In this developmental research, a multi-method approach, including observation, interview, and questionnaire, was employed in four phases, as follows:

Phase 1: content selection

This phase included four parts:

• A comparative study of the registers of selected countries

First, literature was searched in the electronic databases of PubMed, Scopus, Science Direct, and Cochrane, using the keywords “registry, registration, chronic renal failure, chronic kidney disease, ESRD, renal replacement therapy, dialysis, hemodialysis, peritoneal dialysis” and a combination of these keywords to get acquainted with “renal and dialysis registries” in different countries.

The inclusion criteria for the identified registries were being in the English language, accessibility of the website, and public access to the information resources of the registries. After selecting the eligible registries, we reviewed the main dialysis data set and documents related to each of them. Based on the results of this step and the previous step, a set of essential data elements (DEs) is extracted (Fig. 1).

Fig. 1

Flow diagram of literature search and select study

• Questionnaire design

The extracted DEs were structured in the form of a questionnaire composed of five main sections, including new dialysis patients, hemodialysis treatment, an annual assessment of dialysis patients, infectious episodes of hemodialysis, and treatment outcomes of dialysis. The importance of each DE was evaluated in terms of “necessity, relevancy, simplicity, and clarity” by a panel of experts using the snowball sampling method, consisting of six nephrologists who have cooperated with the dialysis centers, two physicians, and nine nurses working in the study site with at least 3 years of experience who have been responsible for or supervised the process of recording clinical information of patients. They were selected in a targeted and accessible way and confirmed the content and final selection of the data elements of the system.

The content validity using the standard CVR and CVI formulas was calculated. The CVR measures the essentiality of an item and varies between 1 and − 1, with a higher score indicating greater agreement among panel members. The CVR formula is CVR=(Ne – N/2)/(N/2), where Ne is the number of panelists who believe an item is “essential” and N is the total number of panelists. Experts have determined the degree of validity of the instrument, which is represented by a numerical value called the CVR. One rule of thumb suggests that a CVR of at least 0.78 indicates the validity of an item or scale [10]. Then, the results were compared with the values in Lawshe’s table for a sample of six. After the process, some elements are removed.

We developed a questionnaire to determine and structure the final DEs of the system into four primary tabs, including the homepage, information recording, reports, and definitions. In the questionnaire, the scale of importance was based on a 5-point Likert scale from “strongly agree” to “strongly disagree,” with the option of “more details (if necessary)” and “other suggestions” at the end of each section. Nine nurses and two nephrologists with at least five years of experience working at the research site reviewed the questions once more.

• Conducting the Delphi technique

After the questionnaire was designed, the classic Delphi technique was used to reach a consensus about confirming or rejecting each data element. The questionnaire was submitted to a Delphi expert panel comprising ten nephrologists with over five years of professional experience in dialysis centers. The one-round Delphi technique was used to determine whether an item was necessary. The results were analyzed on the following basis: confirmation of DEs with an agreement of more than 75%; revision of DEs with an agreement of between 50% and 75% in the next Delphi round; and rejection of DEs with less than 50% agreement.

Phase 2: designing and developing the system

In this phase, after selecting the MDs, the main tasks were determining:

• 1. The data input and output, data sources, and how to do data entry and security control.

• 2. The data dictionary was created to provide detailed definitions of DEs, their attributes, and allowed value ranges.

• 3. Designing report formats.

Then, the entities and their relation tables were determined, and the system database was developed using MySQL software, which is fully compatible with the PHP programming language.

Finally, the web-based registry software was developed in the PhpStorm 2016 programming environment using Laravel, one of the PHP language frameworks. In addition, HTML5, CSS3, Jquery, and Vue.js are used in the design of registry software.

Phase 3: evaluating the usability of the system

• Evaluators

Based on the 10^2 rule for the minimum number of participants needed to complete the usability assessment [11], ten end users took part in the system usability evaluation process after being chosen using the available sampling process.

• Data collection method

Seven scenarios were defined for end users to perform specific tasks. These scenarios are given below:

• 1) Admitting a new patient

• 2) Retrieving a patient with a previous history

• 3) Recording hemodialysis information

• 4) Recording laboratory tests

• 5) Recording paraclinical tests

• 6) Recording treatment outcomes

• 7) Reporting all information in table and graph formats

In doing so, a three-hour workshop was used to acquaint the participants with the registry features. After ten days, these evaluators independently used the system in the given situation to avoid any biases.

The think-aloud method requires participants to think aloud while completing a series of predetermined activities. As they finish the tasks, participants are invited to say whatever comes to mind. This may encompass their looking at, thinking, doing, and feeling. Making thought processes as explicit as possible throughout task performance allows observers to gain insight into the participant’s cognitive processes rather than just their results. All verbalizations follow a formal study methodology that involves transcription and analysis. During a usability test, participants are asked to say and do things. Observers are required to record participant behavior without trying to understand what they are saying or doing. They are specifically asked to record challenging areas. Test sessions may be performed on participants’ computers or in a more controlled setting. Sessions are often audio and video-recorded so that developers can go back and refer to what participants did and how they reacted.

Phase 4: pilot implementation of the system

A pilot implementation allows a developer to validate the solution with a small test group to get feedback before full application deployment. The 160 hemodialysis patients’ paper-pencil dialysis records (from February 28th, 2017, to December 1st, 2020) were imported into e-registry by the two staff who worked at the study site. Through the imported data, all the objectives that were followed in developing this registry and the system functions were checked.

To ensure the correct operation of the system, the researcher asked the users to report any problems while recording and reporting the data. Also, the accuracy of the calculations and reports was validated in two ways: manually and automatically, using the system. Then the results were compared with each other.

Results

The following are the overall findings based on the work phases:

In phase 1, content selection

In the step of comparative study, a literature review, 262 articles were found by searching four databases. After removing duplicates, 206 articles were screened by reviewing the titles and abstracts. Accordingly, 172 unrelated articles were excluded. The full texts of the remaining articles (34 in total) were then examined for eligibility, and we approved them all.

By reviewing these articles [2, 12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44], 49 registries were identified. Based on the inclusion criteria, five registries, including the Australia and New Zealand Dialysis and Transplant Registry (ANZDATA) [45], Malaysian Dialysis and Transplant Registry (MDTR) [46], Singapore Renal Registry (SRR) [47], UK Renal Registry (UKRR) [48] and the United States Renal Data System (USRDS) [49], were selected and studied. The results of the comparative study of selected registry reviews are shown in Tables 1, 2, 3, 4 and 5.

Table 1 Main dialysis dataset in each of the studied registries and the study site (Section 1)

Table 2 Main dialysis dataset in each of the studied registries and the study site (Section 2)

Table 3 Main dialysis dataset in each of the studied registries and the study site (Section 3)

Table 4 Main dialysis dataset in each of the studied registries and the study site (Section 4)

Table 5 Main dialysis dataset in each of the studied registries and the study site (Section 5)

In the step of questionnaire design, the results of the CVR and CVI assessments can be seen in Tables 6, 7, 8, 9 and 10. As shown in these tables, we extracted a total of 118 data elements, which were reduced to 108 elements after the content validity process.

Table 6 Proposed data elements in the content validity evaluation process (Section 1)

Table 7 Proposed data elements in the content validity evaluation process (Section 2)

Table 8 Proposed data elements in the content validity evaluation process (Section 3)

Table 9 Proposed data elements in the content validity evaluation process (Section 4)

Table 10 Proposed data elements in the content validity evaluation process (Section 5)

In the final step, according to results obtained from Delphi (Tables 11, 12, 13 and 14), 57 DEs were selected as MDs for system content.

Table 11 Proposed data elements of the selected content for the system design (Section 1)

Table 12 Proposed data elements of the selected content for the system design (Section 2)

Table 13 Proposed data elements of the selected content for the system design (Section 3)

Table 14 Proposed data elements of the selected content for the system design (Section 4)

In phase 2, designing and developing the system

The data dictionaries were created. Two examples of them are presented in Tables 15 and 16.

Table 15 “Cause of renal failure” data element in the data dictionary

Table 16 “Hgb” data element in the data dictionary

The requested reports are provided in Table 17.

Table 17 Type of registry reports

The database and web-based software were designed and developed for the electronic registry of hemodialysis patients’ data (Figs. 2, 3, 4, 5, 6 and 7).

Fig. 2

The reports from the IEHR

Fig. 3

The patient information in the IEHR

Fig. 4

The hemodialysis information in the IEHR

Fig. 5

The laboratory test in the IEHR

Fig. 6

The preclinical test in the IEHR

Fig. 7

The clinical outcomes in the IEHR

The registry software has the following capabilities: (1) displaying a metric dashboard on a daily, weekly, monthly, and annual basis; (2) searching for and manipulating patient data across all sections; (3) displaying error messages while entering data; (4) reporting data in table and graph formats; and (5) the ability to delete or add basic definitions in the definitions tab.

In phase 3, the usability evaluation

During this evaluation, based on the observations and cases documented by the researcher, issues and barriers that users faced when interacting with the system were identified and classified into two categories: appearance and function, as shown in Table 18.

Table 18 The problem discovered from think aloud method

In phase 4, pilot implementation

The identified problems were related to the following:

• How to enhance the slow speed

• How to get a report graphically for laboratory parameters in the laboratory tests tab

• How to automatically calculate the iron saturation percentage formula in the quarterly test tape

• How to record more than one outcome in the treatment outcomes tab

These problems are fixed by removing bugs, reviewing and configuring the browsers, and listening to the users’ suggestions. Metrics calculations performed manually and automatically using e-registry revealed no differences, confirming the accuracy of the system calculations. Also, the users reported that they could easily and quickly access patient histories and track patient status and trends over time, compared to paper-based medical records.

Discussion

Nephrologists manage dialysis patients based on clinical knowledge in the area of renal care. The demands for quick and easy access to comprehensive clinical data to evaluate the patient’s health status, optimize care, and handle comorbidities are not met by paper-based medical records. Therefore, an information system is required to manage heterogeneous data [50, 51]. To accomplish these objectives, RD registries have been intensively used to collect, store, and analyze data, as well as produce periodic reports for administrative, therapeutic, or scientific purposes [3].

For example, Mendu et al. have described the development of an EHR-based chronic kidney disease (CKD) registry in Massachusetts to obtain strategies to improve the standard of renal care. They believed that the development of registries is a crucial public health strategy that identifies and addresses gaps in the wide spectrum of clinical care for CKD [52]. Venuthurupalli et al. have discussed the production of a CKD registry in Queensland to deploy and administer a database of CKD patients’ data to achieve long-term outcomes, highlighting clinical care patterns and educating clinicians, researchers, and patients with CKD [53].

Heaf et al. have described the Danish Nephrology Registry (DNR), which provides progs and therapeutic information on patients with ESRD, as well as epidemiological data for several international organizations and data required for clinical research [54]. Jin et al. have explained that the goals of the Korean dialysis registry include estimating the number and regional distribution of patients; analyzing patients’ progression of illness and dialysis treatment outcomes; enhancing the standard of care; and providing socio-economic data for long-term health plans [55]. In a study, Ho et al. evaluated the Hong Kong Renal Registry (HKRR). It is a clinical and administrative database that is utilized for research and quality assurance programs [56].

According to Ajami et al., the RD registries used in various nations are notable systems for modeling renal registries because they have described the goals and features of such registries [57, 58]. In this study, we reviewed the data content of renal and dialysis registries in chosen countries as a comparative study. The content of registries can be divided into five categories associated with subcategories. Accordingly, a patient’s demographics, clinical history, dialysis treatments, laboratory findings, and treatment outcomes were the heading categories. After that, DEs were divided into appropriate sections for recording in each of the reviewed registries, following how to do data recording. Here, the most important issue was creating a data dictionary and defining rules for data collection and monthly reports after the data collection, recording, and storage processes.

To achieve a good e-registry, developing databases, managing data, and building a proper UI is critical. The poor and complex UI design makes it difficult to understand and utilize and leads to user dissatisfaction. In addition, usability testing is also an inevitable step [59,60,61,62]. Expert-based approaches often fail to provide precise explanations of the root causes of problems or recommendations for enhancing the user interface. The use of user-based evaluation techniques, such as the think-aloud method, offers a deeper understanding of system usability issues [63, 64].

Regarding usability testing, Peute and Alhadreti compared the effectiveness of both concurrent and retrospective think-aloud methods. They explained that the concurrent method is noticeably more effective in identifying usability issues and also advised using this approach for the formative assessment of health information systems [65, 66]. Zakaria et al. evaluated the usability of the registry of maternity and newborns using concurrent think-aloud and focus group sessions and found the key flaws in the system architecture needed to be rectified by user feedback [67]. Shahraki et al. also found that the evaluation of user experiences is the most important component in system development to address problems, reduce high costs, and boost user acceptance because designers can make use of these to improve the usability and effectiveness of the system [68].

In this study, the concurrent think-aloud method was employed in the usability testing to involve the users with the system’s operation, enhance the UI, and identify the parts that needed improvement.

Before generally implementing health information systems, conducting a pilot system test can help discover any faults or problems because the users connect to the system in their routine [69]. In a study, Solomon et al. explained that the pilot deployment of a registry must take place at an acceptable period and quantity of samples to identify the gaps, remove the problems, and plan an efficient training program for users [70]. Also, Mehmood et al. suggested that a plan for pilot testing that lasted three months was necessary to assess the system’s effectiveness [71].

This study analyzed data from all 160 patients’ medical records at a dialysis center, entered into the IEHR. Users interacted with the system during a pilot phase, providing valuable feedback to improve its capabilities. They reported that accessing patient histories and tracking patient status and trends over time was significantly faster and easier compared to paper-based records.

The study’s findings and methods can be applied to develop similar registries in other countries, as they utilize the results of comparable studies to identify comprehensive data elements. While the study did not have significant limitations, it’s important to note that the system was implemented as a pilot to retrospectively register cases. Future research should focus on evaluating the system’s performance for new cases and measuring its impact on staff performance and patient care.

The study’s strengths include the use of multiple data-gathering methods (questionnaires, observations, and interviews) and the involvement of nurses and other key stakeholders in the design phase. However, the study had several limitations. First, the initial phase focused solely on registries with published design and creation articles, potentially excluding similar registries without such documentation. Second, the usability assessment relied exclusively on a user-based approach, which, while valuable, could have been complemented by expert-based techniques. Nonetheless, the user-centered approach was crucial for identifying real-world usability issues.

Conclusions

Due to the absence of a national kidney disease registry in Iran, we developed e-registry to record and analyze clinical data of hemodialysis patients. This registry aims to provide insights into the health status of hemodialysis patients over time, evaluate treatment effectiveness and outcomes, identify care challenges, generate timely reports, provide alerts for potential risks, and ultimately support informed decision-making for better management of hemodialysis patients.

By comparing registries from selected countries, we created a comprehensive registry design. Pilot testing identified functional deficiencies, which were addressed. This design is not static but will continue to evolve based on user feedback and needs.

As this was primarily a design and development study, specific outcome measurement was not the primary focus. The system’s performance was evaluated by its ability to accurately record clinical data from existing paper-based medical records.

The successful registration of 160 patient records by users during the pilot phase suggests the system’s potential for practical application.