Implementation of a Laboratory Information Management System (LIMS) for microbiology in Timor-Leste: challenges, mitigation strategies, and end-user experiences

Abstract

Background

Effective diagnostic capacity is crucial for clinical decision-making, with up to 70% of decisions in high-resource settings based on laboratory test results. However, in low- and middle-income countries (LMIC) access to diagnostic services is often limited due to the absence of Laboratory Information Management Systems (LIMS). LIMS streamline laboratory operations by automating sample handling, analysis, and reporting, leading to improved quality and faster results. Despite these benefits, sustainably implementing LIMS in LMIC is challenging due to high costs, inadequate infrastructure, and limited technical expertise.

Methods

This study evaluated the implementation of a customised microbiology LIMS at the National Health Laboratory (NHL) in Timor-Leste. The LIMS was deployed in November 2020, with an accompanying online results portal introduced in early 2021. The implementation was assessed via a checklist based on key challenges and requirements for LIMS in LMIC, alongside a post-implementation survey of scientists and clinicians.

Results

The assessment revealed significant improvements in laboratory processes, including enhanced sample throughput, data management, and result reporting. The LIMS reduced transcription errors and standardised reporting of antimicrobial susceptibility testing (AST), improving data quality and accessibility. However, challenges such as unreliable internet connectivity and the need for ongoing funding and technical support persist. The user satisfaction survey, with responses from 19 laboratory scientists and 15 clinicians, revealed positive feedback on workflow improvements and result accessibility, although concerns about internet speed, sustainability, and the need for further training were noted.

Conclusion

This study highlights the importance of careful planning, customisation, and stakeholder engagement in LIMS implementation in LMIC. The success in Timor-Leste demonstrates the potential for improved laboratory quality and patient outcomes, but also underscores the need for ongoing investment in infrastructure, technical expertise, and sustainability planning.

Background

Diagnostic capacity plays an integral role in clinical decision-making [1]. In high-resource healthcare settings up to 70% of clinical decisions are based on clinical laboratory test results, whereas in low- and middle-income countries (LMIC), clinical decision-making may not always rely on laboratory test results due to limited access to these services or through the lack of a Laboratory Information Management System (LIMS) for communicating results in a timely way to clinicians [2].

A LIMS is a software-based solution designed for analytical laboratories. It tracks and collects patient-level data, allows for registration of samples, and reports results via a database. It simplifies and automates the process of sample handling, analysis, and reporting [3]. A LIMS acts as an integral unit for the distribution and analysis of laboratory data generated from different sources and as an interface with automated laboratory systems and platforms. A key benefit of a LIMS is the ability to streamline data flow within the laboratory and centralise information onto one database [4]. In high-resource settings, a LIMS has been shown to shorten time-to-result and improve laboratory data quality [3, 5]. An effective LIMS is a useful tool for patient and sample management and has been identified as a key deficiency area for antimicrobial resistance (AMR) surveillance in LMIC [6,7,8].

Published data indicate that the implementation of LIMS suitable for microbiology in LMIC can help laboratories improve quality standards, reduce the number of transcription errors, reduce turnaround time for patient results, and improve patient outcomes [9]. In a recent publication, Turner et al. outlined the numerous barriers and lack of published studies regarding implementation of a LIMS in LMIC. They describe the lack of a fit-for-purpose and open-source LIMS suitable for use in LMIC, and outline key steps required for future program planning and improvement. Highlighted are the significant improvements needed in information technology infrastructure and data management systems in most LMIC microbiology laboratories, and a lack of investment in AMR surveillance more broadly [6].

LMIC healthcare services often have limited resources, and LIMS implementation requires upfront investment and ongoing operational and maintenance costs. The informatics requirements for diagnostic laboratories are complex, and even more so when required for clinical microbiology and AMR surveillance. Diagnostic microbiology generates large volumes of data that require efficient management, standardisation, accuracy, interoperability and integrity, which can prove difficult in settings with limited resources and guidance [10]. Many LMIC have internet that is slow and expensive, and improvements are difficult to justify to policy makers [11]. Ideally, a LIMS should not be a standalone system for microbiology only but be able to be incorporated across all departments ensuring a cohesive reporting system. Additionally, a LIMS must be customised to meet the specific needs and workflows of diagnostic laboratories in each country. This requires technical microbiology expertise, IT support, and locally-hosted versions of the software [12]. LIMS integration with existing laboratory equipment and systems (e.g. automated AST) is crucial for efficient data exchange and workflow automation. LMIC often use a mix of older, non-digital equipment, or donated analysers that may not be fit for purpose, and may have limited compatibility between systems [13]. Availability of both IT and scientific expertise are essential for LIMS sustainability, and in many LMIC technological literacy is lower than in high-resource settings, requiring longer mentoring before self-sufficiency [14].

When choosing a LIMS, it must comply with relevant regulatory requirements and data protection policies. Regulatory frameworks may be less developed or non-existent in some LMIC, and ensuring compliance can be a challenge [15]. In addition to the above challenges, when considering a LIMS appropriate for microbiology and AMR surveillance additional factors must be carefully considered. Adherence to international standards such as those developed by the European Committee on Antimicrobial Susceptibility Testing (EUCAST) and the Clinical and Laboratory Standards Institute (CLSI), along with the need for cascade reporting to support antimicrobial stewardship, requires a laboratory information management system (LIMS) with customisable expert rules, comments, and breakpoint interpretations. Microbiology results are often a combination of qualitative, semiqualitative and/or quantitative data, leading to increasingly complex data structures. Culture results can frequently contain multiple organisms with unique susceptibility results and comments, meaning the ability to customise a report design unique to each site is paramount to a successful LIMS implementation [16].

Prior to LIMS implementation, microbiology results in Timor-Leste were handwritten on a generic form, and no consistent reporting of antimicrobials was followed. Antimicrobial susceptibility testing (AST) interpretation was performed manually, utilising out-of-date breakpoints resulting in frequent errors in interpretation. Sample throughput was low, with an average of less than 80 samples per week processed for culture. In November 2020, a new commercial LIMS (SchuyLab CGM) was implemented at the National Health Laboratory (NHL), followed in early 2021 by an online results portal (SchuyNet CGM) for clinicians at the Hospital Nacional Guido Valadares (HNGV) in Timor-Leste, and subsequently in five other referral hospitals in Timor-Leste. Prior to implementation, extensive research into an appropriate LIMS for the setting was performed; selection was based on functionality, associated costs, and ease of use for diagnostic microbiology and other pathology departments. After extensive customisation and testing, the LIMS first started being utilised for clinical samples in microbiology in September 2020, and by mid-2021 was also established in the molecular, biochemistry, haematology, and serology departments of NHL and HNGV. As of June 2024, the NHL receives an average of 178 samples per week for microbiology culture, over double the previous throughput of the laboratory. Implementation of the LIMS happened alongside ongoing capacity building in laboratory and clinical microbiology to build understanding of the utility of microbiological diagnostic testing, increase service demand, throughput and quality of results, and support interpretation of AST results in patient care.

In this study, we aimed to evaluate the implemented microbiology LIMS in NHL using a comprehensive self-assessment checklist and to assess the end-user perception of the LIMS through a post-implementation survey of scientists and clinicians.

Methods

LIMS Checklist Development

To thoroughly evaluate the LIMS and compare the fulfilment of essential requirements between systems, we developed and performed a self-assessment against a predefined list of challenges and requirements determined crucial for successful LIMS deployment in LMIC settings. As there is no existing framework for assessment of LIMS suitability for use in LMIC, a unique checklist was developed with the aim of systematically identifying and addressing key barriers and ensuring that the chosen LIMS meets the necessary operational and strategic goals of the NHL. As only a limited number of studies have been published on this topic, the criteria for inclusion were selected on the basis of common barriers and requirements highlighted in the literature and were organised into several themes described below (Fig. 1.) [6, 12]. The complete LIMS self-assessment checklist is provided in the Supplementary Data.

Fig. 1

LIMS implementation challenges. Generated utilising draw.io

Use of the Checklist

This comprehensive checklist has been developed as a tool to systematically assess the appropriateness of a LIMS for use in LMIC. Having a list of specific requirements facilitated a structured approach to ensure that all critical aspects of the LIMS appropriateness for implementation were addressed, providing a clear framework for continuous improvement and ensuring the system's overall effectiveness and sustainability. Although these requirements were considered essential in the context of Timor-Leste, each country will have specific requirements of a LIMS and appropriateness of any system should be considered within the specific country context.

The criteria included for self-assessment were as follows:

Financial Investment: Assessment of the total cost of purchasing and installing the LIMS, including hardware, software, and initial setup services as well as an evaluation of the procurement process and funding sources. Recurring expenses such as software licences, maintenance fees, technical support, and training are evaluated. Identification and evaluation of funding sources, such as government budgets, international grants, and private sector contributions.

Infrastructure: Evaluation of the reliability of the power supply, including the availability of backup generators, that may disrupt LIMS functionality. Assessments of the quality and reliability of internet connections, including measurements of bandwidth, latency, and uptime, which are critical for the LIMS to function properly.

Human Resources: Assessment of the availability of IT support staff with the necessary skills to maintain and troubleshoot the LIMS. Assessment of the current skill levels of laboratory and IT staff and the required ongoing training needs. Evaluation of laboratory personnel familiarity with high-level microbiology workflows and how they relate to LIMS requirements. Assessment of potential ‘super users’ who could lead customisation and training efforts.

System-wide integration: Assessment of the compatibility of the LIMS with existing laboratory equipment and information systems. Evaluation of the ability of the chosen LIMS to interface with automated antimicrobial susceptibility testing (AST) systems and other diagnostic tools. Gauge capacity of LIMS to exchange data with other hospital systems, including standardisation of data exports. The ability of LIMS to interface with a facilities electronic medical record (EMR) if available, for efficient flow of information, error minimisation, and decreasing opportunities for data duplication.

Regulatory Requirements: Ensure that the chosen LIMS complies with relevant local and international regulatory requirements and data protection and privacy policies. Assessment of the ability of the LIMS to ensure strong data security measures, including user authentication, access controls, and data encryption.

Appropriateness for Microbiology: Evaluate ability of the LIMS to support international standards for antimicrobial susceptibility testing (e.g., EUCAST and CLSI), as well as support configuration of the system to use current breakpoints and interpretive criteria. Assessment of the system’s ability to generate customisable reports for microbiology results, including modification of templates for different laboratory sites to ensure they meet the needs of clinicians and laboratory staff. Assessment of the LIMS ability to handle the complex data structures inherent in microbiology results, including qualitative, semi-qualitative, and quantitative data. Ensure that the system can manage multiple organisms and their respective susceptibility profiles within a single sample.

User Satisfaction Survey

To measure end-user satisfaction, all scientists and clinicians who had used the LIMS or online result portal programs were invited to participate in a survey regarding their satisfaction with these services in Timor-Leste. A custom user satisfaction survey was developed and utilised as opposed to a standardised survey method due to the specific context and aims of the LIMS implementation. The priority was to gather practical, immediate feedback from the end-users in a resource-limited setting, where customised, context-specific questions were more suitable than standardised tools. The available validated surveys did not fully address the intricacies of the specific laboratory processes or implementation needs specific to the LMIC context, necessitating a more tailored approach to capture the unique challenges and objectives of the system. The full user satisfaction survey utilised is available in the Supplementary Data.

Conducted in March 2022, one year after the implementation of the system across all departments, the survey was designed in Qualtrics (SAP Software Solutions, Provo, UT) and contained 23 questions for scientists and 25 questions for clinical respondents (see Supplementary Data). Survey links were distributed via an encrypted messaging application (WhatsApp) to approximately 230 clinicians and laboratory staff and responses collected digitally. Informed consent was obtained from all participants. The survey was conducted anonymously; however, participants had the option to provide their contact details if they wished to receive the evaluation outcome and additional feedback. The survey was translated into Tetum, an official language of Timor-Leste, and checked for question appropriateness and saliency by Tetum speakers before distribution. The survey questions were divided into the following themes: Demographics, Service Outcomes, Communication, Sustainability and Usability for analysis. These were recorded and translated into English where required. Free text data were managed using NVivo software, version 12 (QSR International, Melbourne, Australia). Quantitative analysis of multiple-choice questions was performed, presenting the proportion of participants who answered with strongly disagree, disagree, neutral, agree or strongly agree. Free text answers have been presented as quotes where relevant.

Results

LIMS Self-Assessment

Effective LIMS implementation in LMIC such as Timor-Leste face numerous challenges, all of which must be addressed for a sustainable system. Prior to implementation, 16 different LIMS were considered, and the chosen system (SchuyLab CGM) was determined to fulfil the most essential requirements. The self-assessment checklist was utilised to assess the appropriateness of the LIMS implemented in Timor-Leste and a summary of responses is provided below.

Finance: In Timor-Leste, the upfront costs associated with LIMS implementation as well as a prepayment of maintenance fees were funded by the UK aid Fleming Fund Country Grant to Timor-Leste. Ongoing costs that will be the responsibility of the Timor-Leste MoH including maintenance and support fees, estimated hardware replacement or repair costs, internet fees and human resources have been clearly outlined with relevant stakeholders and a gradual funding transition plan established. Although a free, open-source LIMS would be preferable for long-term sustainability and eliminate the need for ongoing government budget allocation, the ongoing costs associated with the system were much lower than that of other commercial LIMS and it was decided the ongoing costs were within the ability of the government to maintain.

Infrastructure: Adequate infrastructure including a reliable power supply with backup generator and internet connectivity is essential. In Timor-Leste, high-cost and poor internet connectivity speeds have been extremely challenging for a service that is now utilised by seven laboratories nation-wide [17]. Internet connectivity in Timor-Leste is unreliable, with average speeds of 4.7 Mbps ranked 174th globally and frequent outages [18]. This issue has not yet been completely resolved, however a fibre optic connection to a government-provided internet service is being conducted with the aim of significantly increasing available bandwidth for the services and reducing funding pressure on the laboratories. In late 2023 a high-capacity generator (300kVA) was installed at the NHL site, ensuring a consistent supply of electricity.

Human Resources: In Timor-Leste, ongoing customisation performed by identified ‘super users’ in each department is being developed but will require extensive training and monitoring to mitigate issues related to staff changeover. For scientists, ongoing in-country mentoring has been utilised to increase capacity for advanced microbiology diagnostics and development of country-specific guidelines for AST interpretation and reporting.

Sustainability: There is currently limited capacity within the laboratory for advanced IT support, so additional support with a remote branch of the commercial LIMS provider based in Malaysia has been established. This allows support in the commonly spoken language of Malay, in a time zone similar to that of Timor-Leste. In future, in-country technical support will be provided by the previously identified ‘super users’ as well as the IT department within the MoH. The planned fibre optic network will ensure ongoing connectivity free of charge to LIMS sites, contributing to sustainability of the service.

System-wide Integration: System-wide integration is considered essential for the sustainability of the LIMS implementation, and currently all analysers utilised in the laboratory, including those used for diagnostic microbiology, are interfaced to allow for senior scientist review of results prior to release and reduce the potential for transcription errors. The ability of the LIMS to interface with and report on results from all laboratory departments is integral, with real-time access to comprehensive patient results aiding clinical decision-making. The chosen LIMS has the capacity to integrate with an EMR when one is implemented in Timor-Leste.

Regulatory Requirements: Timor-Leste currently has no regulatory requirements or enforced data management policies in place for laboratory data, so care was taken to ensure appropriate data management practices that align with international standards. The server where patient laboratory data are stored is located at the laboratory site and under the control of the Timor-Leste MoH. Data are accessible to relevant government programs, and IT support is provided by the Fleming Fund Country Grantees with an aim to hand over full control to MoH prior to completion of the grant.

Appropriate for Microbiology: The NHL currently uses a combination of both CLSI and EUCAST breakpoints due to the varying availability of sheep and horse blood for agar production in the country. The implemented LIMS has the ability to set custom breakpoints for specific organisms, however lacks the ability to combine both minimum inhibitory concentration (MIC) and disc diffusion data within one set of breakpoints. This issue has been overcome with interfacing of the LIMS with the BD Epicentre middleware, which is able to transfer MIC results and antimicrobial interpretations to the LIMS. Disc diffusion results can be entered to the system directly and interpreted automatically. The system is able to generate clear reports that have been customised to display each facility’s details and can support data exports in CSV format to inform national surveillance programs and annual antibiogram production.

The results of the self-assessment checklist are outlined in Table 1. The chosen LIMS was ranked on a scale of 1–5 for each category, with 1 meaning the system does not meet the requirements of that category, and 5 representing a system that fully meets or exceeds the requirements. All categories were weighted equally.

Table 1 Self assessment checklist results

In summary, prior to LIMS implementation laboratory results were handwritten or printed directly from individual analysers, connectivity between departments and specimen workflow was limited, and very little standardisation for result reporting was in place. The LIMS implementation has resolved a number of these issues, and the NHL is now producing consistent, high quality data that has the ability to positively influence patient outcomes and inform policy. Future steps will be to ensure ongoing sustainability of the service after complete handover to management by the MoH.

End-user Satisfaction Survey

Of the approximately 230 clinicians and scientists who received links to the digital survey, 34 (14.7%) people submitted responses – 19 laboratory scientists and 15 doctors. All participants surveyed were aware of the new LIMS and online results portal. Free-text responses under the themes of Communication, Sustainability, Usability were provided by 58% (n = 11) of scientist and 40% (n = 6) of clinical respondents.

Scientist responses

All scientist respondents (n = 13) said they used the LIMS daily in their work. 31% stated they had intermediate computer skills (n = 4), and could work independently, while 69% stated they were experienced and able to assist others (n = 9). Of the respondents, 85% were comfortable using the software and were confident in their ability to help others (n = 11). When asked if they found use of the software easy, and if they felt they had received enough training in its use, 85% of scientists responded positively (n = 11). One statement provided by a scientist was “The LIMS system works very well and is based on modern technology, and makes our work function correctly (since) changing the system from manual into a modern system.”

Most scientist respondents (77%, n = 10) thought that implementation of the LIMS had increased the value of the pathology service to the hospital (Table 2). All scientist participants thought it was important for patient care (n = 13), and that it had improved the workflow in their department (n = 13). One response stated, “Implementation of the LIMS system is excellent because it makes the work between scientists’, nurses and clinicians easier through easier access of results.”

Table 2 Response to questions relating to Service Outcomes – Scientists

When presented with options relating to what stopped them from using the LIMS, the most common response was that of slow internet speed (54%, n = 7). One respondent stated “Personally, I think using LIMS is best. My suggestion is, if possible, to fix/repair the internet in NHL so that if we provide patient results it does not take too long.” Other respondents stated that they wished the LIMS program was available in Tetum or Bahasa Indonesia (n = 2) rather than English, or that they are still using the LIMS even though they find difficulty in using it (n = 2).

When asked for what challenges they foresaw the LIMS services facing in future, a common response (33%, n = 3) was the importance of ongoing funding allocation. One respondent stated “Money. If no more fund then the contract is cut off and back to manual life.” Another stated “Something to do with expiration. No more funding then it might stop working like previous LIMS.”

Clinician responses

Sixty-seven percent of clinicians responded that they use the online results portal every day (n = 6), and the remaining respondents weekly (n = 3). When requested to rate their computer skills, the majority of respondents (56%, n = 5) stated they were experienced and able to assist others, while 44% stated they had intermediate computer skills and could work independently (n = 4). One respondent stated “I'd say that reports are consistent, and results are easy to find. In my case I always use the patient's hospital number, or their date of birth and I can find all the test results.”

Of the respondents, 56% felt confident in the use of the online results portal (n = 5), while the remaining participants required some assistance when using it (n = 4). When asked if they found the online results portal easy to use, 87% (n = 7) said yes. Fifty percent of clinicians surveyed (n = 4) felt they had received enough training in use of the portal, while 25% (n = 2) thought they had not.

Not all participants gave responses to all questions. Of those who responded, all (n = 6) thought the online response portal increased the value of the pathology service to the hospital, thought it was important for patient care (n = 7), and that it had improved the workflow in their department (n = 6) (Table 3). When presented with options that related to what stopped them from using the portal, the most common response was that of slow internet speed (n = 4). One respondent stated, “Sometimes there are some technical issues but it [the LIMS] really helps us to know the results so we can indicate quickly and provide treatment to the patients more carefully.”, while another stated “With the internet issues that we encountered at some point the system can be very slow or difficult to access due to internet connection.” Three respondents thought they did not have enough time to use the portal.

Table 3 Response to questions relating to Service Outcomes – Clinicians

Discussion

Although the challenges associated with LIMS implementation in LMIC are substantial, the mitigation of barriers is possible and should be a focus area for both government and non-governmental organisations involved in diagnostic laboratory strengthening. In Timor-Leste the LIMS has been implemented successfully, however careful management by stakeholders and ongoing commitment to continuation of the service is essential for sustainability. There are currently very limited examples in the literature regarding effective implementation strategies, and although context-specific modification would be required of any general strategy, many of the barriers and enablers we encountered would be consistent across LMIC sites [12]. Awareness of the importance of strong informatics for collection of high-quality data and ongoing investment is required from both national government and partners for AMR surveillance.

The self-assessment of the LIMS implementation demonstrated the ability of the chosen provider to provide a historical database of laboratory results that are easily accessible across the sites, replacing the previous need for paper notebooks that were frequently damaged and difficult to access. Urgent results can be easily accessed from any LIMS or online results portal site. Data extraction and analytics have also been greatly improved by the LIMS implementation, making it much easier to quickly determine sample number throughput for each department and turnaround time. In time, it is hoped that these data can aid stock forecasting and reduce wastage or excess consumables in Timor-Leste.

From the scientist and clinician survey respondents, we observed an overall positive response to the implementation and usability of both the LIMS and online results portal. Although users described finding some difficulties, most have managed to find ways to work around these issues and found the LIMS of benefit to their work and practice. Allowing for free-text answers to some question stems provided useful information in how the services could be improved for end-users in future upgrades. Positive comments relating to the services included the professional presentation of pathology reports as a result of the LIMS implementation, replacing the need for handwritten reports. Concerns raised by participants largely centred around unstable internet connectivity and the sustainability of the services once funding responsibility is transferred to the government in Timor-Leste.

For sustainability of the LIMS and online result portal services, it is integral that NHL and MoH take ownership and allocate sufficient annual budget for maintenance and improvement of service and infrastructure. Sufficiently qualified IT staff need to be employed to continually monitor and repair any infrastructure or communication issues, and sufficient bandwidth ensured to maintain the services.

Limitations

A limitation of the study was the absence of a pre-implementation survey, which restricted the ability to directly compare baseline metrics with post-implementation outcomes, and potentially affected the evaluation of the LIMS system's impact. Only a small number of participants completed the survey, indicating the potential for a selection bias towards respondents who either strongly like or strongly dislike the services. Increasing the sample size by including other referral hospitals in Timor-Leste which now also utilise the services, as well as allowing more time in questionnaire distribution and collection could provide a better response rate and improve insights into the impact of the LIMS on the provision of health care and diagnostic services. In addition, a more thorough qualitative analysis of the user's subjectivity when utilising the services could add more depth. Future research including a combination of interviews and quantitative analysis could provide richer data on individual users’ perceptions and experiences of the system, and understanding on enablers and barriers to optimise effective utilisation.

Conclusion

The LIMS implemented in Timor-Leste is well suited to the LMIC context, satisfying most requirements determined in a self-assessment checklist. Surveyed LIMS users in Timor-Leste have a positive outlook toward the new services, aligning with increased laboratory throughput suggesting buy-in from clinical and laboratory stakeholders. The survey variables used (service outcomes, communication, sustainability and usability) have allowed for preliminary examination of the experience of the LIMS and online results portal implementation for both laboratory staff and clinicians. The outcome of this survey has helped to form the basis of a framework for continuous quality improvement and ongoing modifications to improve user/system interactions.

The LIMS implemented in Timor-Leste is now well-established and able to handle complex microbiology data. However, the landscape of LIMS options appropriate for LMIC contexts is still limited, and a truly fit-for-purpose LIMS for use in LMIC that meets the complex requirements for diagnostic microbiology without the need for extensive backend modifications is not yet available. The maintenance of a LIMS service in LMIC is complex, with a need for significant and ongoing support in both development and implementation, and there are limited resources available to assist implementation. The self-assessment checklist and end-user survey we have described may be of use to others in LMIC settings considering LIMS implementation.