Table 1 Themes and subthemes extracted from the data

1. LLM-Enhanced Healthcare Communication

Explores ethical concerns surrounding the use of LLM in facilitating communication for healthcare purposes, including patient-provider interactions and discussions of preventive measures.

“-“The use of AI [LLMs] to enhance doctor-patient communication could improve outcomes, but we must consider how to prevent harm due to potential miscommunication or misunderstanding.”

2. LLM in Nursing and Care Quality Improvement

Focuses on the ethical aspects of LLM’s role in nursing, such as improving job performance and care quality, while considering the reduced decision-making burden on care providers.

-“I had a nursing Educator during my RN residency program describe some technology similar to this [LLMs] and that it would eventually be able to chart my shift assessment for me.”

3. Ethical Monitoring of LLM Coding in Healthcare

Discusses the need for ethical guidelines and standards in developing and coding LLM tools in healthcare.

-“For LLM coding in healthcare, robust oversight must warrant algorithms are free from biases and respect patient diversity… I fear AI optimized for efficiency could worsen issues if deployed in medical coding without a framework ensuring decisions are fair.”

4. Privacy Ethics in LLM-Enabled Medical Data

Highlights the ethical concerns related to privacy and data security in the context of LLM accessing sensitive medical lab results.

-“Effective anonymization measures are essential for using private health data to train diagnosis LLMs that protect patient confidentiality.”

5. LLM in Emergency Care: Ethical Perspectives

Examines the ethical considerations in using LLM for emergency and outpatient treatment, emphasizing patient safety and treatment efficacy.

-“These [LLM] tools might aid emergency triage, but safeguards are needed to prevent over-reliance on imperfect algorithmic assessments when immediate care is critical.”

6. Ethical Challenges in LLM-Powered Rural Healthcare

Explores ethical questions about using LLM to enhance healthcare accessibility in rural areas, with a focus on patient consent and privacy.

-“Bringing the benefits of AI healthcare LLMs to rural populations raises important questions around equitable access and accountable deployment.”

7. Ethics of LLM Education in Clinical Settings

Addresses the ethical necessity of educating clinical staff about the current applications of LLM in medical practices, including implications for medical training.

-“Incorporating LLMs into clinical education demands more scrutiny to ensure that it enhances rather than detracts from the learning experience.”

8. Ethics of User Experience in LLM Healthcare Applications

Investigates the ethical dimensions of user experience in healthcare LLM tools, focusing on the balance between user input and algorithmic output.

-“My experience so far is that AI can be a false positive machine. But I’ve only used it for LVO and PE detection. It does ok at intracranial hemorrhage.”

9. LLM Training for Mental Health: Ethical Considerations

Looks at the ethical implications of using LLM training data in supporting mental health treatments, including ADHD and other disorders.

- “Mental health LLMs require exceptionally thoughtful development and monitoring to avoid codifying outdated assumptions harmful to vulnerable groups.”

10. Ethical Aspects of LLM application in Diagnostics

Explores the ethical considerations in the LLM-driven diagnosis of health problems, focusing on accuracy, bias, and patient outcomes.

-“If applied to medical imaging diagnostics, LLMs would require extensive validation and ongoing monitoring to avoid missed or spurious diagnoses.”

11. LLM Fairness and Ethics in Healthcare

Discusses the crucial ethical issue of fairness in LLM applications within healthcare, especially in ensuring equitable treatment for all patient demographics.

-“LLMs promising to improve healthcare efficiency must not deprioritize delivering quality care equitably across patient populations.”

12. Ethical Dimensions of LLM in Public Healthcare Resources

Focuses on the ethical implications of LLM in improving the accessibility and availability of public healthcare resources.

- “Allocating LLM resources in public healthcare poses questions about prioritization and access, ensuring technology benefits the many rather than the few.”

13. Trust and Ethics in Healthcare LLM Systems

Addresses the ethical concerns related to the trustworthiness and reliability of LLM systems in healthcare settings.

- “If LLMs inserted into healthcare processes seem like black boxes, it could impede trust-building process in technology and stifle realizing potential benefits.”

14. Ethics of LLM in Enhancing clinical Workflows

Examines the ethical considerations of integrating LLM into clinical workflows, including issues related to protocol compliance and the impact on nursing practices.

“What about an AI that summarizes all the clinic visits or admissions? Or tells you about progression of disease based on several CT scans or tells you the patients hasn’t been filing meds for XYZ reasons. Throw in some risk factor calculators and you got yourself a powerful tool in diagnosis and managing your daily workflow.”