A hybrid framework with large language models for rare disease phenotyping

Table 1 Overall model performance for rare disease identification

	Size	F1	Precision	Recall
SemEHR		0.4866	0.3415	0.8458
Zero-Shot
+AlpaCare	7B	0.5083	0.4083	0.6732
+BioMistral	7B	0.5219	0.4151	0.7028
+OpenBioLLM	8B	0.5568	0.4444	0.7453
+Mistral	7B	0.5743	0.4949	0.6841
+LLaMA3	8B	0.6834	0.6023	0.7897
+Phi3-mini	3.8B	0.6921	0.6197	0.7836
Few-Shot
+AlpaCare	7B	0.5448	0.4839	0.6232
+Mistral	7B	0.5891	0.5497	0.6346
+BioMistral	7B	0.5896	0.5110	0.6968
+OpenBioLLM	8B	0.6726	0.6908	0.6553
+Phi3-mini	3.8B	0.7484	0.6900	0.8176
+LLaMA3	8B	0.7492	0.7071	0.7966
KAG
+AlpaCare	7B	0.5006	0.3964	0.6791
+BioMistral	7B	0.5296	0.4222	0.7103
+OpenBioLLM	8B	0.5547	0.4426	0.7427
+Mistral	7B	0.5647	0.4794	0.6871
+LLaMA3	8B	0.6830	0.6011	0.7907
+Phi3-mini	3.8B	0.6966	0.6221	0.7913

ISSN: 1472-6947