BMC Medical Informatics and Decision Making

Table 3 Retrieval performance of each method alone using a set of sampled Gold Standard cases

From: Large-scale identification of social and behavioral determinants of health from clinical notes: comparison of Latent Semantic Indexing and Generative Pretrained Transformer (GPT) models

		Precision				Recall				F1
SBDH Category	Sampled N (P)	ICD-9	LSI	GPT-3.5	GPT-4	ICD-9	LSI	GPT-3.5	GPT-4	ICD-9	LSI	GPT-3.5	GPT-4
Housing insecurity	80 (53)	0.85	0.95	0.78	0.92	0.64	0.72	0.47	0.62	0.73	0.82	0.59	0.74
Tobacco use	80 (56)	0.95	0.93	0.89	0.88	0.68	0.66	0.43	0.93	0.79	0.77	0.58	0.90
Opiate abuse	80 (36)	0.75	0.63	0.75	0.67	0.83	0.69	0.42	0.83	0.79	0.66	0.54	0.74
Alcohol abuse	80 (52)	0.85	0.95	0.84	0.82	0.65	0.73	0.40	0.90	0.74	0.83	0.55	0.86
Cocaine abuse	80 (43)	0.78	0.80	0.90	0.95	0.72	0.74	0.42	0.81	0.75	0.77	0.57	0.88
Physical & sexual abuse	67 (37)	0.96	0.67	0.88	1.00	0.70	0.49	0.38	0.73	0.81	0.56	0.53	0.84
Unemployed	54 (36)	1.00	1.00	0.85	0.91	0.42	0.81	0.31	0.89	0.59	0.89	0.45	0.90
Legal circumstances	53 (26)	1.00	0.72	0.67	0.78	0.50	0.69	0.38	0.69	0.67	0.71	0.49	0.73
Financial circumstances	46 (18)	1.00	0.61	1.00	0.75	0.33	0.78	0.44	0.50	0.50	0.68	0.62	0.60

The bold text indicate the highest precision, recall, or F1 for each SBDH category (row)

Back to article page

ISSN: 1472-6947

Contact us

General enquiries: journalsubmissions@springernature.com