Advancing AI-driven thematic analysis in qualitative research: a comparative study of nine generative models on Cutaneous Leishmaniasis data

Table 5 The Jaccard index values of the models used, alone or combined, in the qualitative analysis compared with reference A using python3.13.0

Model(s)	Jaccard (A, X1_X2)	Jaccard (A, X3_X4)	Jaccard (A, X1_X2_X3_X4)	Shared sub-themes ∣A∩X1_X2_X3_X4∣	Single sub-themes ∣A∩X1_X2_X3_X4∣	The formula for calculating the Jaccard index for four qualitative syntheses of the same model J (A, X)
B: LlaMA 3.1	0.67	0.63	0.79	19	24	19 / (24 + 19–19)
C: NotebookLM	0.54	0.54	0.63	15	24	15 / (24 + 15–15)
D: Gemini1.5 Adv Ultra	0.58	0.71	0.75	18	24	18 / (24 + 18–18)
E: Claude 3.5 Sonnet	0.50	0.83	0.83	20	24	20 / (24 + 20–20)
F: Chat GPTo1 PRO	0.96	1.00	1.00	24	24	24 / (24 + 24–24)
G: Chat GPTo1	0.87	0.96	1.00	24	24	24 / (24 + 24–24)
H: Grok V2	0.92	0.96	1.00	24	24	24 / (24 + 24–24)
K: DeepSeek V3	0.83	1.00	1.00	24	24	24 / (24 + 24–24)
M: Gemini2.0 Advanced	0.87	0.92	0.92	22	24	22 / (24 + 22–22)

‘X’ can be replaced by the letter B, C, D, E, F, G, H, K, or M. Knowing that B represents the LlaMA 3.1 model, C represents the NotebookLM model, D represents the Gemini1.5 Advanced Ultra model, E represents the Claude 3.5 Sonnet model, F represents the Chat GPTo1 PRO model, G represents the Chat GPTo1 model, H represents the GrokV2 model, K represents the DeepSeekV3 model, and M represents the Gemini2.0 Advanced model. The calculation formula used is as follows J(A, X)=∣A∩X∣/(∣A∣+∣X∣-∣A∩X∣)

ISSN: 1472-6947