Photo Credit: Artur Plawgo

The following is a summary of “Clustering of clinical symptoms using large language models reveals low diagnostic specificity of proposed alternatives to consensus mast cell activation syndrome criteria,” published in the September 2024 issue of Allergy and Immunology by Solomon et al.

The diagnosis of mast cell activation syndrome (MCAS) has seen a notable increase since its initial characterization as a mastocytosis-like disorder. While the consortium MCAS criteria are well-established and widely endorsed, this rise in diagnoses occurs amidst a growing array of proposed alternative MCAS criteria. Effective diagnostic criteria are crucial to minimize the risk of misclassifying unrelated conditions as MCAS. This study evaluated whether symptoms associated with alternative MCAS criteria lead to less precise or consistent diagnostic outcomes, thereby reducing diagnostic specificity. To address this, the researchers employed multiple advanced language models, including ChatGPT, Claude, and Gemini, to estimate the probabilities of diagnoses aligning with consortium and alternative MCAS criteria. 

The study group applied diversity and network analysis methods to assess the precision and specificity of these diagnoses compared to control criteria such as systemic lupus erythematosus (SLE), Kawasaki disease, and migraines. The analysis revealed that alternative MCAS criteria were linked to diagnoses with greater variability (Shannon diversity of 5.8 versus 4.6 for consortium criteria; p=0.004) and reduced precision (mean Bray-Curtis similarity of 0.07 versus 0.19 for consortium criteria; p=0.004). Additionally, the diagnostic networks derived from alternative MCAS criteria showed lower similarity than those derived from distinct SLE criteria (cosine similarity of 0.55 versus 0.86; p=0.0022). 

These findings indicate that alternative MCAS criteria are associated with a broader and less consistent range of diagnoses than consortium criteria. This lack of specificity, particularly when compared to control criteria, suggests that alternative MCAS criteria may contribute to an increased risk of MCAS overdiagnosis, potentially leading to the exclusion of more accurate diagnoses.

Source: sciencedirect.com/science/article/abs/pii/S0091674924009448