Reliability of AI Chatbots in Categorising Foods by Oxalate Content

Abstract

Aim: AI chatbots have shown promise in food classification tasks, but their accuracy in categorising foods based on specific nutritional content, such as oxalates, has not been thoroughly evaluated in the Turkish language. This study assesses the performance of three AI chatbots—ChatGPT 4.0, Gemini, and Microsoft Copilot—in classifying foods according to their oxalate content. Materials and Methods: A dataset of 63 diverse food items, including commonly consumed Turkish foods, was used to evaluate the chatbots’ accuracy across five oxalate categories: little or none, low, moderate, high, and very high. The performance of each model was analysed, and commonly correct and incorrect classifications were identified. Results: ChatGPT 4.0 demonstrated the highest overall accuracy (69.8%), significantly outperforming Gemini (36.5%) and Microsoft Copilot (26.9%). Foods such as spinach and cocoa were consistently classified correctly, while foods like carrot and walnut were commonly misclassified. Statistical analysis using Cochran’s Q test revealed significant differences in accuracy among the chatbots (p-value <0.05). Conclusion: This study highlights the potential of AI chatbots in dietary management, particularly in supporting clinicians who recommend low oxalate diets for patients with conditions such as hyperoxaluria or kidney disease stones. However, it emphasises the need for further refinement to improve accuracy, especially in classifying foods with regional variations or complex compositions commonly encountered in clinical settings. © 2025, Turk Nefroloji Diyaliz ve Transplantasyon Hemsireleri Dernegi. All rights reserved.