eprintid: 17885 rev_number: 8 eprint_status: archive userid: 2 dir: disk0/00/01/78/85 datestamp: 2025-12-09 23:32:23 lastmod: 2025-12-09 23:32:24 status_changed: 2025-12-09 23:32:23 type: article metadata_visibility: show creators_name: Mohsin, Muhammad creators_name: Hashmi, Muhammad Shadab Alam creators_name: Delgado Noya, Irene creators_name: Garay, Helena creators_name: Abdel Samee, Nagwan creators_name: Ashraf, Imran creators_id: creators_id: creators_id: irene.delgado@uneatlantico.es creators_id: helena.garay@uneatlantico.es creators_id: creators_id: title: Dual-modality fusion for mango disease classification using dynamic attention based ensemble of leaf & fruit images ispublished: pub subjects: uneat_sn divisions: uneatlantico_produccion_cientifica divisions: unincol_produccion_cientifica divisions: uninimx_produccion_cientifica divisions: uninipr_produccion_cientifica divisions: unic_produccion_cientifica divisions: uniromana_produccion_cientifica full_text_status: public keywords: Plant disease detection Multimodal approach Class-aware augmentation Modality attention fusion Out-of-distribution abstract: Mango is one of the most beloved fruits and plays an indispensable role in the agricultural economies of many tropical countries like Pakistan, India, and other Southeast Asian countries. Similar to other fruits, mango cultivation is also threatened by various diseases, including Anthracnose and Red Rust. Although farmers try to mitigate such situations on time, early and accurate detection of mango diseases remains challenging due to multiple factors, such as limited understanding of disease diversity, similarity in symptoms, and frequent misclassification. To avoid such instances, this study proposes a multimodal deep learning framework that leverages both leaf and fruit images to improve classification performance and generalization. Individual CNN-based pre-trained models, including ResNet-50, MobileNetV2, EfficientNet-B0, and ConvNeXt, were trained separately on curated datasets of mango leaf and fruit diseases. A novel Modality Attention Fusion (MAF) mechanism was introduced to dynamically weight and combine predictions from both modalities based on their discriminative strength, as some diseases are more prominent on leaves than on fruits, and vice versa. To address overfitting and improve generalization, a class-aware augmentation pipeline was integrated, which performs augmentation according to the specific characteristics of each class. The proposed attention-based fusion strategy significantly outperformed individual models and static fusion approaches, achieving a test accuracy of 99.08%, an F1 score of 99.03%, and a perfect ROC-AUC of 99.96% using EfficientNet-B0 as the base. To evaluate the model’s real-world applicability, an interactive web application was developed using the Django framework and evaluated through out-of-distribution (OOD) testing on diverse mango samples collected from public sources. These findings underline the importance of combining visual cues from multiple organs of plants and adapting model attention to contextual features for real-world agricultural diagnostics. date: 2025-11 publication: Scientific Reports volume: 15 number: 1 id_number: doi:10.1038/s41598-025-26052-7 refereed: TRUE issn: 2045-2322 official_url: http://doi.org/10.1038/s41598-025-26052-7 access: open language: en citation: Artículo Materias > Alimentación Universidad Europea del Atlántico > Investigación > Producción Científica Fundación Universitaria Internacional de Colombia > Investigación > Producción Científica Universidad Internacional Iberoamericana México > Investigación > Artículos y libros Universidad Internacional Iberoamericana Puerto Rico > Investigación > Producción Científica Universidad Internacional do Cuanza > Investigación > Producción Científica Universidad de La Romana > Investigación > Producción Científica Abierto Inglés Mango is one of the most beloved fruits and plays an indispensable role in the agricultural economies of many tropical countries like Pakistan, India, and other Southeast Asian countries. Similar to other fruits, mango cultivation is also threatened by various diseases, including Anthracnose and Red Rust. Although farmers try to mitigate such situations on time, early and accurate detection of mango diseases remains challenging due to multiple factors, such as limited understanding of disease diversity, similarity in symptoms, and frequent misclassification. To avoid such instances, this study proposes a multimodal deep learning framework that leverages both leaf and fruit images to improve classification performance and generalization. Individual CNN-based pre-trained models, including ResNet-50, MobileNetV2, EfficientNet-B0, and ConvNeXt, were trained separately on curated datasets of mango leaf and fruit diseases. A novel Modality Attention Fusion (MAF) mechanism was introduced to dynamically weight and combine predictions from both modalities based on their discriminative strength, as some diseases are more prominent on leaves than on fruits, and vice versa. To address overfitting and improve generalization, a class-aware augmentation pipeline was integrated, which performs augmentation according to the specific characteristics of each class. The proposed attention-based fusion strategy significantly outperformed individual models and static fusion approaches, achieving a test accuracy of 99.08%, an F1 score of 99.03%, and a perfect ROC-AUC of 99.96% using EfficientNet-B0 as the base. To evaluate the model’s real-world applicability, an interactive web application was developed using the Django framework and evaluated through out-of-distribution (OOD) testing on diverse mango samples collected from public sources. These findings underline the importance of combining visual cues from multiple organs of plants and adapting model attention to contextual features for real-world agricultural diagnostics. metadata Mohsin, Muhammad; Hashmi, Muhammad Shadab Alam; Delgado Noya, Irene; Garay, Helena; Abdel Samee, Nagwan y Ashraf, Imran mail SIN ESPECIFICAR, SIN ESPECIFICAR, irene.delgado@uneatlantico.es, helena.garay@uneatlantico.es, SIN ESPECIFICAR, SIN ESPECIFICAR (2025) Dual-modality fusion for mango disease classification using dynamic attention based ensemble of leaf & fruit images. Scientific Reports, 15 (1). ISSN 2045-2322 document_url: http://repositorio.unini.edu.mx/id/eprint/17885/1/s41598-025-26052-7.pdf