Volume 9, Issue 2 (Journal of Clinical and Basic Research (JCBR) 2025)
jcbr 2025, 9(2): 28-33 |
Back to browse issues page
Download citation:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
Rakhshi R, Khosravi T, Rahimzadeh A, Shahraki F, Oladnabi M. Can AI illustrate science? A comparative benchmarking study of text-to-image artificial intelligence models for scientific communication. jcbr 2025; 9 (2) :28-33
URL: http://jcbr.goums.ac.ir/article-1-515-en.html
URL: http://jcbr.goums.ac.ir/article-1-515-en.html
1- Student Research Committee, Golestan University of Medical Sciences, Gorgan, Iran
2- Gorgan Congenital Malformations Research Center, Golestan University of Medical Sciences, Gorgan, Iran; Ischemic Disorders Research Center, Golestan University of Medical Sciences, Gorgan, Iran ,oladnabidozin@yahoo.com
2- Gorgan Congenital Malformations Research Center, Golestan University of Medical Sciences, Gorgan, Iran; Ischemic Disorders Research Center, Golestan University of Medical Sciences, Gorgan, Iran ,
Abstract: (302 Views)
Background: This study investigates the capability of artificial intelligence (AI) to effectively illustrate scientific concepts for communication and educational purposes. It aims to evaluate the performance of three leading text-to-image Generative Artificial Intelligence (GenAI) models-Midjourney, DALL-E, and Stable Diffusion-in generating scientifically accurate visuals.
Methods: To assess the models, we employed a benchmarking approach, generating 120 images based on scientifically informed prompts. Each model's output was analyzed for aesthetics, core scientific concept representation, contextual relevance, and factual accuracy.
Results: The evaluation revealed that while GenAI models excelled in aesthetic quality, achieving a score of 90.83%, their success in capturing core scientific concepts was moderate at 48.30%. More concerning were the significant limitations in contextual relevance (9.17%) and factual accuracy, which scored a troubling 0%.
Conclusion: These findings underscore the current deficiencies of GenAI in producing effective educational illustrations. They highlight the urgent need for targeted training using domain-specific scientific datasets to enhance the precision of generated visual aids. Although the potential for AI in scientific communication is promising, substantial advancements are required to ensure both factual and contextual accuracy, facilitating a clearer understanding of complex concepts.
Methods: To assess the models, we employed a benchmarking approach, generating 120 images based on scientifically informed prompts. Each model's output was analyzed for aesthetics, core scientific concept representation, contextual relevance, and factual accuracy.
Results: The evaluation revealed that while GenAI models excelled in aesthetic quality, achieving a score of 90.83%, their success in capturing core scientific concepts was moderate at 48.30%. More concerning were the significant limitations in contextual relevance (9.17%) and factual accuracy, which scored a troubling 0%.
Conclusion: These findings underscore the current deficiencies of GenAI in producing effective educational illustrations. They highlight the urgent need for targeted training using domain-specific scientific datasets to enhance the precision of generated visual aids. Although the potential for AI in scientific communication is promising, substantial advancements are required to ensure both factual and contextual accuracy, facilitating a clearer understanding of complex concepts.
Article Type: Research |
Subject:
Education Management
References
1. Ertel W. Machine Learning and Data Mining. Introduction to artificial intelligence: Springer; 2018;175–243. [View at Publisher] [DOI] [Google Scholar]
2. Martinez R. Artificial intelligence: Distinguishing between types & definitions. Nevada Law Journal. 2019;19(3):9. [View at Publisher] [Google Scholar]
3. Mukhamediev RI, Popova Y, Kuchin Y, Zaitseva E, Kalimoldayev A, Symagulov A, et al. Review of Artificial Intelligence and Machine Learning Technologies: Classification, Restrictions, Opportunities and Challenges. Mathematics. 2022;10(15):2552. [View at Publisher] [DOI] [Google Scholar]
4. Michalski RS, Carbonell JG, Mitchell TM. Machine learning: An artificial intelligence approach: Springer Science and Business Media; 2013. [View at Publisher] [Google Scholar]
5. Goldberg Y. Neural network methods for natural language processing: Springer Nature; 2022. [View at Publisher] [Google Scholar]
6. Baidoo-Anu D, Ansah LO. Education in the era of generative artificial intelligence (AI): Understanding the potential benefits of ChatGPT in promoting teaching and learning. Journal of AI. 2023;7(1):52-62. [View at Publisher] [DOI] [Google Scholar]
7. Noy S, Zhang W. Experimental evidence on the productivity effects of generative artificial intelligence. Science.2023;381(6654):187-92. [View at Publisher] [DOI] [Google Scholar]
8. Dwivedi YK, Kshetri N, Hughes L, Slade EL, Jeyaraj A, Kar AK, et al. "So what if ChatGPT wrote it?" Multidisciplinary perspectives on opportunities, challenges and implications of generative conversational AI for research, practice and policy. International Journal of Information Management. 2023;71:102642. [View at Publisher] [DOI] [Google Scholar]
9. Brynjolfsson E, Li D, Raymond LR. Generative AI at work. National Bureau of Economic Research; 2023. [View at Publisher] [DOI] [Google Scholar]
10. Enjellina ,Beyan EVP, Rossy AGC. A Review of AI Image Generator: Influences, Challenges, and Future Prospects for Architectural Field. Journal of Artificial Intelligence in Architecture. 2023;2(1):53-65. [View at Publisher] [DOI] [Google Scholar]
11. Khosravi T, Al Sudani ZM, Oladnabi M. To what extent does ChatGPT understand genetics? Innovations in Education and Teaching International. 2023:1320-9. [View at Publisher] [DOI] [Google Scholar]
12. Richards C. Technical and scientific illustration. Information design: Research and practice. 2017:85-106. [View at Publisher] [Google Scholar]
13. Kim H-Y. Statistical notes for clinical researchers: Chi-squared test and Fisher's exact test. Restor Dent Endod. 2017;42(2):152-5. [View at Publisher] [DOI] [Google Scholar]
14. Gaddis G. Advanced biostatistics: Chi‐square, ANOVA, regression, and multiple regression. Doing Research in Emergency and Acute Care: Making Order Out of Chaos. 2015:213-22. [View at Publisher] [DOI] [Google Scholar]
15. Kara S. Prospective Visual Arts Teachers' Innovation Skills and Attitudes towards Computer Assisted Instruction. International Journal of Technology in Education and Science. 2020;4(2):98-107. [View at Publisher] [DOI] [Google Scholar]
16. Serafini F. Reading the visual: An introduction to teaching multimodal literacy: Teachers College Press; 2014. [View at Publisher] [Google Scholar]
17. Hafeez M. Systematic review on modern learning approaches, critical thinking skills and students learning outcomes. Indonesian Journal Of Educational Research and Review. 2021;4(1):167-78. [View at Publisher] [DOI] [Google Scholar]
18. Amano T, González-Varo JP, Sutherland WJ. Languages are still a major barrier to global science. PLoS biology. 2016;14(12):e2000933. [View at Publisher] [DOI] [PMID] [Google Scholar]
19. Amano T, Rios Rojas C, Boum II Y, Calvo M, Misra BB. Ten tips for overcoming language barriers in science. Nat Hum Behav. 2021;5(9):1119-22. [View at Publisher] [DOI] [PMID] [Google Scholar]
20. Wang M, Wu B, Kinshuk, Chen N-S, Spector JM. Connecting problem-solving and knowledge-construction processes in a visualization-based learning environment. Comput Educ. 2013;68:293-306. [View at Publisher] [DOI] [Google Scholar]
21. Zhang K, Aslan AB. AI technologies for education: Recent research & future directions. Comput Educ.: Artif Intell. 2021;2:100025. [View at Publisher] [DOI] [Google Scholar]
22. Lim WM, Gunasekara A, Pallant JL, Pallant JI, Pechenkina E. Generative AI and the future of education: Ragnarök or reformation? A paradoxical perspective from management educators. The International Journal of Management Education. 2023;21(2):100790. [View at Publisher] [DOI] [Google Scholar]
23. Liang P. Semi-supervised learning for natural language: Massachusetts Institute of Technology; 2005. [View at Publisher] [Google Scholar]
24. Radford A, Narasimhan K, Salimans T, Sutskever I. Improving language understanding by generative pre-training. 2018. [View at Publisher] [Google Scholar]
25. Selwyn N. The future of AI and education: Some cautionary notes. Eur J Educ. 2022;57(4):620-31. [View at Publisher] [DOI] [Google Scholar]
26. Bender EM, Gebru T, McMillan-Major A, Shmitchell S. On the dangers of stochastic parrots: Can language models be too big? Proceedings of the 2021 ACM conference on fairness, accountability, and transparency; 2021;610-23. [View at Publisher] [DOI] [Google Scholar]
27. Obermeyer Z, Powers B, Vogeli C, Mullainathan S. Dissecting racial bias in an algorithm used to manage the health of populations. Science. 2019;366(6464):447-53. [View at Publisher] [DOI] [PMID] [Google Scholar]
28. Floridi L, Chiriatti M. GPT-3: Its nature, scope, limits, and consequences. Minds Mach. 2020;30(4):681-94. [View at Publisher] [DOI] [Google Scholar]
Send email to the article author
| Rights and permissions | |
 |
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License. |
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0).