A Case Study on the Diminishing Popularity of Encoder-Only Architectures in Machine Learning Models

ba60f6118d1e33cbb5-1f3a 20240320063841363 beie:beie director@blueeyesintelligence.org WEB-FORM International Journal of Innovative Technology and Exploring Engineering IJITEE 22783075 10.35940/ijitee https://www.ijitee.org/ 03 30 2024 13 4 A Case Study on the Diminishing Popularity of Encoder-Only Architectures in Machine Learning Models Department of Data Science, Northeastern University, San Jose, California, United States of America (USA). Praveen Kumar Sridhar https://orcid.org/0009-0001-6486-1614 Nitin Srinivasan https://orcid.org/0009-0001-6472-4441 Department of Computer Science, Google, Sunnyvale, California, United States of America (USA). Adithyan Arun Kumar https://orcid.org/0000-0001-9790-2657 Department of Information Security, Salesforce, San Jose, California, United States of America (USA). Gowthamaraj Rajendran https://orcid.org/0000-0001-9422-3907 Department of Information Security, Splunk, San Jose, California, United States of America (USA). Kishore Kumar Perumalsamy https://orcid.org/0009-0009-5661-7644 Department of Computer Science, Adobe, San Jose, California, United States of America (USA). This paper examines the shift from encoder-only to decoder and encoder-decoder models in machine learning, highlighting the decline in popularity of encoder-only architectures. It explores the reasons behind this trend, such as the advancements in decoder models that offer superior generative capabilities, flexibility across various domains, and enhancements in unsupervised learning techniques. The study also discusses the role of prompting techniques in simplifying model architectures and enhancing model versatility. By analyzing the evolution, applications, and shifting preferences within the research community and industry, this paper aims to provide insights into the changing landscape of machine learning model architectures. 03 30 2024 22 27 CC BY-NC-ND 4.0 10.35940/BEIESP.CrossMarkPolicy www.ijitee.org true International Journal of Innovative Technology and Exploring Engineering (IJITEE) No, I did not receive. No conflicts of interest to the best of our knowledge. No, the article does not require ethical approval and consent to participate with evidence. Not relevant. All authors have equal participation in this article. 10.35940/ijitee.D9827.13040324 https://www.ijitee.org/portfolio-item/D982713040324/