--- title: "Gen AI is losing it's edge: what can you do about it?" description: After a period of improvement, LLMs (large language models) perform worse. Here's why you need to use domain knowledge to guard against harm that causes. image: https://www.missiondrive.io/hubfs/DALL%C2%B7E%202024-04-16%2014.55.30%20-%20A%20conceptual%20art%20piece%20depicting%20a%20large%20language%20model%20as%20a%20massive%2c%20intricate%20machine%20with%20tangled%20wires%20and%20confusing%20controls.%20In%20the%20foreground%2c%20.png --- [Skip to content](https://www.missiondrive.io/blogs/you-need-domain-knowledge-to-know-if-gen-ai-is-performing-badly#main-content) Close - [Services](https://www.missiondrive.io/services) - [Hubspot](https://www.missiondrive.io/services/hubspot) - [Inbound Marketing](https://www.missiondrive.io/services/inbound-digital-marketing) - [Digital Sales](https://www.missiondrive.io/services/digital-sales) - [Creative Content](https://www.missiondrive.io/services/creative-content) - [Ethical Digital Marketing](https://www.missiondrive.io/services/ethical-digital-marketing) - [Data Training & Consultancy](https://www.missiondrive.io/services/data-training-and-consultancy) - [Contact](https://www.missiondrive.io/contact-us) - [About](https://www.missiondrive.io/about-mission-drive) - [Resources](https://www.missiondrive.io/resources) SEARCH [![logo](https://www.missiondrive.io/hs-fs/hubfs/LOGOTYPE_positive-100-2.jpg?width=1039&height=427&name=LOGOTYPE_positive-100-2.jpg "logo")](https://missiondrive.io/)[![Mission Drive](https://www.missiondrive.io/hs-fs/hubfs/logo.png?width=125&height=51&name=logo.png)](https://missiondrive.io/) - [Services](https://www.missiondrive.io/services) - [Hubspot](https://www.missiondrive.io/services/hubspot) - [Inbound Marketing](https://www.missiondrive.io/services/inbound-digital-marketing) - [Digital Sales](https://www.missiondrive.io/services/digital-sales) - [Creative Content](https://www.missiondrive.io/services/creative-content) - [Ethical Digital Marketing](https://www.missiondrive.io/services/ethical-digital-marketing) - [Data Training & Consultancy](https://www.missiondrive.io/services/data-training-and-consultancy) - [Contact](https://www.missiondrive.io/contact-us) - [About](https://www.missiondrive.io/about-mission-drive) - [Resources](https://www.missiondrive.io/resources) Quick results for "{search\_term}" - Menu Item 1 - Sub-menu Item 1 - Another Item - Sub-menu Item 2 - Menu Item 2 - Yet Another Item - Menu Item 3 - Menu Item 4 - Menu Item 1 - Sub-menu Item 1 - Another Item - Sub-menu Item 2 - Menu Item 2 - Yet Another Item - Menu Item 3 - Menu Item 4 - Menu Item 1 - Sub-menu Item 1 - Another Item - Sub-menu Item 2 - Menu Item 2 - Yet Another Item - Menu Item 3 - Menu Item 4 - Menu Item 1 - Sub-menu Item 1 - Another Item - Sub-menu Item 2 - Menu Item 2 - Yet Another Item - Menu Item 3 - Menu Item 4 - Menu Item 1 - Sub-menu Item 1 - Another Item - Sub-menu Item 2 - Menu Item 2 - Yet Another Item - Menu Item 3 - Menu Item 4 - Menu Item 1 - Sub-menu Item 1 - Another Item - Sub-menu Item 2 - Menu Item 2 - Yet Another Item - Menu Item 3 - Menu Item 4 ![shutterstock_558507064-595946-edited](https://www.missiondrive.io/hs-fs/hubfs/shutterstock_558507064-595946-edited.jpg?width=300&name=shutterstock_558507064-595946-edited.jpg) ###### An inexpensive, quick and easy way to build beautiful responsive website pages without coding knowledge [LEARN MORE](https://www.missiondrive.io/guides-and-tools) HubSpot Our certified experts help you use HubSpot to power growth with automation, data insights and AI. [Read more](https://www.missiondrive.io/services/hubspot) Ethical Digital Marketing Join our community of digital marketers learning how to use their values as their marketing superpower. [Read more](https://www.missiondrive.io/services/ethical-digital-marketing) Inbound Marketing Campaigns, SEO, social media and content that builds your audience and creates qualified leads. [Read more](https://www.missiondrive.io/services/inbound-digital-marketing) Digital Sales AI tools, automation and data-informed insights for better prospecting and lead nurturing. [Read more](https://www.missiondrive.io/services/digital-sales) Creative Content Web pages, blogs, graphic design, webinars and videos that educate and inspire your audience. [Read more](https://www.missiondrive.io/services/creative-content) Account based marketing Win work faster, from high-value customers with marketing tactics tailored for sales. [Read more](https://www.missiondrive.io/services/account-based-marketing) ![The Beginners guide to generating Inbound Leads - menu-1](https://www.missiondrive.io/hs-fs/hubfs/The%20Beginners%20guide%20to%20generating%20Inbound%20Leads%20-%20menu-1.png?width=200&name=The%20Beginners%20guide%20to%20generating%20Inbound%20Leads%20-%20menu-1.png) ###### Want to learn the secrets of lead gen? Download our free guide now.  [LEARN MORE](https://www.missiondrive.io/guides-and-tools/generating-inbound-leads-ebook) ###### Looking for Data training and consultancy? [Read more](https://www.missiondrive.io/services/data-training-and-consultancy) ![KWS_0414](https://www.missiondrive.io/hs-fs/hubfs/Imported_Blog_Media/KWS_0414.jpg?width=300&name=KWS_0414.jpg) About us We are a specialist data literacy & digital marketing consultancy. [Read more](https://www.missiondrive.io/about-mission-drive) --- - [Our story](https://www.missiondrive.io/about-mission-drive#Our-Story) - [Our team](https://www.missiondrive.io/about-mission-drive#The-Team) - [Our clients](https://www.missiondrive.io/about-mission-drive/clients-and-testimonials#Our-Clients) - [Testimonials](https://www.missiondrive.io/about-mission-drive/clients-and-testimonials#Testimonials) - [Social Impact](https://www.missiondrive.io/about-mission-drive/social-impact) - [Careers](https://www.missiondrive.io/about-mission-drive/careers) Blogs Discover quick tips, innovative insights, and fresh ideas to inspire your mission. [Read now](https://www.missiondrive.io/blogs) Events & Webinars Join us to expand your knowledge and forge valuable connections. [Watch now](https://www.missiondrive.io/events-and-webinars) Guides & Tools Free resources that help you put responsible marketing into action. [Download now](https://www.missiondrive.io/guides-and-tools) ###### Keep up to date with all of our latest blogs, events, webinars and new content [Subscribe](https://www.missiondrive.io/en/blogs#Newsletter-sign-up) ![](https://www.missiondrive.io/hs-fs/hubfs/DALL%C2%B7E%202024-04-16%2014.55.30%20-%20A%20conceptual%20art%20piece%20depicting%20a%20large%20language%20model%20as%20a%20massive%2c%20intricate%20machine%20with%20tangled%20wires%20and%20confusing%20controls.%20In%20the%20foreground%2c%20.png?width=300&name=DALL%C2%B7E%202024-04-16%2014.55.30%20-%20A%20conceptual%20art%20piece%20depicting%20a%20large%20language%20model%20as%20a%20massive,%20intricate%20machine%20with%20tangled%20wires%20and%20confusing%20controls.%20In%20the%20foreground,%20.png) [Simon Bullmore](https://www.missiondrive.io/blogs/author/simon-bullmore)17/4/2024 41:11 AM2 min read # Gen AI is losing it's edge: what can you do about it? [Previous](https://www.missiondrive.io/blogs/little-missions/why-you-need-to-know-your-topic-before-ai) [Next](https://www.missiondrive.io/blogs/you-need-domain-knowledge-to-know-if-gen-ai-is-performing-badly-0) In the realm of generative artificial intelligence, tools like ChatGPT have transformed how we conduct research, proofread content, and edit documents. However, my recent experiences suggest that the outputs from these systems have become inconsistent, unreliable, and generally more mediocre. I'm not alone in this observation. ## AI models are prone to deteriorating performance Recent research spearheaded by prominent institutions, including Stanford University, UC Berkeley, Princeton University, and teams at Google, has shed light on a troubling trend: after a period of notable advancements, the performance of large language models (LLMs)—the powerhouse behind generative AI—might actually start to deteriorate. This phenomenon has been particularly noticeable with ChatGPT, likely because its widespread popularity invites increased scrutiny. It remains uncertain whether its competitors, such as Google's Gemini and Anthropic's Claude, will undergo similar performance declines. I won't go into the technical reasons for declining LLM performance here. For a deeper dive into the scientific insights on this issue, I recommend checking out the latest publications from these research institutions and the resource list below.  For people keen to adopt or continue using Gen AI the potential decline in performance raises crucial questions about the sustainability and future improvements of these AI systems. What can you do? ## Domain expertise to the rescue, again This issue underscores a fundamental truth about using AI or any data-driven technology: the importance of domain and sector experience. To use AI tools "appropriately"—that is, in ways that optimise your tasks while minimising potential harms—a thorough understanding of both the technology and its application context is essential. Historically, the effective use of data techniques—be it in data science, analytics, or artificial intelligence—has always required a nuanced understanding of the relevant domain. Without this expertise, even the most advanced tools can lead to suboptimal outcomes or unintended consequences. The bottom line? If you're not a domain expert, *always* engage one when using an AI tool. Or, tread very carefully... For example, we encourage businesses that use Gen AI to help with content and copy research to ensure that domain experts are part of the editorial process so that toxic or false information doesn't accidentally seep into copy and cause reputational harm. For those looking to use AI effectively, consider exploring resources that offer best practices in AI implementation across various industries. [MIT’s Technology Review](https://www.technologyreview.com/) provides excellent insights into how businesses are integrating AI into their operations safely and effectively. Finally, here's a list of research and articles on this topic: - Chen and Zou (Stanford) and Zaharia's (Berkeley) research paper: [https://arxiv.org/pdf/2307.09009.pdf](https://arxiv.org/pdf/2307.09009.pdf) - [Changmao Li](https://arxiv.org/search/cs?searchtype=author&query=Li,+C), [Jeffrey Flanigan (UC) on](https://arxiv.org/search/cs?searchtype=author&query=Flanigan,+J) LLMs struggle with new data - [https://arxiv.org/abs/2312.16337](https://arxiv.org/abs/2312.16337). - A general discussion on the topic from TechTarget- [https://www.techtarget.com/whatis/feature/Model-collapse-explained-How-synthetic-training-data-breaks-AI](https://www.techtarget.com/whatis/feature/Model-collapse-explained-How-synthetic-training-data-breaks-AI). - A discussion of the Stanford and Berkely research in Search Engine Journal- [https://www.searchenginejournal.com/chatgpt-quality-worsened/492145/](https://www.searchenginejournal.com/chatgpt-quality-worsened/492145/) Love you. Bye!   --- Found this Little Missions interesting? [Subscribe](https://www.missiondrive.io/blogs#Newsletter-sign-up) to get Little Missions delivered straight to your inbox.   ![avatar](https://app.hubspot.com/settings/avatar/a02d4d979773954384fd0540d5ec91c7) #### [Simon Bullmore](https://www.missiondrive.io/blogs/author/simon-bullmore) Simon helps our clients develop effective growth strategies and data literacy programmes. With a background in business psychology, Simon has worked in data, business development and training for over 17 years. This includes leading the learning programme at the Tim Berners-Lee founded Open Data Institute, and the launch of Harvard Business School's first European office. [mailto:simonb@missiondrive.io](mailto:simonb@missiondrive.io) ## RELATED ARTICLES ![widescreen background_whiteongrey ](https://www.missiondrive.io/hs-fs/hubfs/widescreen%20background_whiteongrey%20.png?width=300&name=widescreen%20background_whiteongrey%20.png) ![Mission Drive Logo - white](https://www.missiondrive.io/hs-fs/hubfs/Mission%20Drive%20Logo%20-%20white.png?width=100&height=41&name=Mission%20Drive%20Logo%20-%20white.png) Helping business leaders, innovation teams, L&D & HR engage effectively with their audiences about data. London | Brighton | UK [![Hubspot solutions partner](https://3064814.fs1.hubspotusercontent-na1.net/hub/3064814/hubfs/Hubspot%20solutions%20partner.png?width=100&name=Hubspot%20solutions%20partner.png)](https://app.hubspot.com/portal-recommend/l?slug=ecosystem/marketplace/solutions/mission-drive) [Services](https://www.missiondrive.io/services) --- - [Hubspot](https://www.missiondrive.io/services/hubspot) - [Inbound Marketing](https://www.missiondrive.io/services/inbound-digital-marketing) - [Digital Sales](https://www.missiondrive.io/services/digital-sales) - [Creative Content](https://www.missiondrive.io/services/creative-content) - [Ethical digital marketing](https://www.missiondrive.io/services/ethical-digital-marketing) - [Account based marketing](https://www.missiondrive.io/services/account-based-marketing) - [Data Training & Consultancy](https://www.missiondrive.io/services/data-training-and-consultancy) [![Asana Ambassador](https://3064814.fs1.hubspotusercontent-na1.net/hub/3064814/hubfs/Asana%20Ambassador.png?width=110&name=Asana%20Ambassador.png)](https://asana.com/) [Community](https://www.missiondrive.io/community) --- [![B1G1 members](https://3064814.fs1.hubspotusercontent-na1.net/hub/3064814/hubfs/B1G1-mmbr-smbl-dig_vert-white.png?width=100&name=B1G1-mmbr-smbl-dig_vert-white.png)](https://www.missiondrive.io/) [Company](https://www.missiondrive.io/about-mission-drive) --- - [About Us](https://www.missiondrive.io/about-mission-drive) - [The Team](https://www.missiondrive.io/about-mission-drive#The-Team) - [Our clients](https://www.missiondrive.io/about-mission-drive/clients-and-testimonials#Our-Clients) - [Testimonials](https://www.missiondrive.io/about-mission-drive/clients-and-testimonials#Testimonials) - [Social Impact](https://www.missiondrive.io/about-mission-drive/social-impact) - [Careers](https://www.missiondrive.io/about-mission-drive/careers) [![ODI Members](https://3064814.fs1.hubspotusercontent-na1.net/hub/3064814/hubfs/member-72-stacked-%23E5007-100px.png?width=100&name=member-72-stacked-%23E5007-100px.png)](https://www.theodi.org/service/membership-networking) [Resources](https://www.missiondrive.io/resources) --- - [Blogs](https://www.missiondrive.io/blogs) - [Events & Webinars](https://www.missiondrive.io/events-and-webinars) - [Guides & Tools](https://www.missiondrive.io/guides-and-tools) [![Silicon Brighton Members](https://3064814.fs1.hubspotusercontent-na1.net/hub/3064814/hubfs/Silicon%20Brighton%20Members.png?width=125&name=Silicon%20Brighton%20Members.png)](https://siliconbrighton.com/) [Social](https://www.missiondrive.io/contact-us#Connect) --- - [Twitter](https://twitter.com/mymissiondrive) - [LinkedIn](https://www.linkedin.com/company/missiondrive) - [Instagram](https://www.instagram.com/missiondrive/) - [Facebook](https://www.facebook.com/mymissiondrive) - [YouTube](https://www.youtube.com/@missiondrive) - [Threads](https://www.threads.net/@missiondrive) [![Data Ethics Professionals](https://3064814.fs1.hubspotusercontent-na1.net/hub/3064814/hubfs/Data-Ethics-Professional-White.webp?width=125&name=Data-Ethics-Professional-White.webp)](https://www.theodi.org/global-network-directory/data-ethics-professionals-and-facilitators/) [Connect](https://www.missiondrive.io/contact-us) --- - [Contact Us](https://www.missiondrive.io/contact-us) - [Newsletter](https://www.missiondrive.io/resources#Newsletter-sign-up) - [Accessibility Statement](https://www.missiondrive.io/about-mission-drive/accessibility-statement) - [Terms of Service](https://www.missiondrive.io/about-mission-drive/terms-of-service) - [Privacy Policy](https://www.missiondrive.io/about-mission-drive/privacy-policy) - © 2003 - 2026 ```json { "@context" : "https://schema.org", "@type" : "Organization", "address" : { "@type" : "PostalAddress", "addressCountry" : "United Kingdom", "addressLocality" : "Brighton", "addressRegion" : "Sussex", "postalCode" : "", "streetAddress" : "Mission Drive HQ" }, "knowsLanguage" : "en", "logo" : { "@type" : "ImageObject", "url" : "https://cdn2.hubspot.net/hubfs/3064814/LOGOTYPE_positive-100-2.jpg" }, "name" : "Mission Drive", "url" : "https://www.missiondrive.io/blogs/you-need-domain-knowledge-to-know-if-gen-ai-is-performing-badly" } ``` ```json { "@context" : "https://schema.org", "@type" : "BlogPosting", "abstract" : "After a period of improvement, LLMs (large language models) perform worse. Here's why you need to use domain knowledge to guard against harm that causes.", "author" : { "@type" : "Person", "name" : "Simon Bullmore" }, "dateModified" : "April 17, 2024 10 AM", "datePublished" : "April 17, 2024 10 AM", "headline" : "Gen AI is losing it's edge: what can you do about it?", "image" : "https://missiondrive.io/hubfs/DALL%C2%B7E%202024-04-16%2014.55.30%20-%20A%20conceptual%20art%20piece%20depicting%20a%20large%20language%20model%20as%20a%20massive%2c%20intricate%20machine%20with%20tangled%20wires%20and%20confusing%20controls.%20In%20the%20foreground%2c%20.png", "inLanguage" : "en", "keywords" : "[AI, Technology, Data Ethics, Little Missions, Creative content]", "mainEntityOfPage" : { "@type" : "WebPage" }, "name" : "Gen AI is losing it's edge: what can you do about it?", "publisher" : { "@type" : "Organization", "logo" : { "@type" : "ImageObject", "url" : "https://cdn2.hubspot.net/hubfs/3064814/LOGOTYPE_positive-100-2.jpg" }, "name" : "Mission Drive" }, "text" : "In the realm of generative artificial intelligence, tools like ChatGPT have transformed how we conduct research, proofread content, and edit documents. However, my recent experiences suggest that the outputs from these systems have become inconsistent, unreliable, and generally more mediocre. I'm not alone in this observation. AI models are prone to deteriorating performance Recent research spearheaded by prominent institutions, including Stanford University, UC Berkeley, Princeton University, and teams at Google, has shed light on a troubling trend: after a period of notable advancements, the performance of large language models (LLMs)—the powerhouse behind generative AI—might actually start to deteriorate. This phenomenon has been particularly noticeable with ChatGPT, likely because its widespread popularity invites increased scrutiny. It remains uncertain whether its competitors, such as Google's Gemini and Anthropic's Claude, will undergo similar performance declines. I won't go into the technical reasons for declining LLM performance here. For a deeper dive into the scientific insights on this issue, I recommend checking out the latest publications from these research institutions and the resource list below. For people keen to adopt or continue using Gen AI the potential decline in performance raises crucial questions about the sustainability and future improvements of these AI systems. What can you do? Domain expertise to the rescue, again This issue underscores a fundamental truth about using AI or any data-driven technology: the importance of domain and sector experience. To use AI tools \"appropriately\"—that is, in ways that optimise your tasks while minimising potential harms—a thorough understanding of both the technology and its application context is essential. Historically, the effective use of data techniques—be it in data science, analytics, or artificial intelligence—has always required a nuanced understanding of the relevant domain. Without this expertise, even the most advanced tools can lead to suboptimal outcomes or unintended consequences. The bottom line? If you're not a domain expert, always engage one when using an AI tool. Or, tread very carefully... For example, we encourage businesses that use Gen AI to help with content and copy research to ensure that domain experts are part of the editorial process so that toxic or false information doesn't accidentally seep into copy and cause reputational harm. For those looking to use AI effectively, consider exploring resources that offer best practices in AI implementation across various industries. MIT’s Technology Review provides excellent insights into how businesses are integrating AI into their operations safely and effectively. Finally, here's a list of research and articles on this topic: Chen and Zou (Stanford) and Zaharia's (Berkeley) research paper: https://arxiv.org/pdf/2307.09009.pdf Changmao Li, Jeffrey Flanigan (UC) on LLMs struggle with new data - https://arxiv.org/abs/2312.16337. A general discussion on the topic from TechTarget- https://www.techtarget.com/whatis/feature/Model-collapse-explained-How-synthetic-training-data-breaks-AI. A discussion of the Stanford and Berkely research in Search Engine Journal- https://www.searchenginejournal.com/chatgpt-quality-worsened/492145/ Love you. Bye! Found this Little Missions interesting? Subscribe to get Little Missions delivered straight to your inbox.", "url" : "https://www.missiondrive.io/blogs/you-need-domain-knowledge-to-know-if-gen-ai-is-performing-badly", "wordCount" : "742" } ``` ```json { "@context" : "https://schema.org", "@type" : "BlogPosting", "author" : { "@type" : "Person", "name" : "Simon Bullmore", "url" : "https://www.missiondrive.io/blogs/author/simon-bullmore" }, "dateModified" : "2024-04-17T10:48:11.573Z", "datePublished" : "2024-04-17T10:41:04.000Z", "headline" : "Gen AI is losing it's edge: what can you do about it?", "image" : [ "https://www.missiondrive.io/hubfs/DALL%C2%B7E%202024-04-16%2014.55.30%20-%20A%20conceptual%20art%20piece%20depicting%20a%20large%20language%20model%20as%20a%20massive%2c%20intricate%20machine%20with%20tangled%20wires%20and%20confusing%20controls.%20In%20the%20foreground%2c%20.png" ], "mainEntityOfPage" : { "@id" : "https://www.missiondrive.io/blogs/you-need-domain-knowledge-to-know-if-gen-ai-is-performing-badly", "@type" : "WebPage" }, "publisher" : { "@type" : "Organization", "logo" : { "@type" : "ImageObject", "url" : "https://www.missiondrive.io/hubfs/LOGOTYPE_positive-100-2.jpg" }, "name" : "Mission Drive Marketing LTD" } } ```