Why Policymakers Should Rethink AI’s Role in Reviving the Struggling UK Economy

Generative artificial intelligence is being rapidly integrated across various sectors, from assisting consultants in diagnosing cancer to helping teachers with lesson plans, while also saturating social media with derivative content. However, a growing chorus of critics is questioning the real benefits that this technology can bring to the UK’s sluggish economy. A major concern remains that large language models (LLMs) often fabricate information, a problematic feature known as “hallucination.”

In a recent blog post, barrister Tahir Khan highlighted instances where lawyers utilized LLMs for drafting legal filings or arguments, only to encounter fictitious Supreme Court cases, bogus regulations, and nonexistent laws embedded within the AI-generated outputs. Khan cautioned that “hallucinated legal texts” may look convincingly formatted with citations and judicial opinions, thereby misleading even seasoned legal professionals.

Similarly, broadcaster Adam Buxton, during a podcast episode, shared excerpts from a book he purchased online that claimed to compile quotes and anecdotes about his life, many of which seemed authentic but were entirely fabricated. Tech-skeptic journalist Ed Zitron emphasized in a blog post that the tendency of chatbots like ChatGPT to assert false information limits their utility for business customers, who require truthfulness in their communications.

Academics from the University of Glasgow have articulated that these models are designed to predict plausible sentences rather than reason or solve problems, advocating for the term “bullshit” instead of “hallucinations” to describe their inaccuracies. In a paper titled “ChatGPT is bullshit,” Michael Townsen Hicks and colleagues suggest that large language models primarily aim to mimic human speech, leading to inherent inaccuracies in the information they generate.

This issue is not a mere technical glitch but rather woven into the fabric of the models. A recent study published in New Scientist indicated that these inaccuracies are becoming increasingly prevalent. Even advanced AI systems labeled as “large reasoning models” experience “accuracy collapse” when tackling complex challenges, as indicated by a widely circulated paper from Apple.

Despite their limitations, LLMs still prove beneficial for various analytical tasks, but reliance on chatbots as definitive sources of information remains risky, as those in the legal field have discovered. If LLMs are primarily producing misleading information, several important implications arise.

First, this raises significant concerns regarding the extent to which AI should supplant human workers who bear ultimate accountability for their outputs. Nobel laureate Daron Acemoglu has articulated that, due to accuracy issues, generative AI will likely only replace a narrowly defined set of roles in the near future. He estimates that it may affect about 5% of the economy, primarily within roles focused on data summarization and pattern recognition. Acemoglu advocates for more research to develop AI tools that augment rather than replace human workers.

Secondly, if the benefits of AI are uneven, society should be cautious about the costs it is willing to accept, ensuring that those costs are absorbed and mitigated by the creators of these models. This includes not only massive energy consumption but also the significant repercussions for politics and democracy due to the proliferation of fabricated content. Sandra Wachter from the Oxford Internet Institute recently likened the situation to littering in nature, noting that it becomes increasingly difficult to navigate a polluted landscape.

Finally, while it is essential for governments to remain open to adopting new technologies like AI, this should occur with a clear comprehension of their capabilities and limitations, combined with a healthy skepticism towards exaggerated claims from proponents. In last week’s spending review, ministers effectively acknowledged the importance of “digitisation” alongside AI in enhancing public services. Before a significant replacement of civil servants with chatbots occurs, the UK’s citizens are seeking alternatives for communication, such as receiving information from their doctors in formats that differ from standard letters.

ChatGPT and its counterparts possess remarkable capabilities; they can synthesize extensive information and present it in diverse styles. However, as anyone familiar with charming yet deceptive individuals can attest, relying solely on such technologies to resolve all issues is unwise, and maintaining a critical perspective is crucial.