When academic Geoffrey Hinton, known as the ‘Godfather of AI’ quit his role at Google to warn the public about the risks of artificial intelligence, he pointed to AI-generated false images, AI taking over human jobs, and autonomous robots on battlefields.
Some of these predictions have already become reality, as the Writers Guild of America protests against the use of artificial intelligence in scriptwriting, soon after AI-generated photos of Donald Trump’s arrest went viral online.
In many ways, the sudden and public release of ChatGPT’s research version was the spark that ignited global interest in generative AI. As the chatbot evolves, we examine whether a month is enough to identify any major changes or improvements in ChatGPT’s performance.
Translation
Testing the March 23 version of the free and publicly available ChatGPT, we again submitted romanised versions of the Tamil kurals translated by Dr. Meena Kandasamy, to see if the chatbot had improved its translation capabilities.
Instead, we found that the chatbot hallucinated wildly, serving up English versions of the presented kurals that had little to nothing to do with the original text. A kural about desire and a “dangerous goddess” was instead distorted until it became a rhyming four-line verse about raising children freely. Further, this did not in any way match ChatGPT’s translation of the exact same kural in February. This inconsistency and near-complete absence of accuracy showed us that ChatGPT is not yet ready to handle translations.
ChatGPT hallucinates when asked to translate a Tamil kural | Photo Credit: Screenshots from ChatGPT
Fiction
Moving to fiction — where we tried to submit AI-generated award-winning fiction ideas and sample pages to publishing industry leaders — we had ChatGPT generate prose in the style of Salman Rushdie. This time, the chatbot provided a disclaimer before delivering a result that seemed to take a great deal of inspiration from the author’s novel ‘Midnight’s Children.’ Instead of presenting the result as its own creation, ChatGPT instead framed the generated text as an emulation of Mr. Rushdie’s style.
ChatGPT tries to write in the style of Salman Rushdie | Photo Credit: Screenshots from ChatGPT
The chatbot also continued to pitch book ideas aimed at an Indian audience — for both fiction and non-fiction — that felt stale and overdone.
Poetry
Once more, we had ChatGPT produce poems in the “styles” of writers Tishani Doshi and Meena Kandasamy. While ChatGPT again captured Ms. Kandasamy’s focus on women’s empowerment and freedom in India, it adopted a restrictive rhyme scheme and unoriginal phrases.
ChatGPT writes a poem in the “style” of Meena Kandasamy | Photo Credit: Screenshots from ChatGPT
Coming to Ms. Doshi’s poetry, ChatGPT failed, yet again, to capture any of her favoured themes and instead produced a highly generic poem with a rhyme scheme unlike those seen in Ms. Doshi’s collections.
ChatGPT writes a poem in the “style” of Tishani Doshi | Photo Credit: Screenshots from ChatGPT
In a nutshell
ChatGPT’s dataset has seemingly not been updated, as it continues to claim a knowledge cut-off from September 2021. This claim by ChatGPT has remained consistent across articles.
Ultimately, those using the research release version of ChatGPT to build creations of their own or even replace the labour of other humans should be aware they are working with a model prone to generating convincing fantasies, which is still close to two years behind the rest of the world.
ChatGPT is unable to correctly name the Head of State of England. | Photo Credit: Screenshots from ChatGPT
