Discussion on the Reliability of AI Detectors Following Allegations of AI Use in Novel
Read more
IOL
iol.co.za

Discussion on the Reliability of AI Detectors Following Allegations of AI Use in Novel

Canadian-Haitian writer, poet, and novelist Télisson Orélien found himself at the center of a scandal on September 22nd after an anonymous account on X accused him of using artificial intelligence to write the novel 'C'était ça ou mourir,' which became a hit of the autumn literary season.

These claims regarding the use of AI tools to create Orélien's bestseller sparked a wave of discussion in the French literary world and called into question the methods for detecting AI-generated content.

The scandal began with a post on X asserting that the Pangram program, designed to detect AI, concluded that the narrative about a Haitian migrant traveling to Canada was largely generated by a text generator such as Claude or ChatGPT.

Orélien, who is 38 years old, categorically denies these accusations and stated his intention to prove authorship of the book, which has been sold to publishers in over 20 countries. He recently emphasized: 'I did not write this novel with my head, but with my guts and heart.'

AFP agency tested excerpts from the book with various AI detection tools, receiving diverse results—ranging from 'very likely written by a human' to '100 percent' generated by AI.

Currently, there are numerous online tools available to identify AI-generated text, many of which are intended for educational or publishing purposes. These programs typically compare large volumes of AI-generated text with human-written material.

The performance of these tools has significantly improved in recent years. In a study published in October 2025 titled 'Artificial Writing and Automated Detection,' researchers from the University of Chicago found that Pangram achieves 'nearly zero error rates' when analyzing medium and long passages.

Thierry Poabo, a researcher from the French national research institute CNRS and a natural language processing specialist, explained that the programs search for 'linguistic features.' He specified that a classic example is the triadic construction of sentences (the use of three subordinate clauses) or dashes.

These characteristic features were previously often found in generated texts because the tools were trained on American scientific literature or American novels, noted Poabo. Pangram claims it is capable of 'detecting AI text by analyzing and understanding subtle nuances in writing.'

It is also warned that monotonous sentence structures are a red flag, and certain words or phrases appear regularly. Although many tools publish accuracy percentages, these figures cannot be verified and do not account for the risk of false positives.

Max Spero, CEO and co-founder of Pangram, told the American publication The Atlantic that the tool should not be the final judge, but merely a starting point for deeper investigation.

According to Thierry Poabo, these detection systems also face the 'moving target problem' because AI programs 'are constantly evolving.' The expert also points to the potential difficulty in identifying AI use in novels. He notes that 'the tool might catch something specific in the writing, something repetitive, but that could be exactly what constitutes the writer's style.'

Orélien reported this week that his style is based on Haitian and Caribbean traditions, telling the newspaper Liberation that the software 'is accustomed to the French literary tradition and classical authors.'

Benoît Raphaël, an entrepreneur and writer specializing in AI, conducted his own study using Pangram to check Orélien's work, and the results were also positive for AI. He informed the magazine Nouvel Obs that 'the fact that he received a score of around 100 percent AI or even 90 percent is quite indicative.'

Despite the book receiving widespread critical acclaim and selling quickly in France before the scandal began, some doubts had been raised previously. Léa Bori in her podcast Torchon might have been the first to publicly suggest at the beginning of this month that the novel was written by AI due to the repetitive nature of the sentences and the excessive use of analogies, which is a hallmark of AI. Fabrice Collen in the newspaper Canard Enchaîné also criticized the formulaic structure of the sentences, noting that he 'was fruitlessly searching for a unique voice' in the prose.

Popular