Evidence available publicly neither proves nor disproves the claim regarding use of AI in Imran Khan’s Economist article

The iVerify Pakistan team has checked this content and labelled it unproven since there is no definitive way to confirm the authorship of the essay attributed to PTI chief Imran Khan in British publication The Economist on January 4, 2024.

To arrive to this verdict, the iVerify Pakistan team has investigated the various claims about the article’s authorship from various parties.

HOW IT STARTED

A news report was published in the leading English publication Dawn on Jan 9 with the headline: “AI used to write Economist piece, claims Imran”

The report said that while talking to journalists allowed to cover court proceedings inside the prison after attending two trials at the Adiala Jail on Jan 8, Imran had “made a surprising claim, saying that an essay recently published by The Economist under his name was actually ‘AI-generated’”.

The report further said: “Confirming the contents of the essay, Mr Khan said he did not write the piece himself, rather it was based on points he had dictated, which were put into words through the use of artificial intelligence.”

It added that questioned about the article’s origin, “sources close to Mr Khan told Dawn that it contained facts mentioned by the PTI founder at different points in time, adding that the article was merely a consolidation of facts already available on social media platforms.

“The sources said that Mr Khan had shared these details with some visitors who met him in prison, and they may have confided them to someone in the magazine, who consolidated these facts into the shape of an article.”

Later in the day, claims circulated on social media from multiple journalists, quoting the Dawn print report, that a Jan 4 article by Imran in The Economist was written through the aid of artificial intelligence (AI).

Journalist and anchorperson Gharidah Farooqui’s post on X, with over 41,000 views, said: “Now that Imran Khan himself has admitted he did not write the article but AI did for him; A- It’s proven the article is in-fact ‘ghostwriting’, B- being a journalist I’d like to know what … The Economist say about publishing an article which is not the real work/writing of a political leader … AI writing is widely considered an insult by the real writers.”

She questioned whether the British publication had “set the tradition for AI writings being accepted, published and promoted?”

Meanwhile, journalist Syed Talat Hussain said the British publication – terming it as “The Conomist” – had “conned readers into thinking as if it were a genuine write-up”, adding that “all AI work is considered inauthentic and marked zero even for 7 graders.”

His post on X gained over 105,000 views.

Similarly, Geo News’ UK correspondent Murtaza Ali Shah shared a post with the following caption:

“PTI: Imran Khan wrote The Economist article himself accusing Pakistan military and US of removing him.

“Dawn: Imran Khan told journalists he wrote the article using AI.”

Meanwhile, the PTI rejected the claims with a Jan 9 clarification saying news reports about the article’s content and mode of publication did “not reflect the actual state of facts regarding the matter”.

The party said the piece was authored by Imran and “in no way … has been compiled through the use of artificial means, including AI”.

The PTI rebuttal was viewed over 220,000 times and shared 2,700 times.

METHODOLOGY

The iVerify Pakistan team sought to investigate the issue due to the competing nature of the claims about the article’s authorship, the virality of the aforementioned claims as well as the significance of Imran Khan as a former prime minister and the public’s interest in the use of AI.

Investigating the original Jan 4 article in The Economist, the essay is written in the first person and features the following description for the PTI chief at the bottom: “Imran Khan is the founder and former chairman of Pakistan Tehreek-e-Insaf and was prime minister of Pakistan from 2018 to 2022.”

A Jan 5 Dawn report on the same said the article was “attributed” to Imran.

The report added that “sources within the party were hesitant to comment on how the writing may have been relayed to the publication from inside prison, they insisted that the words were indeed those of Mr Khan.”

It also said that some observers had “expressed doubts” about Imran’s authorship of the article but “many noted that the tone and content of the article was consistent with his views”.

The report further said the article subsequently “went viral on social media”.

The Economist website does not address its editorial policy on the use of AI-generated content. The iVerify Pakistan team has reached out to The Economist‘s editorial team for clarity on the matter.

Later on Jan 5, caretaker Information Minister Murtaza Solangi said the government would be writing the publication’s editor about the article “purportedly written by Imran Khan”.

“It is puzzling and disconcerting that such an esteemed media outlet published an article in the name of an individual who is in jail and has been convicted.

“We believe it is critically essential to uphold ethical standards and promote responsible journalism. We would like to know how the editorial decision was made, and what considerations were taken into account regarding the legitimacy and credibility of the content by The Economist.

“We would also be interested to know if The Economist has ever published such ghost articles by jailed politicians ever from any other part of the world,” the minister, a former journalist himself, had said in a post with 1.5 million views.

According to a Dawn.com report on Jan 8, Imran was quizzed about the matter himself by reporters during the Jan 8 court hearing and reportedly said: “I own the column published in an international journal. I had given verbal guidelines regarding the column. The article was written and published as a result of these guidelines. I had verbally dictated the article.”

The PTI chief did, however, say that it was “the age of artificial intelligence”.

The same was corroborated by a Jan 9 Geo report.

However, the Jan 9 print report by Dawn on the same court hearing varied slightly, pointing to the inclusion of AI in the writing process.

This report in question led to the claims under investigation of the PTI chief admitting to the article being written through AI, followed by the party’s denial that it was authored by Imran himself.

Analysing the original essay through software and tools to detect AI-generated content yielded the following results:

GPTZero: “This text is most likely to be written by a human. There is a 2% probability this text was entirely written by AI.”

zerogpt: “Your text is human written” with 0.49% of the text “suspected to be most likely generated by AI”.

winston ai: “Winston has detected the text as 100% human. Our scan was unable to detect material use of AI text generation tools.”

Undetectable AI: “Your content appears human.”

BACKGROUND

On August 5, 2023, the PTI chief was sentenced to three years of imprisonment by an Islamabad trial court in a case related to state gifts. The case, filed by the Election Commission of Pakistan, had accused the PTI leader of not mentioning the details of state gifts in his tax declarations.

The Islamabad High Court set aside the sentence on Aug 29, 2023, and ordered his release. However, he was not freed as he was undergoing trial in the cipher and other cases.

The PTI chief currently remains incarcerated in Rawalpindi’s Adiala Jail.

The site served as the court for the Jan 8 hearings where Imran made the claim about the article’s origin while talking to journalists allowed to cover proceedings inside the prison.

CONCLUSION

The iVerify team has determined that the claims regarding the article’s authorship, whether it was written by Imran or through AI, remain unproven.

This is because the PTI chief himself reportedly told journalists inside the court about the origin and authorship of the piece, which were subsequently published on Dawn online and in print on Jan 8 and Jan 9, respectively.

To verify whether he said he had verbally dictated the article or it was “AI-generated” would require either an audio or video recording of the PTI chief’s Jan 8 conversation with reporters.

However, recordings from inside the courtroom are not permitted without the judge’s approval and therefore, there is no way to source back or refer to a recording of the conversation Imran had with reporters on Jan 8.

According to the court rules and etiquette from the Balochistan High Court website: “Taking audio or video recordings or photographs is not allowed in courtrooms.”

Regarding the results generated by the AI-detection tools, a June 2, 2023, Washington Post article by technology columnist Geoffrey A. Fowler says:

“Turns out, we can’t reliably detect writing from artificial intelligence programs like ChatGPT.

“Even worse, scientists increasingly say using software to accurately spot AI might simply be impossible.”

It further said that educational software company Turnitin reported its AI-cheating detector operating on “more than 38 million student essays since April has more of a reliability problem than it initially suggested”.

Similarly, London-based editing and proofreading company Oxbridge Editing said in a September 24, 2023, blog post: “The accuracy of these tools is far from ideal. The most important problem with AI detection tools is that they have high false positive rates. This means they are likely to identify human-written text as being written by AI, even if no AI was used to generate content.”

The same concern was echoed by digital rights expert Usama Khilji. Approached for comment on the reliability of AI-detection tools, he said they “have a fair chance of false positives because of the sophistication of generative AI tools that are available.”

The iVerify Pakistan team thus cannot rely on the results generated through the AI-detection tools to pass decisive declarations on a piece of content.

Therefore, uncertainty remains on the article’s authorship and not enough evidence is available to pass a definitive verdict about it until further comment on the matter from the PTI chief.