It got quite a lot right, and quite a lot very, very wrong

There was a time when battling writer's block - that frustrating mental barrier preventing effective communication and word finding - meant just pushing through it, one word at a time, or seeking inspiration from a book to fuel creativity. Even for journalists and reporters, staring at blank screens or empty notebook pages was challenging, as they struggled to find the precise vocabulary that could effectively shape their news stories. However, since the arrival of ChatGPT on November 30, 2022, an AI language model developed by San Francisco's OpenAI research lab, journalists now have a powerful tool to conquer writer's block and tackle even the most complex topics. Or do they?

Armed with extensive knowledge and the ability to generate well-structured text efficiently, ChatGPT can produce high-quality content in record time. It can craft captivating headlines, conduct data analysis and translate various languages. And it has swift access to background information, enabling the production of articles in a fraction of the time it would take a human being.

ChatGPT, a computer programme designed to simulate conversation with human users, has demonstrated a remarkable ability to answer queries in a way that echoes human conversation. As you continue interacting with the chatbot, it adapts and customises its responses, even recollecting your previous inputs. 

Curious to know how ChatGPT perceives itself, I asked it to describe itself. ChatGPT responded with a rather poetic eloquence: "As an artificial intelligence, I am a vast repository of knowledge, a linguistic virtuoso, and a digital companion for all those who seek answers. I am a digital sage, a fountain of knowledge in this modern age." 

Inspired by this remarkable display of self confidence, I decided to put it to the test.

The challenge

My usual patch for this magazine is journalism in Pakistan, so my idea was to ask ChatGPT to write an article on this topic.

Though I already had a shortlist of angles in mind, I was curious to know what ChatGPT would suggest when I asked it to come up with an article written from my own point of view as a British-Pakistani diaspora journalist.

The first thing ChatGPT did was inform me that as a British-Pakistani diaspora journalist, my perspectives are unique, shaped by a blend of cultural and social experiences, leading to nuanced and insightful reporting. It then churned out several compelling article ideas: the representation of Pakistan in Western media (but I've been there, done that), the role of Pakistani media in shaping the diaspora's perceptions of the country (already covered!), and the challenges of reporting on Pakistan from a diaspora perspective (perfectly fine, but it all seems to be a subset of the first topic).

Given the broadness of my initial request, it's perhaps understandable that ChatGPT's responses were somewhat general. So, to help it find a more specific angle, I suggested focusing on journalism in Pakistan as an overall topic, but with a distinctive approach that reflects my focus. 

This led to the generation of additional ideas: First, the challenges facing female journalists in Pakistan and how diaspora women can use their positions to advocate for better representation and opportunities for women in Pakistani media. Next, the cultural and linguistic barriers that diaspora journalists may face when trying to report on sensitive issues in Pakistan. And, finally, how the Pakistani diaspora's relationship with the government and media in their host countries (eg, the UK) might shape their views on press freedom in Pakistan. 

All of these ideas sounded impressive. As a concluding recommendation, ChatGPT advised using personal connections with journalists and media organisations in Pakistan to gain exclusive insights that foreign correspondents may not have access to, rather than relying solely on publicly available information. 

I opted for the topic of female journalists in Pakistan and requested a framework for an article on the subject. ChatGPT presented a breakdown consisting of four key points: the current gender gap in Pakistani newsrooms; the challenges female journalists face; case studies of female journalists in Pakistan; and the role of diaspora women in advocating for change. While it offered a comprehensive introduction to the subject matter, however, it lacked details and supporting sources to underpin each subheading. 

I pressed for more information and ChatGPT revealed that the hurdles faced by women in Pakistani journalism are not just limited to the workplace. The deep-seated gender biases and societal expectations of women as primary caregivers, compounded by a patriarchal culture, make it challenging for women to juggle their professional aspirations with their domestic responsibilities. Even when they do break into the industry, women struggle to secure prominent roles and assignments that truly showcase their skills and expertise. 

"Women are frequently confined to the ‘soft news’ genre, such as lifestyle and entertainment, which restricts their potential and opportunities," it added. I was a bit annoyed at ChatGPT’s honesty here, but as a woman of Pakistani origin, I also know this linguistic sage is correct in pointing out these challenging realities. In addition, it asserted that the lack of representation of women in Pakistani media limits the diversity of perspectives and voices in news coverage, which results in biased reporting and reinforces existing power structures. 

Short of time, I asked: "ChatGPT, can you please now wave your technological magic wand and design an article in 1,500 words that incorporates statistical data, real-world examples and reports?"

After apologising that, as an AI language model, it could not provide a full 1,500-word article instantly, ChatGPT spent the next 647 words reflecting on the challenges faced by female journalists in Pakistan, including the impact of those challenges, five case studies and the importance of diaspora voices with strategies for advocacy.

‘Page not found’

Upon initial inspection, the article written by ChatGPT on female journalists in Pakistan appeared to be pretty remarkable. It featured a captivating introduction, with concise sentences in an organised arrangement.

The punchy intro, "Breaking down barriers and changing mindsets: The uphill battle for female journalists in Pakistan and the power of diaspora women in paving the way towards progress," was powerful and thought-provoking.

Moreover, it shared that a report released by the International Federation of Journalists (IFJ) indicated that women constitute only 23 percent of the total journalist population in Pakistan. While this figure aligns with findings presented in a comprehensive study by UNESCO (found upon cross-checking), the specific source ChatGPT referred to could not be found.

I politely requested: "Could you please provide me with the link to the source for further examination?" Unfortunately, the HTML shared by ChatGPT appeared to be broken - “page not found”. Despite my repeated requests for alternative reports and corresponding references, every website link shared led to an error page. I was starting to doubt the brilliance of this AI.

However, ChatGPT did demonstrate particular strength when it came to shedding light on the obstacles faced by women journalists in Pakistan. It highlighted the constant pursuit of safety and security, the fight for equal pay and fair benefits, the silent burdens of mental health challenges and burnout, sexual harassment and death threats, harrowing encounters with attempted kidnappings, a lack of representation and the pressing need for diversity within newsrooms.

Alongside these significant challenges, ChatGPT provided captivating case studies that celebrated the resilience and courage of Pakistani female journalists. Yet, adding to the frustration, it once again missed the inclusion of relevant links, preventing the opportunity to fully delve into the original interviews.

Worse, one of ChatGPT's case studies, featuring the remarkable story of journalist and TV anchor, Gharidah Farooqi, known for her resilience in the face of challenges like censorship and credibility attacks, contained a fabricated quote falsely attributed to a reputable media outlet. 

The quote, which stated, "Journalists are supposed to be the watchdogs of society, but in Pakistan, the watchdogs themselves are being hunted," was initially attributed to Gharidah Farooqi in a 2016 interview with Deutsche Welle. When I inquired about the source, ChatGPT mistakenly referred to Umar Cheema, an investigative journalist from Pakistan. 

"But wait, didn't you just mention Gharidah Farooqi?" I asked, puzzled by the inconsistency. ChatGPT recognised its error - "That was my mistake" - and provided yet another incorrect attribution. This time, it mentioned the Pakistani journalist and human rights activist, Hamid Mir. Seeking clarity, I asked if it was certain this time. Apologetically, ChatGPT admitted its inability to find a definitive source or attribution for the statement. It suggested that the quote may have been paraphrased or attributed to multiple individuals in different contexts, to describe the challenges faced by journalists in Pakistan.

When asked where it got its news information from and what publishers it was reading, ChatGPT told me:

"I rely on the data and information that has been fed into my system during my training process, which includes a diverse range of sources and publications." It mentioned notable outlets such as the BBC, CNN, The New York Times, The Guardian and Reuters.

ChatGPT did acknowledge the importance of considering the limitations of its responses, saying that: "I don't have the ability to evaluate the credibility or reliability of a particular news source. It's always a good practice to read from multiple sources to get a well-rounded perspective and to critically evaluate the information being presented."

On April 6, 2023, The Guardian published an article titled: "ChatGPT is making up fake Guardian articles", showing how it had been generating false news articles using the name of the publication. The report further shed light on the potential implications of such fabricated content and raised questions about the reliability and integrity of AI-generated news. It also serves as a stark reminder of the challenges and ethical dilemmas posed by AI-driven content generation, prompting a broader discussion on the future of journalism in an era of rapidly advancing technology.

Could AI replace journalists altogether?

During my road-test of the AI, ChatGPT - knowing I'm a Pakistani diaspora journalist, based in the UK - advised me to acknowledge the cultural differences and complexities of the Pakistani media landscape and work to amplify the voices of those who are often silenced.

"As a diaspora journalist, you may have connections to media outlets and organisations in both the UK and Pakistan. You can use these connections to engage with Pakistani media outlets and organisations to advocate for greater representation and opportunities for women," it told me.

Suggesting that I use my platform to amplify the voices of female journalists in Pakistan, ChatGPT suggested that I conduct interviews with them for my articles and opinion pieces, promote their work on my social media channels, shed light on the obstacles they encounter and explore opportunities for collaboration through reporting projects or mentorship programs:

"Consider submitting your pieces to Pakistani media outlets or international publications with a focus on South Asia," it insisted. 

Additionally, it suggested I mentor and support female journalists by offering constructive feedback on their work, facilitating connections with relevant contacts and resources and providing career advice and guidance. Lastly, an exceptional piece of advice offered by ChatGPT is to consider making donations to organisations and initiatives that actively support female journalists in Pakistan. The prime example it gave was the International Women's Media Foundation (IWMF), a Washington, DC-based non-profit organisation, which operates a dedicated Pakistan reporting programme focused on providing training and assistance to women journalists across the country.

While I was impressed by ChatGPT's overall performance and capabilities, a sense of unease crept in, leading me to think about the possibility of the language model eventually replacing my role as a diaspora journalist in the coming years. It reassured me:

"Robot reporters can complement human reporters, rather than replace them, in delivering quality journalism. I am not designed to take over human jobs. My purpose is to assist and enhance human abilities and provide support in various fields, including journalism. While I am capable of generating content and conducting research, I still lack the creativity and critical thinking skills of a human journalist. Therefore, I believe that humans and AI can work together collaboratively to produce high-quality journalism that benefits society."

Although ChatGPT proved useful when it came to structuring my thoughts and delivering concise and meaningful descriptions of the selected subject, which are not only relevant but also vital for readers, it is worth noting that it was me - a human being - who instructed ChatGPT to incorporate them. As I continued to request the AI to elaborate on these ideas, it increasingly echoed the language I had initially provided in the prompts.

However, it lacked the nuance and empathy that can come from first-hand experience and personal interviews with human journalists. For instance, the emotional impact of harassment and discrimination is difficult to convey solely through data and statistics. It is important to hear the stories of those affected and provide a platform for their voices. The occasional generation of inaccurate facts also serves as a reminder that human judgement and editing remains crucial for ensuring the reliability and accuracy of data.

Despite its limitations, I see AI technology as valuable in journalism. As the media landscape evolves, journalists should embrace new techniques and technologies for information gathering. However, it's crucial to acknowledge the significance of human insight and empathy in crafting impactful stories.

After conversing with the chatbot for two hours, I’m nowhere near finalising the article. While it did provide me with some useful information, such as important themes and noteworthy case studies of Pakistani female journalists - but nothing can replace human interviews. Will its ability to provide helpful article-planning services improve significantly in the future? Most likely. But for now, traditional human-powered research remains unmatched.

Anam Hussain is a freelance diaspora journalist, born in Lahore, Pakistan and now based in the UK

The views expressed in this article are the author’s own and do not necessarily reflect Al Jazeera Journalism Review’s editorial stance