Al Jazeera Journalism Review

Outside image

Using Open-Source Intelligence (OSINT) in Journalism

Where once journalists relied on sources for information - also known as ‘human intelligence’ (HUMINT) - they now increasingly rely on ‘open-source’ intelligence (OSINT) gathered from the internet, satellite imagery, corporate databases and much, much more 

 

Many of us know the scene in All the President’s Men, Hollywood’s interpretation of the Watergate scandal, when “Deep Throat” is standing in a car park basement in Washington DC. The man we now know to be the former Associate Director of the FBI, Mark Felt, was the secret informant who gave clues such as “follow the money” to the Washington Post journalist, Bob Woodward. 

Finding the evidence that brought down US President Richard Nixon was a watershed event in investigative journalism and has rightly entered its folklore. To break investigative stories in the 1970s, you needed to develop sources. 

The skill sets needed for success were once described by the late Nick Tomalin as “ratlike cunning, a plausible manner, and a little literary ability”. Tomalin was killed while reporting the Arab-Israeli war in 1973, a year after Woodward broke the Watergate story. Investigative journalists were usually required to obtain confidential documents from people who did not hand them over without a great deal of persuasion. 

The investigator had to nurture sources over time. They’d met in restaurants, coffee shops or late at night in bars. They persuaded whistle-blowers to do the right thing; they gained their trust so that the identity of a source would not be revealed. Managing a source was a critical skill for an investigator. 

osint
A historical marker stands outside the parking garage underneath the Oakhill Office Building in Rosslyn, Virginia, where Washington Post reporter Bob Woodward would meet his source - known as 'Deep Throat' - to exchange notes about the Watergate scandal [Chip Somodevilla/Getty Images]

HUMINT - or human intelligence - was the cornerstone of investigative journalism and obtaining information that no one else had was essential for an exclusive. A journalist usually carried only a notebook and a tape recorder. When I started in journalism, there were no mobile phones. There was little methodology to investigative journalism. Success depended on who you knew and how effectively you exploited them. Then along came the computer and everything changed. Technology altered journalism, both its practice and its consumption.

The roots of OSINT lie in Computer Assisted Research (CAR), which began by exploring and analysing databases. In doing this we can discover patterns, trends and anomalies that may be useful in producing new information. The practical use of this methodology emerged with the Freedom of Information Act in the United States, which was introduced in the 1960s to open the workings of government to public scrutiny. 

Philip Meyer, a pioneer of CAR, called it “precision journalism”. It was inspired by the methodology of social sciences where a journalist used evidence to prove his assertion. A methodology was born that inspired a distinct storytelling style that distinguishes investigative from conventional journalism. Conventional journalism is reactive and observational. It describes the world as it is seen, usually after something has happened. 

Investigative journalism, by contrast, seeks to prove that some aspect of what the public thinks it knows about the world is wrong. Like a policeman or prosecutor, an investigator will discover a lead or obtain prima facie evidence that supports a hypothesis that “X is lying” or “X is corrupt”. The investigation will aim to prove this supposition. If it can’t, the investigation is dropped. This investigative methodology, known as hypothesis-based narrative, replaced conventional character-based or travelogue storytelling. Evidence gathering became the glue that holds the narrative together. 

osint
Reporter Nicholas Tomalin is pictured 10 minutes before he was killed by a bomb on the Golan Heights on October 17, 1973 during the Yom Kippur war [Fred Ihrt/LightRocket via Getty Images]

Decades ago, Philip Meyer made the prophetic statement: “When information was scarce, most of our efforts were devoted to hunting and gathering. Now that information is abundant, processing is more important.” In the last decade, open-source intelligence (OSINT) has emerged as a journalistic science, as the vast resource of data collected from social networks and internet-connected devices is mined for information beyond just databases.

The volume of data created, captured and consumed globally is projected to be around 200 billion gigabytes a year in 2025 (up from 70 billion in 2020). Every minute on Facebook, around half a million comments are posted and 150,000 photos are uploaded. More than four million hours of content is uploaded to YouTube every day. Add to that, 700 million tweets per day Investigative journalism will increasingly rely on tapping these sources. 

We are not discovering truths that are strictly hidden from us - they are not confidential - but we are assembling information in a fashion that reveals new truths. We are unpicking the resources available online to tell the story behind the picture, the story that the metadata provides, or the story that shipping or flight data tells us about an event. 

For filmmakers dealing with investigative content, there are new challenges. There will be more use of computer-generated imagery to tell the story and less use of video. There will be a need to harmonise different sources, such as vertical aspect ratio imagery, publicly generated and low-definition content with professional standards. 

osint

Graphic designers, data scientists and filmmakers will need to work together in ways that presently rarely exist. The model of television production needs to adapt to a new method of storytelling. With the amount of information in the public domain, investigative journalists of the future may be less concerned with obtaining secret data than finding ways to make sense of public data, and tell stories based on that. More complex computer-based tools, such as data mining programmes, geographic information systems, demographic databases and so forth can be used to identify patterns, anomalies and discrepancies in data. 

Much of the new technology surrounding open source intelligence will involve machine learning, that is when a computer model is trained to analyse data much faster than a human being. In effect, you train a computer to do the hunting for you. It means that investigative journalists no longer need to only learn how to write and turn on a tape recorder or camera. They will need to learn the tools of the internet. 

While computer scientists will write the programmes, journalists will need to understand the science of OSINT. OSINT is not a substitute but a complement for HUMINT. For most investigations, journalists need to use human sources as well as data. 

Investigative journalists still need “ratlike cunning, a plausible manner, and a little literary ability”. But they also need to understand how to get value from the abundance of information on the Internet. 

The Al Jazeera Media Institute Open-Source Investigation Handbook provides an invaluable guide to achieve this. 

 

More Articles

A Year of Genocide and Bias: Western Media's Whitewashing of Israel's Ongoing War on Gaza

Major Western media outlets continue to prove that they are a party in the war of narratives, siding with the Israeli occupation. The article explains how these major Western media outlets are still refining their techniques of bias in favor of the occupation, even a year after the genocide in Palestine.

Mohammad Zeidan
Mohammad Zeidan Published on: 9 Oct, 2024
Monitoring of Journalistic Malpractices in Gaza Coverage

On this page, the editorial team of the Al Jazeera Journalism Review will collect news published by media institutions about the current war on Gaza that involves disinformation, bias, or professional journalistic standards and its code of ethics.

A picture of the Al Jazeera Media Institute's logo, on a white background.
Al Jazeera Journalism Review Published on: 24 Sep, 2024
Testimonies of the First Witness of the Sabra & Shatila Massacre

The Sabra and Shatila massacre in 1982 saw over 3,000 unarmed Palestinian refugees brutally killed by Phalangist militias under the facilitation of Israeli forces. As the first journalist to enter the camps, Japanese journalist Ryuichi Hirokawa provides a harrowing first-hand account of the atrocity amid a media blackout. His testimony highlights the power of bearing witness to a war crime and contrasts the past Israeli public outcry with today’s silence over the ongoing genocide in Gaza.

Mei Shigenobu مي شيغينوبو
Mei Shigenobu Published on: 18 Sep, 2024
Journalist Mothers in Gaza: Living the Ordeal Twice

Being a journalist, particularly a female journalist covering the genocide in Palestine without any form of protection, makes practicing journalism nearly impossible. When the journalist is also a mother haunted by the fear of losing her children, working in the field becomes an immense sacrifice.

Amani Shninu
Amani Shninu Published on: 15 Sep, 2024
Anonymous Sources in the New York Times... Covering the War with One Eye

The use of anonymous sources in journalism is considered, within professional and ethical standards, a “last option” for journalists. However, analysis of New York Times data reveals a persistent pattern in the use of “anonymity” to support specific narratives, especially Israeli narratives.

Mohammad Zeidan
Mohammad Zeidan Published on: 8 Sep, 2024
Cameroonian Journalists at the Center of Fighting Illegal Fishing

While the EU’s red card to Cameroon has undeniably tarnished its image, it has paradoxically unlocked the potential of Cameroonian journalists and ignited a movement poised to reshape the future. Through this shared struggle, journalists, scientists, conservationists, storytellers, and government officials have united, paving the way for a new era of ocean advocacy.

Shuimo Trust Dohyee
Shuimo Trust Dohyee Published on: 21 Aug, 2024
The Gaza Journalist and the "Heart and Mind" Struggle

Inside the heart of a Palestinian journalist living in Gaza, there are two personas: one is a human who wants to protect his own life and that of his family, and the other is a journalist committed to safeguarding the lives of the people by holding on to the truth and staying in the field. Between these two extremes, or what journalist Maram Hamid describes as the struggle between the heart and the mind, the Palestinian journalist continues to share a narrative that the occupation intended to keep "away from the camera."

Maram
Maram Humaid Published on: 18 Aug, 2024
Journalists Recount the Final Moments of Ismail Al-Ghoul

Journalists remembering the slain reporter of Al Jazeera in Northern Gaza, Ismail Al Ghoul. "He insisted on continuing his coverage from the northern part of the Gaza Strip, despite the challenges and obstacles he faced. He was arrested and interrogated by the Israeli army, his brother was killed in an Israeli airstrike, and his father passed away during treatment abroad."

Mohammad Abu Don
Mohammad Abu Don Published on: 11 Aug, 2024
Analysis: Media Disinformation and UK Far-Right Riots

Analysis on the impact of media disinformation on public opinion, particularly during UK riots incited by far-right groups. A look at how sensationalist media can directly influence audience behavior, as per the Hypodermic Needle Theory, leading to normalized discrimination and violence. The need for responsible journalism is emphasized to prevent such harmful effects.

Anam Hussain
Anam Hussain Published on: 8 Aug, 2024
Challenges for Female Journalists in Crisis Zones of Cameroon

Testimonies of what female journalists in Cameroon are facing and how they are challenging these difficulties.

Akem
Akem Nkwain Published on: 30 Jul, 2024
From TV Screens to YouTube: The Rise of Exiled Journalists in Pakistan

Pakistani journalists are leveraging YouTube to overcome censorship, connecting with global audiences, and redefining independent reporting in their homeland.

Anam Hussain
Anam Hussain Published on: 28 Jul, 2024
Daughters of Data: African Female Journalists Using Data to Reveal Hidden Truths

A growing network of African women journalists, data scientists, and tech experts is amplifying female voices and highlighting underreported stories across the continent by producing data-driven projects and leveraging digital technologies in storytelling.

Nalova Akua
Nalova Akua Published on: 23 Jul, 2024
Are Podcasts the Future of African Broadcasting?

The surge of podcasts across Africa is a burgeoning trend, encompassing a wide array of themes and subjects, and swiftly expanding across various nations.

Derick Matsengarwodzi
Derick Matsengarwodzi Published on: 11 Jul, 2024
Video Volunteers: How India’s Marginalised Groups Tell Their Own Stories

Video creators like Rohini Pawar and Shabnam Begum have transcended societal challenges by producing influential videos with Video Volunteers, highlighting social issues within marginalized communities. Their work exemplifies the transformative power of storytelling in fostering grassroots change and empowerment across India.

Hanan Zaffa
Hanan Zaffar, Jyoti Thakur Published on: 3 Jul, 2024
Climate Journalism in Vietnam's Censored Landscape

In Vietnam, climate journalists face challenges due to censorship and restrictions on press freedom, making it difficult to report environmental issues accurately. Despite these obstacles, there are still journalists working to cover climate stories creatively and effectively, highlighting the importance of climate journalism in addressing environmental concerns.

AJR Contributor Published on: 26 Jun, 2024
Challenges of Investigating Subculture Stories in Japan as a Foreign Correspondent

Japan's vibrant subcultures and feminist activists challenge the reductive narratives often portrayed in Western media. To understand this dynamic society authentically, journalists must approach their reporting with patience, commitment, and empathy, shedding preconceptions and engaging deeply with the nuances of Japanese culture.

Johann Fleuri
Johann Fleuri Published on: 24 Jun, 2024
Covering the War on Gaza: As a Journalist, Mother, and Displaced Person

What takes precedence: feeding a hungry child or providing professional coverage of a genocidal war? Journalist Marah Al Wadiya shares her story of balancing motherhood, displacement, psychological turmoil, and the relentless struggle to find safety in an unsafe region.

Marah Al Wadiya
Marah Al Wadiya Published on: 29 May, 2024
Fighting Misinformation and Disinformation to Foster Social Governance in Africa

Experts in Africa are using various digital media tools to raise awareness and combat the increasing usage of misinformation and disinformation to manipulate social governance.

Derick Matsengarwodzi
Derick Matsengarwodzi Published on: 22 May, 2024
"I Am Still Alive!": The Resilient Voices of Gaza's Journalists

The Israeli occupation has escalated from targeting journalists to intimidating and killing their families. Hisham Zaqqout, Al Jazeera's correspondent in Gaza talks about his experience covering the war and the delicate balance between family obligations and professional duty.

Hisham Zakkout Published on: 15 May, 2024
Under Fire: The Perilous Reality for Journalists in Gaza's War Zone

Journalists lack safety equipment and legal protection, highlighting the challenges faced by journalists in Gaza. While Israel denies responsibility for targeting journalists, the lack of international intervention leaves journalists in Gaza exposed to daily danger.

Linda Shalash
Linda Shalash Published on: 9 May, 2024
Elections and Misinformation – India Case Study

Realities are hidden behind memes and political satire in the battle for truth in the digital age. Explore how misinformation is influencing political decisions and impacting first-time voters, especially in India's 2024 elections, and how journalists fact-check and address fake news, revealing the true impact of misinformation and AI-generated content.

Safina
Safina Nabi Published on: 30 Apr, 2024
Amid Increasing Pressure, Journalists in India Practice More Self-Censorship

In a country where nearly 970 million people are participating in a crucial general election, the state of journalism in India is under scrutiny. Journalists face harassment, self-censorship, and attacks, especially under the current Modi-led government. Mainstream media also practices self-censorship to avoid repercussions. The future of journalism in India appears uncertain, but hope lies in the resilience of independent media outlets.

Hanan Zaffa
Hanan Zaffar, Jyoti Thakur Published on: 25 Apr, 2024
The Privilege and Burden of Conflict Reporting in Nigeria: Navigating the Emotional Toll

The internal struggle and moral dilemmas faced by a conflict reporter, as they grapple with the overwhelming nature of the tragedies they witness and the sense of helplessness in the face of such immense suffering. It ultimately underscores the vital role of conflict journalism in preserving historical memory and giving a voice to the voiceless.

Hauwa Shaffii Nuhu
Hauwa Shaffii Nuhu Published on: 17 Apr, 2024
Journalism in chains in Cameroon

Investigative journalists in Cameroon sometimes use treacherous means to navigate the numerous challenges that hamper the practice of their profession: the absence of the Freedom of Information Act, the criminalisation of press offenses, and the scare of the overly-broad anti-terrorism law.

Nalova Akua
Nalova Akua Published on: 12 Apr, 2024