Al Jazeera Journalism Review

Outside image

What is an open-source investigation?

In the second part of our special series on using open-source intelligence in journalism, we look at what makes a good investigation

 

An open source investigation (OSINT) uses intelligence gathering techniques and technologies including satellite imagery, social media posts and user-generated content to uncover the invisible. In recent years, open source investigations have become one of journalism’s most valuable tools, largely due to its ability to tap into vast amounts of publicly available online information to reveal otherwise untold stories. 

Collecting and analysing publicly available data and information from across the internet can include anything from analysing an IP address all the way through to interrogating public governmental records. 
 

Read more:

Open-Source Investigations, Part 1: Using open-source intelligence in journalism

Al Jazeera Media Institute Open-Source Investigation Handbook

 

What is OSINT? Open source intelligence is the application of intelligence gathering techniques and technology to investigations that make use of open source data. 

Security adjunct professor at Columbia University Mark M Lowenthal defines OSINT as “any and all information that can be obtained from the overt collection: all media types, government reports, and other files, scientific research and reports, business information providers, the Internet, etc”. 

The learning process of how to use open source tools is constantly evolving. The Al Jazeera Media Institute Open-Source Investigation Handbook provides core elements and tools for journalists who are interested in conducting open-source investigations. It introduces a framework and outlines ethical approaches, while examining case studies, to analyse the fundamentals of online search and research techniques for investigations. 

Whether it involves using search engines to gather documentation, examines videos and satellite imagery to collect critical evidence, or evaluates data gathered from an online database, this handbook offers journalists the necessary skills to acquire and verify documentation.

From early conflict and environmental monitoring to high-profile investigations such as Anatomy of a Killing, using advanced open source techniques has quickly developed to become a crucial practice for journalists in both long-form investigations and breaking news. Open source techniques involve researching, selecting, archiving and analysing information from publicly available sources. 

OSINT15

An effective open source investigation begins by addressing these three questions: What do we need to know? Why do we need to know it? Who might have the information we need?

While various open source guidebooks available to investigative journalists differ on the exact process that should be followed in an open source investigation, they all agree on certain fundamentals. 

First, you must have a clear strategy and framework in place for acquiring and using open source information. This involves identifying which investigation to pursue and how to transform your findings into an engaging story. 

Second, you must identify a set of tools and techniques for collecting and processing open source information without compromising the safety of your subject matter or those involved in investigating the story. 

Third, you should develop the right strategies to validate your findings. Collaboration is an important consideration here. 

Finally, as in many parts of the world information is heavily controlled, knowing how to preserve and archive data remains an important element that can affect accountability mechanisms.

 

Benefits 

● Wide array of information to collect 

● No or low barriers to access 

● Easy-to-locate publicly available data 

 

Risks 

● Identity exposure 

● Counterattacks from online adversaries 

● Collection of misinformation

 

Debunking Myths about OSINT 

● Open Source investigation is just Googling 

● Open Source investigation is only for cybersecurity professionals 

● Open Source investigation is only for tech-savvy individuals or experts 

● Open Source investigation is surveillance and violates privacy 

 

Tips for an open source investigator 

1. YOUR SECURITY IS PARAMOUNT – Don’t forget to keep your identity hidden while searching. 

2. BE CAREFUL – You really need a good eye for small details. A successful open source investigator has sharp observational skills to detect even the slightest bit of information that might contribute to the bigger picture. 

3. PERSEVERANCE – To be a successful open source investigator, you need to stick out the seemingly never-ending process of compiling data and research.

osint

Case Study

Forced out - Measuring the scale of the conflict in South Sudan 

In 2019, Al Jazeera’s AJLabs data journalism team, in partnership with the Pulitzer Center, published an open source investigation to better understand the complexities and scale of displacement and land rights in South Sudan. 

For this story, Carolyn Thompson and Kristen van Schie worked with land rights experts and statisticians to survey more than 35,000 random phone numbers across South Sudan in order to paint an accurate picture of displacement across the world’s youngest nation which had descended into civil war. 

As of 2019, nearly 2.5 million refugees had fled to neighbouring countries. 

osint

As many journalists are denied access to or even barred from reporting within the country, Al Jazeera used a mobile phone survey to gather information from those in places traditional journalism cannot reach. That data, which included questions on demographics, displacement, destruction, and plans to return home was then verified against other reporting tools, including satellite imagery, on-the-ground interviews, UN reports, public records, photos of the destruction and testimonies from internally displaced people and refugees. 

The result was an interactive longform which included maps, videos, infographics and before-and-after sliders. 

In 2020, the Investigative Reporters and Editors (IRE) awarded the story third place in the Philip Meyer Award for “an outstanding example of a determined group of reporters using social science methods to get to the root causes of a refugee crisis, even with severely limited press freedom, possible government interference, and a scared population.”

This article has been adapted from the Al Jazeera Media Institute’s Open-Source Investigation Handbook

 

More Articles

Censorship, Militarisation, and Dismantlement: How Public Media Became a Political Battlefield in Latin America

Public media in Latin America, such as Brazil's EBC and Argentina's Télam, are being undermined through militarisation and dismantlement, threatening their role as public institutions. These actions jeopardise media independence and weaken their ability to serve the public interest, posing a serious risk to democracy.

Rita Freire Published on: 19 Dec, 2024
Independent Syrian Journalism: From Revolution to Assad's Fall

Independent Syrian journalism played a pivotal role in exposing regime corruption and documenting war crimes during the 13-year revolution, despite immense risks to journalists, including imprisonment, assassination, and exile. Operating from abroad, these journalists pioneered investigative and open-source reporting, preserving evidence, and shaping narratives that challenged the Assad regime's propaganda.

Ahmad Haj Hamdo
Ahmad Haj Hamdo Published on: 17 Dec, 2024
Journalists and the Gen–Z protest in Kenya

Caught between enraged protesters and aggressive police officers, journalists risked their lives to keep the world informed about the Gen–Z protests in Kenya. However, these demonstrations also exposed deeper issues regarding press freedom, highlighting a troubling aspect of Ruto’s government.

Shuimo Trust Dohyee
Shuimo Trust Dohyee Published on: 12 Dec, 2024
Behind the Burka: Journalism and Survival Under Taliban Rule

An account of a female Afghan journalist who persisted in her work in spite of the Taliban's comeback, using her writing to expose the harsh realities of oppression and promote women's rights. In defiance of the Taliban government's prohibitions on female education, she oversaw underground schools for girls and reported under a pseudonym while constantly fearing for her safety.

Khadija Haidary
Khadija Haidary Published on: 8 Dec, 2024
Fact or Fiction? Quantifying the 'Truth' in True-Crime Podcasts

Over the centuries, true crime narratives have migrated across mediums—from tabloids and books to documentaries, films, and, most recently, podcasts. Despite these evolutions, one constant endures: the storytellers’ drive to detail the darkest corners of human behaviour and the insatiable curiosity of their audiences.

Suvrat Arora
Suvrat Arora Published on: 28 Nov, 2024
Why Are Journalists Being Silenced in Kashmir?

Since the revocation of Article 370 in 2019, press freedom in Indian-administered Kashmir has sharply declined, with local journalists facing harassment, surveillance, and charges under anti-terror laws, while foreign correspondents are denied access or deported for critical reporting. These measures, aimed at controlling the region’s narrative and projecting normalcy, have drawn widespread criticism from international watchdogs, who warn of increasing suppression of both domestic and foreign media.

headshot
AJR Correspondent Published on: 27 Nov, 2024
Gender Inequity in Sports Reporting: Female Journalists Demand Equality

Gender inequality persists in sports journalism, with female reporters significantly under-represented, as shown by studies revealing that only 5.1% of sports articles are written by women. Advocates call for equal representation, more inclusive hiring practices, and a broader focus on women's sports to challenge stereotypes, improve coverage, and give women a stronger voice in shaping sports narratives.

Akem
Akem Nkwain Published on: 18 Nov, 2024
Challenging the Narrative: Jeremy Scahill on the Need for Adversarial Journalism

Investigative journalist Jeremy Scahill calls for a revival of "adversarial journalism" to reinstate crucial professional and humanitarian values in mainstream Western media, especially regarding the coverage of the Gaza genocide.

Mohammad Zeidan
Mohammad Zeidan Published on: 10 Nov, 2024
Monitoring of Journalistic Malpractices in Gaza Coverage

On this page, the editorial team of the Al Jazeera Journalism Review will collect news published by media institutions about the current war on Gaza that involves disinformation, bias, or professional journalistic standards and its code of ethics.

A picture of the Al Jazeera Media Institute's logo, on a white background.
Al Jazeera Journalism Review Published on: 23 Oct, 2024
A Year of Genocide and Bias: Western Media's Whitewashing of Israel's Ongoing War on Gaza

Major Western media outlets continue to prove that they are a party in the war of narratives, siding with the Israeli occupation. The article explains how these major Western media outlets are still refining their techniques of bias in favor of the occupation, even a year after the genocide in Palestine.

Mohammad Zeidan
Mohammad Zeidan Published on: 9 Oct, 2024
Testimonies of the First Witness of the Sabra & Shatila Massacre

The Sabra and Shatila massacre in 1982 saw over 3,000 unarmed Palestinian refugees brutally killed by Phalangist militias under the facilitation of Israeli forces. As the first journalist to enter the camps, Japanese journalist Ryuichi Hirokawa provides a harrowing first-hand account of the atrocity amid a media blackout. His testimony highlights the power of bearing witness to a war crime and contrasts the past Israeli public outcry with today’s silence over the ongoing genocide in Gaza.

Mei Shigenobu مي شيغينوبو
Mei Shigenobu Published on: 18 Sep, 2024
Journalist Mothers in Gaza: Living the Ordeal Twice

Being a journalist, particularly a female journalist covering the genocide in Palestine without any form of protection, makes practicing journalism nearly impossible. When the journalist is also a mother haunted by the fear of losing her children, working in the field becomes an immense sacrifice.

Amani Shninu
Amani Shninu Published on: 15 Sep, 2024
Anonymous Sources in the New York Times... Covering the War with One Eye

The use of anonymous sources in journalism is considered, within professional and ethical standards, a “last option” for journalists. However, analysis of New York Times data reveals a persistent pattern in the use of “anonymity” to support specific narratives, especially Israeli narratives.

Mohammad Zeidan
Mohammad Zeidan Published on: 8 Sep, 2024
Cameroonian Journalists at the Center of Fighting Illegal Fishing

While the EU’s red card to Cameroon has undeniably tarnished its image, it has paradoxically unlocked the potential of Cameroonian journalists and ignited a movement poised to reshape the future. Through this shared struggle, journalists, scientists, conservationists, storytellers, and government officials have united, paving the way for a new era of ocean advocacy.

Shuimo Trust Dohyee
Shuimo Trust Dohyee Published on: 21 Aug, 2024
The Gaza Journalist and the "Heart and Mind" Struggle

Inside the heart of a Palestinian journalist living in Gaza, there are two personas: one is a human who wants to protect his own life and that of his family, and the other is a journalist committed to safeguarding the lives of the people by holding on to the truth and staying in the field. Between these two extremes, or what journalist Maram Hamid describes as the struggle between the heart and the mind, the Palestinian journalist continues to share a narrative that the occupation intended to keep "away from the camera."

Maram
Maram Humaid Published on: 18 Aug, 2024
Journalists Recount the Final Moments of Ismail Al-Ghoul

Journalists remembering the slain reporter of Al Jazeera in Northern Gaza, Ismail Al Ghoul. "He insisted on continuing his coverage from the northern part of the Gaza Strip, despite the challenges and obstacles he faced. He was arrested and interrogated by the Israeli army, his brother was killed in an Israeli airstrike, and his father passed away during treatment abroad."

Mohammad Abu Don
Mohammad Abu Don Published on: 11 Aug, 2024
Analysis: Media Disinformation and UK Far-Right Riots

Analysis on the impact of media disinformation on public opinion, particularly during UK riots incited by far-right groups. A look at how sensationalist media can directly influence audience behavior, as per the Hypodermic Needle Theory, leading to normalized discrimination and violence. The need for responsible journalism is emphasized to prevent such harmful effects.

Anam Hussain
Anam Hussain Published on: 8 Aug, 2024
Challenges for Female Journalists in Crisis Zones of Cameroon

Testimonies of what female journalists in Cameroon are facing and how they are challenging these difficulties.

Akem
Akem Nkwain Published on: 30 Jul, 2024
From TV Screens to YouTube: The Rise of Exiled Journalists in Pakistan

Pakistani journalists are leveraging YouTube to overcome censorship, connecting with global audiences, and redefining independent reporting in their homeland.

Anam Hussain
Anam Hussain Published on: 28 Jul, 2024
Daughters of Data: African Female Journalists Using Data to Reveal Hidden Truths

A growing network of African women journalists, data scientists, and tech experts is amplifying female voices and highlighting underreported stories across the continent by producing data-driven projects and leveraging digital technologies in storytelling.

Nalova Akua
Nalova Akua Published on: 23 Jul, 2024
Are Podcasts the Future of African Broadcasting?

The surge of podcasts across Africa is a burgeoning trend, encompassing a wide array of themes and subjects, and swiftly expanding across various nations.

Derick Matsengarwodzi
Derick Matsengarwodzi Published on: 11 Jul, 2024
Video Volunteers: How India’s Marginalised Groups Tell Their Own Stories

Video creators like Rohini Pawar and Shabnam Begum have transcended societal challenges by producing influential videos with Video Volunteers, highlighting social issues within marginalized communities. Their work exemplifies the transformative power of storytelling in fostering grassroots change and empowerment across India.

Hanan Zaffa
Hanan Zaffar, Jyoti Thakur Published on: 3 Jul, 2024
Climate Journalism in Vietnam's Censored Landscape

In Vietnam, climate journalists face challenges due to censorship and restrictions on press freedom, making it difficult to report environmental issues accurately. Despite these obstacles, there are still journalists working to cover climate stories creatively and effectively, highlighting the importance of climate journalism in addressing environmental concerns.

AJR Contributor Published on: 26 Jun, 2024
Challenges of Investigating Subculture Stories in Japan as a Foreign Correspondent

Japan's vibrant subcultures and feminist activists challenge the reductive narratives often portrayed in Western media. To understand this dynamic society authentically, journalists must approach their reporting with patience, commitment, and empathy, shedding preconceptions and engaging deeply with the nuances of Japanese culture.

Johann Fleuri
Johann Fleuri Published on: 24 Jun, 2024